scholarly journals SISTEM PERINGKAS OTOMATIS ABSTRAKTIF DENGAN MENGGUNAKAN RECURRENT NEURAL NETWORK

2018 ◽  
Vol 2 (1) ◽  
pp. 65 ◽  
Author(s):  
Kuncoro Yoko ◽  
Viny Christanti Mawardi ◽  
Janson Hendryli
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Model System ◽  
Text Summarization ◽  
Word Embedding ◽  
System Evaluation ◽  
Data Set ◽  
Hidden States ◽  
Bahasa Indonesia

Abstractive Text Summarization try to creates a shorter version of a text while preserve its meaning. We try to use Recurrent Neural Network (RNN) to create summaries of Bahasa Indonesia text. We get corpus from Detik dan Kompas site news. We used word2vec to create word embedding from our corpus then train our data set with RNN to create a model. This model used to generate news. We search the best model by changing word2vec size and RNN hidden states. We use system evaluation and Q&A Evaluation to evaluate our model. System evaluation showed that model with 6457 data set, 200 word2vec size, and 256 RNN hidden states gives best accuracy for 99.8810%. This model evaluated by Q&A Evaluation. Q&A Evaluation showed that the model gives 46.65% accurary.

scholarly journals Real-time event detection using recurrent neural network in social sensors

2019 ◽  
Vol 15 (6) ◽  
pp. 155014771985649 ◽  
Author(s):  
Van Quan Nguyen ◽  
Tien Nguyen Anh ◽  
Hyung-Jeong Yang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
Event Detection ◽  
Word Embedding ◽  
Series Data ◽  
Accuracy Score ◽  
Data Set ◽  
Size Limitation

We proposed an approach for temporal event detection using deep learning and multi-embedding on a set of text data from social media. First, a convolutional neural network augmented with multiple word-embedding architectures is used as a text classifier for the pre-processing of the input textual data. Second, an event detection model using a recurrent neural network is employed to learn time series data features by extracting temporal information. Recently, convolutional neural networks have been used in natural language processing problems and have obtained excellent results as performing on available embedding vector. In this article, word-embedding features at the embedding layer are combined and fed to convolutional neural network. The proposed method shows no size limitation, supplementation of more embeddings than standard multichannel based approaches, and obtained similar performance (accuracy score) on some benchmark data sets, especially in an imbalanced data set. For event detection, a long short-term memory network is used as a predictor that learns higher level temporal features so as to predict future values. An error distribution estimation model is built to calculate the anomaly score of observation. Events are detected using a window-based method on the anomaly scores.

scholarly journals A Pointer Generator Network Model to Automatic Text Summarization and Headline Generation

2019 ◽  
Vol 8 (5S3) ◽  
pp. 447-451
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Recurrent Neural Network ◽  
Text Summarization ◽  
Daily Mail ◽  
Automatic Text Summarization ◽  
Generator Model ◽  
Abstractive Summarization ◽  
Automatic Text

In a world where information is growing rapidly every single day, we need tools to generate summary and headlines from text which is accurate as well as short and precise. In this paper, we have described a method for generating headlines from article. This is done by using hybrid pointer-generator network with attention distribution and coverage mechanism on article which generates abstractive summarization followed by the application of encoder-decoder recurrent neural network with LSTM unit to generate headlines from the summary. Hybrid pointer generator model helps in removing inaccuracy as well as repetitions. We have used CNN / Daily Mail as our dataset.

scholarly journals Automatic Text Summarization of COVID-19 Research Articles Using Recurrent Neural Networks and Coreference Resolution

2021 ◽  
Author(s):  
Mahsa Afsharizadeh ◽  
Hossein Ebrahimpour-Komleh ◽  
Ayoub Bagheri
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Text Summarization ◽  
Medical Community ◽  
Research Articles ◽  
Coreference Resolution ◽  
Extractive Summarization ◽  
Automatic Text Summarization ◽  
Summarization System ◽  
Automatic Text

Purpose: Pandemic COVID-19 has created an emergency for the medical community. Researchers require extensive study of scientific literature in order to discover drugs and vaccines. In this situation where every minute is valuable to save the lives of hundreds of people, a quick understanding of scientific articles will help the medical community. Automatic text summarization makes this possible. Materials and Methods: In this study, a recurrent neural network-based extractive summarization is proposed. The extractive method identifies the informative parts of the text. Recurrent neural network is very powerful for analyzing sequences such as text. The proposed method has three phases: sentence encoding, sentence ranking, and summary generation. To improve the performance of the summarization system, a coreference resolution procedure is used. Coreference resolution identifies the mentions in the text that refer to the same entity in the real world. This procedure helps to summarization process by discovering the central subject of the text. Results: The proposed method is evaluated on the COVID-19 research articles extracted from the CORD-19 dataset. The results show that the combination of using recurrent neural network and coreference resolution embedding vectors improves the performance of the summarization system. The Proposed method by achieving the value of ROUGE1-recall 0.53 demonstrates the improvement of summarization performance by using coreference resolution embedding vectors in the RNN-based summarization system. Conclusion: In this study, coreference information is stored in the form of coreference embedding vectors. Jointly use of recurrent neural network and coreference resolution results in an efficient summarization system.

scholarly journals Mining e-cigarette adverse events in social media using Bi-LSTM recurrent neural network with word embedding representation

2017 ◽  
Vol 25 (1) ◽  
pp. 72-80 ◽  
Author(s):  
Jiaheng Xie ◽  
Xiao Liu ◽  
Daniel Dajun Zeng
Keyword(s):  
Neural Network ◽  
Social Media ◽  
Adverse Events ◽  
Recurrent Neural Network ◽  
Deep Neural Network ◽  
Word Embedding ◽  
Named Entity ◽  
Text Input ◽  
Safety Surveillance ◽  
F Measure

Abstract Objective Recent years have seen increased worldwide popularity of e-cigarette use. However, the risks of e-cigarettes are underexamined. Most e-cigarette adverse event studies have achieved low detection rates due to limited subject sample sizes in the experiments and surveys. Social media provides a large data repository of consumers’ e-cigarette feedback and experiences, which are useful for e-cigarette safety surveillance. However, it is difficult to automatically interpret the informal and nontechnical consumer vocabulary about e-cigarettes in social media. This issue hinders the use of social media content for e-cigarette safety surveillance. Recent developments in deep neural network methods have shown promise for named entity extraction from noisy text. Motivated by these observations, we aimed to design a deep neural network approach to extract e-cigarette safety information in social media. Methods Our deep neural language model utilizes word embedding as the representation of text input and recognizes named entity types with the state-of-the-art Bidirectional Long Short-Term Memory (Bi-LSTM) Recurrent Neural Network. Results Our Bi-LSTM model achieved the best performance compared to 3 baseline models, with a precision of 94.10%, a recall of 91.80%, and an F-measure of 92.94%. We identified 1591 unique adverse events and 9930 unique e-cigarette components (ie, chemicals, flavors, and devices) from our research testbed. Conclusion Although the conditional random field baseline model had slightly better precision than our approach, our Bi-LSTM model achieved much higher recall, resulting in the best F-measure. Our method can be generalized to extract medical concepts from social media for other medical applications.

scholarly journals Learning Dynamic Factors to Improve the Accuracy of Bus Arrival Time Prediction via a Recurrent Neural Network

2019 ◽  
Vol 11 (12) ◽  
pp. 247
Author(s):  
Xin Zhou ◽  
Peixin Dong ◽  
Jianping Xing ◽  
Peijia Sun
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Public Transportation ◽  
Prediction Accuracy ◽  
Arrival Time ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Support Vector ◽  
Data Set ◽  
Dynamic Factors

Accurate prediction of bus arrival times is a challenging problem in the public transportation field. Previous studies have shown that to improve prediction accuracy, more heterogeneous measurements provide better results. So what other factors should be added into the prediction model? Traditional prediction methods mainly use the arrival time and the distance between stations, but do not make full use of dynamic factors such as passenger number, dwell time, bus driving efficiency, etc. We propose a novel approach that takes full advantage of dynamic factors. Our approach is based on a Recurrent Neural Network (RNN). The experimental results indicate that a variety of prediction algorithms (such as Support Vector Machine, Kalman filter, Multilayer Perceptron, and RNN) have significantly improved performance after using dynamic factors. Further, we introduce RNN with an attention mechanism to adaptively select the most relevant input factors. Experiments demonstrate that the prediction accuracy of RNN with an attention mechanism is better than RNN with no attention mechanism when there are heterogeneous input factors. The experimental results show the superior performances of our approach on the data set provided by Jinan Public Transportation Corporation.

scholarly journals Identification of a Typical CSTR Using Optimal Focused Time Lagged Recurrent Neural Network Model with Gamma Memory Filter

2009 ◽  
Vol 2009 ◽  
pp. 1-7
Author(s):  
S. N. Naikwad ◽  
S. V. Dudul
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Recurrent Neural Network ◽  
Neural Network Model ◽  
Learning Rule ◽  
Continuous Stirred Tank Reactor ◽  
Back Propagation Algorithm ◽  
Data Set ◽  
Highly Nonlinear ◽  
Time Lagged

A focused time lagged recurrent neural network (FTLR NN) with gamma memory filter is designed to learn the subtle complex dynamics of a typical CSTR process. Continuous stirred tank reactor exhibits complex nonlinear operations where reaction is exothermic. It is noticed from literature review that process control of CSTR using neuro-fuzzy systems was attempted by many, but optimal neural network model for identification of CSTR process is not yet available. As CSTR process includes temporal relationship in the input-output mappings, time lagged recurrent neural network is particularly used for identification purpose. The standard back propagation algorithm with momentum term has been proposed in this model. The various parameters like number of processing elements, number of hidden layers, training and testing percentage, learning rule and transfer function in hidden and output layer are investigated on the basis of performance measures like MSE, NMSE, and correlation coefficient on testing data set. Finally effects of different norms are tested along with variation in gamma memory filter. It is demonstrated that dynamic NN model has a remarkable system identification capability for the problems considered in this paper. Thus FTLR NN with gamma memory filter can be used to learn underlying highly nonlinear dynamics of the system, which is a major contribution of this paper.

scholarly journals Ekstraksi Relasi Antar Entitas di Bahasa Indonesia Menggunakan Neural Network

2021 ◽  
Vol 3 (02) ◽  
pp. 49-54
Author(s):  
Ananta Tio Putra ◽  
Eunike Kardinata ◽  
Hartarto Junaedi ◽  
Francisca Chandra ◽  
Joan Santoso
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Relation Extraction ◽  
Support Vector ◽  
Short Term ◽  
Tree Kernel ◽  
Kernel Support Vector Machine ◽  
Long Short Term Memory ◽  
Bahasa Indonesia

Dengan perkembangan zaman yang begitu pesat, berdampak pada perkembangan data pula. Salah satu bentuk data yang paling banyak saat ini berupa data tekstual seperti artikel sederhana maupun dokumen lain yang terdapat di internet. Agar data tekstual tersebut dapat dimengerti dan dimanfaatkan dengan baik oleh manusia, maka perlu di proses dan disederhanakan agar menjadi informasi yang ringkas dan jelas. Oleh karena itu, semakin berkembang pula penelitian dalam bidang Information Extraction (IE) dan salah satu contoh penelitian di IE adalah Relation Extraction (RE). Penelitian RE sudah banyak dilakukan terutama pada Bahasa Inggris dimana resourcenya sudah termasuk banyak. Metode yang digunakan pun bermacam-macam seperti kernel, tree kernel, support vector machine, long short-term memory, convulution recurrent neural network, dan lain sebagainya. Pada penelitian kali ini adalah penelitian RE pada Bahasa Indonesia dengan menggunakan metode convulution recurrent neural network yang sudah dipergunakan untuk RE Bahasa Inggris. Dataset yang digunakan pada penelitian ini adalah dataset Bahasa Indonesia yang berasal dari file xml wikipedia. File xml wikipedia ini kemudian diproses sehingga menghasilkan dataset seperti yang digunakan pada CRNN dalam Bahasa inggris yaitu dalam format SemEval-2 Task 8. Uji coba dilakukan dengan berbagai macam perbandingan data training dan testing yaitu 80:20, 70:30, dan 60:40. Selain itu, parameter pooling untuk CRNN yang digunakan ada dua macam yaitu ‘att’ dan ‘max’. Dari uji coba yang dilakukan, hasil yang didapatkan adalah bervariasi mulai dari mendekati maupun lebih baik bila dibandingkan dengan CRNN dengan menggunakan dataset Bahasa inggris sehingga dapat disimpulkan bahwa dengan CRNN ini bisa digunakan untuk proses RE pada Bahasa Indonesia apabila dataset yang digunakan sesuai dengan penelitian sebelumnya.

scholarly journals A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction

2017 ◽  
Author(s):  
Yao Qin ◽  
Dongjin Song ◽  
Haifeng Chen ◽  
Wei Cheng ◽  
Guofei Jiang ◽  
...  
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Recurrent Neural Network ◽  
Empirical Studies ◽  
Time Series Prediction ◽  
Attention Mechanism ◽  
Time Step ◽  
Narx Models ◽  
Dual Stage ◽  
Hidden States

The Nonlinear autoregressive exogenous (NARX) model, which predicts the current value of a time series based upon its previous values as well as the current and past values of multiple driving (exogenous) series, has been studied for decades. Despite the fact that various NARX models have been developed, few of them can capture the long-term temporal dependencies appropriately and select the relevant driving series to make predictions. In this paper, we propose a dual-stage attention-based recurrent neural network (DA-RNN) to address these two issues. In the first stage, we introduce an input attention mechanism to adaptively extract relevant driving series (a.k.a., input features) at each time step by referring to the previous encoder hidden state. In the second stage, we use a temporal attention mechanism to select relevant encoder hidden states across all time steps. With this dual-stage attention scheme, our model can not only make predictions effectively, but can also be easily interpreted. Thorough empirical studies based upon the SML 2010 dataset and the NASDAQ 100 Stock dataset demonstrate that the DA-RNN can outperform state-of-the-art methods for time series prediction.

Sign in / Sign up

Export Citation Format

Share Document