scholarly journals Deep Learning with Long Short Term Memory Based Sequence-to-Sequence Model for Rainfall-Runoff Simulation

Water ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 437
Author(s):  
Heechan Han ◽  
Changhyun Choi ◽  
Jaewon Jung ◽  
Hung Soo Kim
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Support Vector ◽  
Sequence Structure ◽  
Short Term ◽  
Term Memory ◽  
Runoff Forecasting ◽  
Proposed Model ◽  
Runoff Prediction ◽  
Long Short Term Memory

Accurate runoff prediction is one of the important tasks in various fields such as agriculture, hydrology, and environmental studies. Recently, with massive improvements of computational system and hardware, the deep learning-based approach has recently been applied for more accurate runoff prediction. In this study, the long short-term memory model with sequence-to-sequence structure was applied for hourly runoff predictions from 2015 to 2019 in the Russian River basin, California, USA. The proposed model was used to predict hourly runoff with lead time of 1–6 h using runoff data observed at upstream stations. The model was evaluated in terms of event-based performance using the statistical metrics including root mean square error, Nash-Sutcliffe Efficiency, peak runoff error, and peak time error. The results show that proposed model outperforms support vector machine and conventional long short-term memory models. In addition, the model has the best predictive ability for runoff events, which means that it can be effective for developing short-term flood forecasting and warning systems. The results of this study demonstrate that the deep learning-based approach for hourly runoff forecasting has high predictive power and sequence-to-sequence structure is effective method to improve the prediction results.

A semiautomatic annotation approach for sentiment analysis

2021 ◽  
pp. 016555152110065
Author(s):  
Rahma Alahmary ◽  
Hmood Al-Dossari
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Short Term Memory ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Annotation Process ◽  
Learning Classifiers ◽  
Long Short Term Memory

Sentiment analysis (SA) aims to extract users’ opinions automatically from their posts and comments. Almost all prior works have used machine learning algorithms. Recently, SA research has shown promising performance in using the deep learning approach. However, deep learning is greedy and requires large datasets to learn, so it takes more time for data annotation. In this research, we proposed a semiautomatic approach using Naïve Bayes (NB) to annotate a new dataset in order to reduce the human effort and time spent on the annotation process. We created a dataset for the purpose of training and testing the classifier by collecting Saudi dialect tweets. The dataset produced from the semiautomatic model was then used to train and test deep learning classifiers to perform Saudi dialect SA. The accuracy achieved by the NB classifier was 83%. The trained semiautomatic model was used to annotate the new dataset before it was fed into the deep learning classifiers. The three deep learning classifiers tested in this research were convolutional neural network (CNN), long short-term memory (LSTM) and bidirectional long short-term memory (Bi-LSTM). Support vector machine (SVM) was used as the baseline for comparison. Overall, the performance of the deep learning classifiers exceeded that of SVM. The results showed that CNN reported the highest performance. On one hand, the performance of Bi-LSTM was higher than that of LSTM and SVM, and, on the other hand, the performance of LSTM was higher than that of SVM. The proposed semiautomatic annotation approach is usable and promising to increase speed and save time and effort in the annotation process.

scholarly journals Forecasting Day-Ahead Hourly Photovoltaic Power Generation Using Convolutional Self-Attention Based Long Short-Term Memory

Energies ◽  
2020 ◽  
Vol 13 (15) ◽  
pp. 4017 ◽  
Author(s):  
Dukhwan Yu ◽  
Wonik Choi ◽  
Myoungsoo Kim ◽  
Ling Liu
Keyword(s):  
Power Generation ◽  
Deep Learning ◽  
Prediction Accuracy ◽  
Short Term Memory ◽  
Local Context ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Long Short Term Memory ◽  
Pv Power Generation

The problem of Photovoltaic (PV) power generation forecasting is becoming crucial as the penetration level of Distributed Energy Resources (DERs) increases in microgrids and Virtual Power Plants (VPPs). In order to improve the stability of power systems, a fair amount of research has been proposed for increasing prediction performance in practical environments through statistical, machine learning, deep learning, and hybrid approaches. Despite these efforts, the problem of forecasting PV power generation remains to be challenging in power system operations since existing methods show limited accuracy and thus are not sufficiently practical enough to be widely deployed. Many existing methods using long historical data suffer from the long-term dependency problem and are not able to produce high prediction accuracy due to their failure to fully utilize all features of long sequence inputs. To address this problem, we propose a deep learning-based PV power generation forecasting model called Convolutional Self-Attention based Long Short-Term Memory (LSTM). By using the convolutional self-attention mechanism, we can significantly improve prediction accuracy by capturing the local context of the data and generating keys and queries that fit the local context. To validate the applicability of the proposed model, we conduct extensive experiments on both PV power generation forecasting using a real world dataset and power consumption forecasting. The experimental results of power generation forecasting using the real world datasets show that the MAPEs of the proposed model are much lower, in fact by 7.7%, 6%, 3.9% compared to the Deep Neural Network (DNN), LSTM and LSTM with the canonical self-attention, respectively. As for power consumption forecasting, the proposed model exhibits 32%, 17% and 44% lower Mean Absolute Percentage Error (MAPE) than the DNN, LSTM and LSTM with the canonical self-attention, respectively.

CONVOLUTIONAL LONG-SHORT TERM MEMORY NETWORKS MODEL FOR LONG DURATION EEG SIGNAL CLASSIFICATION

2019 ◽  
Vol 19 (01) ◽  
pp. 1940005 ◽  
Author(s):  
ULAS BARAN BALOGLU ◽  
ÖZAL YILDIRIM
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Learning Algorithms ◽  
Eeg Signals ◽  
Short Term ◽  
Term Memory ◽  
Long Duration ◽  
Proposed Model ◽  
Clinical Scenarios ◽  
Long Short Term Memory

Background and objective: Deep learning structures have recently achieved remarkable success in the field of machine learning. Convolutional neural networks (CNN) in image processing and long-short term memory (LSTM) in the time-series analysis are commonly used deep learning algorithms. Healthcare applications of deep learning algorithms provide important contributions for computer-aided diagnosis research. In this study, convolutional long-short term memory (CLSTM) network was used for automatic classification of EEG signals and automatic seizure detection. Methods: A new nine-layer deep network model consisting of convolutional and LSTM layers was designed. The signals processed in the convolutional layers were given as an input to the LSTM network whose outputs were processed in densely connected neural network layers. The EEG data is appropriate for a model having 1-D convolution layers. A bidirectional model was employed in the LSTM layer. Results: Bonn University EEG database with five different datasets was used for experimental studies. In this database, each dataset contains 23.6[Formula: see text]s duration 100 single channel EEG segments which consist of 4097 dimensional samples (173.61[Formula: see text]Hz). Eight two-class and three three-class clinical scenarios were examined. When the experimental results were evaluated, it was seen that the proposed model had high accuracy on both binary and ternary classification tasks. Conclusions: The proposed end-to-end learning structure showed a good performance without using any hand-crafted feature extraction or shallow classifiers to detect the seizures. The model does not require filtering, and also automatically learns to filter the input as well. As a result, the proposed model can process long duration EEG signals without applying segmentation, and can detect epileptic seizures automatically by using the correlation of ictal and interictal signals of raw data.

scholarly journals Previsão de casos de dengue através de Machine Learning e Deep Learning: uma revisão sistemática

2021 ◽  
Vol 10 (11) ◽  
pp. e33101119347
Author(s):  
Ewethon Dyego de Araujo Batista ◽  
Wellington Candeia de Araújo ◽  
Romeryto Vieira Lira ◽  
Laryssa Izabel de Araujo Batista
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Short Term Memory ◽  
Mean Absolute Error ◽  
Absolute Error ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
De Medicina

Introdução: a dengue é uma arbovirose causada pelo vírus DENV e transmitida para o homem através do mosquito Aedes aegypti. Atualmente, não existe uma vacina eficaz para combater todas as sorologias do vírus. Diante disso, o combate à doença se volta para medidas preventivas contra a proliferação do mosquito. Os pesquisadores estão utilizando Machine Learning (ML) e Deep Learning (DL) como ferramentas para prever casos de dengue e ajudar os governantes nesse combate. Objetivo: identificar quais técnicas e abordagens de ML e de DL estão sendo utilizadas na previsão de dengue. Métodos: revisão sistemática realizada nas bases das áreas de Medicina e de Computação com intuito de responder as perguntas de pesquisa: é possível realizar previsões de casos de dengue através de técnicas de ML e de DL, quais técnicas são utilizadas, onde os estudos estão sendo realizados, como e quais dados estão sendo utilizados? Resultados: após realizar as buscas, aplicar os critérios de inclusão, exclusão e leitura aprofundada, 14 artigos foram aprovados. As técnicas Random Forest (RF), Support Vector Regression (SVR), e Long Short-Term Memory (LSTM) estão presentes em 85% dos trabalhos. Em relação aos dados, na maioria, foram utilizados 10 anos de dados históricos da doença e informações climáticas. Por fim, a técnica Root Mean Absolute Error (RMSE) foi a preferida para mensurar o erro. Conclusão: a revisão evidenciou a viabilidade da utilização de técnicas de ML e de DL para a previsão de casos de dengue, com baixa taxa de erro e validada através de técnicas estatísticas.

scholarly journals Climate-driven Model Based on Long Short-Term Memory and Bayesian Optimization for Multi-day-ahead Daily Streamflow Forecasting

2021 ◽  
Author(s):  
Yani Lian ◽  
Jungang Luo ◽  
Jingmin Wang ◽  
Ganggang Zuo
Keyword(s):  
Climate Change ◽  
Short Term Memory ◽  
Bayesian Optimization ◽  
Support Vector ◽  
Streamflow Forecasting ◽  
Short Term ◽  
Term Memory ◽  
Daily Streamflow ◽  
Runoff Forecasting ◽  
Long Short Term Memory

Abstract Many previous studies have developed decomposition and ensemble models to improve runoff forecasting performance. However, these decomposition-based models usually introduce large decomposition errors into the modeling process. Since the variation in runoff time series is greatly driven by climate change, many previous studies considering climate change focused on only rainfall-runoff modeling, with few meteorological factors as input. Therefore, a climate-driven streamflow forecasting (CDSF) framework was proposed to improve the runoff forecasting accuracy. This framework is realized using principal component analysis (PCA), long short-term memory (LSTM) and Bayesian optimization (BO) referred to as PCA-LSTM-BO. To validate the effectiveness and superiority of the PCA-LSTM-BO method with which one autoregressive LSTM model and two other CDSF models based on PCA, BO, and either support vector regression (SVR) or, gradient boosting regression trees (GBRT), namely, PCA-SVR-BO and PCA-GBRT-BO, respectively, were compared. A generalization performance index based on the Nash-Sutcliffe efficiency (NSE), called the GI(NSE) value, is proposed to evaluate the generalizability of the model. The results show that (1) the proposed model is significantly better than the other benchmark models in terms of the mean square error (MSE<=185.782), NSE>=0.819, and GI(NSE) <=0.223 for all the forecasting scenarios; (2) the PCA in the CDSF framework can improve the forecasting capacity and generalizability; (3) the CDSF framework is superior to the autoregressive LSTM models for all the forecasting scenarios; and (4) the GI(NSE) value is demonstrated to be effective in selecting the optimal model with a better generalizability.

scholarly journals Deepfake tweets classification using stacked Bi-LSTM and words embedding

2021 ◽  
Vol 7 ◽  
pp. e745
Author(s):  
Vaibhav Rupapara ◽  
Furqan Rustam ◽  
Aashir Amaar ◽  
Patrick Bernard Washington ◽  
Ernesto Lee ◽  
...  
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Support Vector ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Tree Classifier ◽  
Long Short Term Memory ◽  
Digital Manipulation ◽  
Lstm Network

The spread of altered media in the form of fake videos, audios, and images, has been largely increased over the past few years. Advanced digital manipulation tools and techniques make it easier to generate fake content and post it on social media. In addition, tweets with deep fake content make their way to social platforms. The polarity of such tweets is significant to determine the sentiment of people about deep fakes. This paper presents a deep learning model to predict the polarity of deep fake tweets. For this purpose, a stacked bi-directional long short-term memory (SBi-LSTM) network is proposed to classify the sentiment of deep fake tweets. Several well-known machine learning classifiers are investigated as well such as support vector machine, logistic regression, Gaussian Naive Bayes, extra tree classifier, and AdaBoost classifier. These classifiers are utilized with term frequency-inverse document frequency and a bag of words feature extraction approaches. Besides, the performance of deep learning models is analyzed including long short-term memory network, gated recurrent unit, bi-direction LSTM, and convolutional neural network+LSTM. Experimental results indicate that the proposed SBi-LSTM outperforms both machine and deep learning models and achieves an accuracy of 0.92.

scholarly journals Non-Intrusive Load Identification Method Based on Improved Long Short Term Memory Network

Energies ◽  
2021 ◽  
Vol 14 (3) ◽  
pp. 684
Author(s):  
Jiateng Song ◽  
Hongbin Wang ◽  
Mingxing Du ◽  
Lei Peng ◽  
Shuai Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Rapid Development ◽  
Series Data ◽  
Load Identification ◽  
Short Term ◽  
Term Memory ◽  
Identification Method ◽  
Proposed Model ◽  
Long Short Term Memory

Non-intrusive load monitoring (NILM) is an important research direction and development goal on the distribution side of smart grid, which can significantly improve the timeliness of demand side response and users’ awareness of load. Due to rapid development, deep learning becomes an effective way to optimize NILM. In this paper, we propose a novel load identification method based on long short term memory (LSTM) on deep learning. Sequence-to-point (seq2point) learning is introduced into LSTM. The innovative combination of the LSTM and the seq2point brings their respective advantages together, so that the proposed model can accurately identify the load in process of time series data. In this paper, we proved the feature of reducing identification error in the experimental data, from three datasets, UK-DALE dataset, REDD dataset, and REFIT dataset. In terms of mean absolute error (MAE), the three datasets have increased by 15%, 14%, and 18% respectively; in terms of normalized signal aggregate error (SAE), the three datasets have increased by 21%, 24%, and 30% respectively. Compared with the existing models, the proposed model has better accuracy and generalization in identifying three open source datasets.

scholarly journals A COMBINED DEEP LEARNING MODEL FOR PERSIAN SENTIMENT ANALYSIS

2019 ◽  
Vol 20 (1) ◽  
pp. 129-139 ◽  
Author(s):  
Zahra Bokaee Nezhad ◽  
Mohammad Ali Deihimi
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Long Short Term Memory ◽  
Deep Learning Model

With increasing members in social media sites today, people tend to share their views about everything online. It is a convenient way to convey their messages to end users on a specific subject. Sentiment Analysis is a subfield of Natural Language Processing (NLP) that refers to the identification of users’ opinions toward specific topics. It is used in several fields such as marketing, customer services, etc. However, limited works have been done on Persian Sentiment Analysis. On the other hand, deep learning has recently become popular because of its successful role in several Natural Language Processing tasks. The objective of this paper is to propose a novel hybrid deep learning architecture for Persian Sentiment Analysis. According to the proposed model, local features are extracted by Convolutional Neural Networks (CNN) and long-term dependencies are learned by Long Short Term Memory (LSTM). Therefore, the model can harness both CNN's and LSTM's abilities. Furthermore, Word2vec is used for word representation as an unsupervised learning step. To the best of our knowledge, this is the first attempt where a hybrid deep learning model is used for Persian Sentiment Analysis. We evaluate the model on a Persian dataset that is introduced in this study. The experimental results show the effectiveness of the proposed model with an accuracy of 85%. ABSTRAK: Hari ini dengan ahli yang semakin meningkat di laman media sosial, orang cenderung untuk berkongsi pandangan mereka tentang segala-galanya dalam talian. Ini adalah cara mudah untuk menyampaikan mesej mereka kepada pengguna akhir mengenai subjek tertentu. Analisis Sentimen adalah subfield Pemprosesan Bahasa Semula Jadi yang merujuk kepada pengenalan pendapat pengguna ke arah topik tertentu. Ia digunakan dalam beberapa bidang seperti pemasaran, perkhidmatan pelanggan, dan sebagainya. Walau bagaimanapun, kerja-kerja terhad telah dilakukan ke atas Analisis Sentimen Parsi. Sebaliknya, pembelajaran mendalam baru menjadi popular kerana peranannya yang berjaya dalam beberapa tugas Pemprosesan Bahasa Asli (NLP). Objektif makalah ini adalah mencadangkan senibina pembelajaran hibrid yang baru dalam Analisis Sentimen Parsi. Menurut model yang dicadangkan, ciri-ciri tempatan ditangkap oleh Rangkaian Neural Convolutional (CNN) dan ketergantungan jangka panjang dipelajari oleh Long Short Term Memory (LSTM). Oleh itu, model boleh memanfaatkan kebolehan CNN dan LSTM. Selain itu, Word2vec digunakan untuk perwakilan perkataan sebagai langkah pembelajaran tanpa pengawasan. Untuk pengetahuan yang terbaik, ini adalah percubaan pertama di mana model pembelajaran mendalam hibrid digunakan untuk Analisis Sentimen Persia. Kami menilai model pada dataset Persia yang memperkenalkan dalam kajian ini. Keputusan eksperimen menunjukkan keberkesanan model yang dicadangkan dengan ketepatan 85%.

scholarly journals Entity-Extraction Using Hybrid Deep-Learning Approach for Hindi text

2021 ◽  
Vol 15 (3) ◽  
pp. 1-11
Author(s):  
Richa Sharma ◽  
Sudha Morwal ◽  
Basant Agarwal
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Research Work ◽  
Entity Recognition ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Long Short Term Memory

This article presents a neural network-based approach to develop named entity recognition for Hindi text. In this paper, the authors propose a deep learning architecture based on convolutional neural network (CNN) and bi-directional long short-term memory (Bi-LSTM) neural network. Skip-gram approach of word2vec model is used in the proposed model to generate word vectors. In this research work, several deep learning models have been developed and evaluated as baseline systems such as recurrent neural network (RNN), long short-term memory (LSTM), Bi-LSTM. Furthermore, these baseline systems are promoted to a proposed model with the integration of CNN and conditional random field (CRF) layers. After a comparative analysis of results, it is verified that the performance of the proposed model (i.e., Bi-LSTM-CNN-CRF) is impressive. The proposed system achieves 61% precision, 56% recall, and 58% F-measure.

scholarly journals Container Volume Prediction Using Time-Series Decomposition with a Long Short-Term Memory Models

2021 ◽  
Vol 11 (19) ◽  
pp. 8995
Author(s):  
Eunju Lee ◽  
Dohee Kim ◽  
Hyerim Bae
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Prediction Models ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
External Variables ◽  
Long Short Term Memory ◽  
Better Than

The purpose of this study is to improve the prediction of container volumes in Busan ports by applying external variables and time-series data decomposition methods to deep learning prediction models. Previous studies on container volume forecasting were based on traditional statistical methodologies, such as ARIMA, SARIMA, and regression. However, these methods do not explain the complexity and variability of data caused by changes in the external environment, such as the global financial crisis and economic fluctuations. Deep learning can explore the inherent patterns of data and analyze the characteristics (time series, external environmental variables, and outliers); hence, the accuracy of deep learning-based volume prediction models is better than that of traditional models. However, this does not include the study of overall trends (upward, steady, or downward). In this study, a novel deep learning prediction model is proposed that combines prediction and trend identification of container volume. The proposed model explores external variables that are related to container volume, combining port volume time-series decomposition with external variables and deep learning-based multivariate long short-term memory (LSTM) prediction. The results indicate that the proposed model performs better than the traditional LSTM model and follows the trend simultaneously.

Sign in / Sign up

Export Citation Format

Share Document