scholarly journals Multimedia Data Modelling Using Multidimensional Recurrent Neural Networks

Symmetry ◽  
2018 ◽  
Vol 10 (9) ◽  
pp. 370
Author(s):  
Zhen He ◽  
Shaobing Gao ◽  
Liang Xiao ◽  
Daxue Liu ◽  
Hangen He
Keyword(s):  
Short Term Memory ◽  
Multimedia Data ◽  
Cross Layer ◽  
Data Modelling ◽  
Short Term ◽  
Time Step ◽  
Machine Learning Approach ◽  
Long Short Term Memory ◽  
Hidden States ◽  
Text Images

Modelling the multimedia data such as text, images, or videos usually involves the analysis, prediction, or reconstruction of them. The recurrent neural network (RNN) is a powerful machine learning approach to modelling these data in a recursive way. As a variant, the long short-term memory (LSTM) extends the RNN with the ability to remember information for longer. Whilst one can increase the capacity of LSTM by widening or adding layers, additional parameters and runtime are usually required, which could make learning harder. We therefore propose a Tensor LSTM where the hidden states are tensorised as multidimensional arrays (tensors) and updated through a cross-layer convolution. As parameters are spatially shared within the tensor, we can efficiently widen the model without extra parameters by increasing the tensorised size; as deep computations of each time step are absorbed by temporal computations of the time series, we can implicitly deepen the model with little extra runtime by delaying the output. We show by experiments that our model is well-suited for various multimedia data modelling tasks, including text generation, text calculation, image classification, and video prediction.

scholarly journals Near Real-Time Global Solar Radiation Forecasting at Multiple Time-Step Horizons Using the Long Short-Term Memory Network

Energies ◽  
2020 ◽  
Vol 13 (14) ◽  
pp. 3517 ◽  
Author(s):  
Anh Ngoc-Lan Huynh ◽  
Ravinesh C. Deo ◽  
Duc-Anh An-Vo ◽  
Mumtaz Ali ◽  
Nawin Raj ◽  
...  
Keyword(s):  
Deep Learning ◽  
Solar Radiation ◽  
Short Term Memory ◽  
Global Solar Radiation ◽  
Multiple Time ◽  
Short Term ◽  
Time Step ◽  
Term Memory ◽  
Multiple Time Step ◽  
Long Short Term Memory

This paper aims to develop the long short-term memory (LSTM) network modelling strategy based on deep learning principles, tailored for the very short-term, near-real-time global solar radiation (GSR) forecasting. To build the prescribed LSTM model, the partial autocorrelation function is applied to the high resolution, 1 min scaled solar radiation dataset that generates statistically significant lagged predictor variables describing the antecedent behaviour of GSR. The LSTM algorithm is adopted to capture the short- and the long-term dependencies within the GSR data series patterns to accurately predict the future GSR at 1, 5, 10, 15, and 30 min forecasting horizons. This objective model is benchmarked at a solar energy resource rich study site (Bac-Ninh, Vietnam) against the competing counterpart methods employing other deep learning, a statistical model, a single hidden layer and a machine learning-based model. The LSTM model generates satisfactory predictions at multiple-time step horizons, achieving a correlation coefficient exceeding 0.90, outperforming all of the counterparts. In accordance with robust statistical metrics and visual analysis of all tested data, the study ascertains the practicality of the proposed LSTM approach to generate reliable GSR forecasts. The Diebold–Mariano statistic test also shows LSTM outperforms the counterparts in most cases. The study confirms the practical utility of LSTM in renewable energy studies, and broadly in energy-monitoring devices tailored for other energy variables (e.g., hydro and wind energy).

scholarly journals Deep Learning for Subtyping and Prediction of Diseases: Long-Short Term Memory

2021 ◽  
Author(s):  
Hayrettin Okut
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Short Term ◽  
Time Step ◽  
Term Memory ◽  
The Neural Network ◽  
Long Short Term Memory ◽  
Sequential Information ◽  
Hidden Layer

The long short-term memory neural network (LSTM) is a type of recurrent neural network (RNN). During the training of RNN architecture, sequential information is used and travels through the neural network from input vector to the output neurons, while the error is calculated and propagated back through the network to update the network parameters. Information in these networks incorporates loops into the hidden layer. Loops allow information to flow multi-directionally so that the hidden state signifies past information held at a given time step. Consequently, the output is dependent on the previous predictions which are already known. However, RNNs have limited capacity to bridge more than a certain number of steps. Mainly this is due to the vanishing of gradients which causes the predictions to capture the short-term dependencies as information from earlier steps decays. As more layers in RNN containing activation functions are added, the gradient of the loss function approaches zero. The LSTM neural networks (LSTM-ANNs) enable learning long-term dependencies. LSTM introduces a memory unit and gate mechanism to enable capture of the long dependencies in a sequence. Therefore, LSTM networks can selectively remember or forget information and are capable of learn thousands timesteps by structures called cell states and three gates.

scholarly journals Detection of Atrial Fibrillation Using a Machine Learning Approach

Information ◽  
2020 ◽  
Vol 11 (12) ◽  
pp. 549
Author(s):  
Sidrah Liaqat ◽  
Kia Dashtipour ◽  
Adnan Zahid ◽  
Khaled Assaleh ◽  
Kamran Arshad ◽  
...  
Keyword(s):  
Machine Learning ◽  
Atrial Fibrillation ◽  
Deep Learning ◽  
Short Term Memory ◽  
Learning Algorithms ◽  
Experimental Results ◽  
Short Term ◽  
Machine Learning Approach ◽  
Long Short Term Memory ◽  
Electrocardiogram Ecg

The atrial fibrillation (AF) is one of the most well-known cardiac arrhythmias in clinical practice, with a prevalence of 1–2% in the community, which can increase the risk of stroke and myocardial infarction. The detection of AF electrocardiogram (ECG) can improve the early detection of diagnosis. In this paper, we have further developed a framework for processing the ECG signal in order to determine the AF episodes. We have implemented machine learning and deep learning algorithms to detect AF. Moreover, the experimental results show that better performance can be achieved with long short-term memory (LSTM) as compared to other algorithms. The initial experimental results illustrate that the deep learning algorithms, such as LSTM and convolutional neural network (CNN), achieved better performance (10%) as compared to machine learning classifiers, such as support vectors, logistic regression, etc. This preliminary work can help clinicians in AF detection with high accuracy and less probability of errors, which can ultimately result in reduction in fatality rate.

scholarly journals Research on power demand forecasting based on attention mechanism and deep learning network

2022 ◽  
Vol 355 ◽  
pp. 02022
Author(s):  
Chenglong Zhang ◽  
Li Yao ◽  
Jinjin Zhang ◽  
Junyong Wu ◽  
Baoguo Shan ◽  
...  
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Demand Forecasting ◽  
Attention Mechanism ◽  
Power Demand ◽  
Short Term ◽  
Time Step ◽  
Long Short Term Memory ◽  
Deep Learning Network ◽  
Deep Learning Model

Combining actual conditions, power demand forecasting is affected by various uncertain factors such as meteorological factors, economic factors, and diversity of forecasting models, which increase the complexity of forecasting. In response to this problem, taking into account that different time step states will have different effects on the output, the attention mechanism is introduced into the method proposed in this paper, which improves the deep learning model. Improved models of convolutional neural networks (CNN) and long short-term memory (LSTM) that combine the attention mechanism are proposed respectively. Finally, according to the verification results of actual examples, it is proved that the proposed method can obtain a smaller error and the prediction performance are better compared with other models.

Deep Neural Network-Based Crime Prediction Using Twitter Data

2021 ◽  
Vol 11 (1) ◽  
pp. 15-30
Author(s):  
Chamith Sandagiri ◽  
Banage T. G. S. Kumara ◽  
Banujan Kuhaneswaran
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Short Term ◽  
Crime Prediction ◽  
Twitter Data ◽  
Machine Learning Approach ◽  
Long Short Term Memory ◽  
Twitter Users ◽  
Better Than

Crimes have affected the quality of life and economic growth of the country badly. The authors can identify the crime patterns and predict the crimes by detecting and analyzing the historical data. However, some crimes are unregistered and unsolved due to a lack of evidence. Thus, detecting crimes is a still challenging task. Individuals can use social media like Twitter to detect crime-related activities. Because Twitter users sometimes convey messages related to their surrounding environment, this paper proposed a machine learning approach to predict crimes. The proposed framework consists of three modules: data (tweet) collecting, detecting crimes, and predicting crime. Long short-term memory (LSTM) neural network model was used as a proposed approach for crime prediction. Experimental results found that by achieving the highest precision of 82.5%, precision of 86.4%, and recall of 80.4%, the proposed LSTM-based approach worked better than the other approaches.

scholarly journals Comparison of Long Short Term Memory Networks and the Hydrological Model in Runoff Simulation

Water ◽  
2020 ◽  
Vol 12 (1) ◽  
pp. 175 ◽  
Author(s):  
Hongxiang Fan ◽  
Mingliang Jiang ◽  
Ligang Xu ◽  
Hua Zhu ◽  
Junxiang Cheng ◽  
...  
Keyword(s):  
Short Term Memory ◽  
Window Size ◽  
Test Period ◽  
Data Driven ◽  
Meteorological Variables ◽  
Short Term ◽  
Time Step ◽  
Term Memory ◽  
Runoff Modeling ◽  
Long Short Term Memory

Runoff modeling is one of the key challenges in the field of hydrology. Various approaches exist, ranging from physically based over conceptual to fully data driven models. In this paper, we propose a data driven approach using the state-of-the-art Long-Short-Term-Memory (LSTM) network. The proposed model was applied in the Poyang Lake Basin (PYLB) and its performance was compared with an Artificial Neural Network (ANN) and the Soil & Water Assessment Tool (SWAT). We first tested the impacts of the number of previous time step (window size) in simulation accuracy. Results showed that a window in improper large size will dramatically deteriorate the model performance. In terms of PYLB, a window size of 15 days might be appropriate for both accuracy and computational efficiency. We then trained the model with 2 different input datasets, namely, dataset with precipitation only and dataset with all available meteorological variables. Results demonstrate that although LSTM with precipitation data as the only input can achieve desirable results (where the NSE ranged from 0.60 to 0.92 for the test period), the performance can be improved simply by feeding the model with more meteorological variables (where NSE ranged from 0.74 to 0.94 for the test period). Moreover, the comparison results with the ANN and the SWAT showed that the ANN can get comparable performance with the SWAT in most cases whereas the performance of LSTM is much better. The results of this study underline the potential of the LSTM for runoff modeling especially for areas where detailed topographical data are not available.

Sign in / Sign up

Export Citation Format

Share Document