Learning long-term dependencies in NARX recurrent neural networks

1996 ◽  
Vol 7 (6) ◽  
pp. 1329-1338 ◽  
Author(s):  
Tsungnan Lin ◽  
B.G. Horne ◽  
P. Tino ◽  
C.L. Giles
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks

Learning long-term dependencies with recurrent neural networks

2008 ◽  
Vol 71 (13-15) ◽  
pp. 2481-2488 ◽  
Author(s):  
Anton Maximilian Schaefer ◽  
Steffen Udluft ◽  
Hans-Georg Zimmermann
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks

scholarly journals Long-Term Traffic Scheduling Based on Stacked Bidirectional Recurrent Neural Networks in Inter-Datacenter Optical Networks

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 182296-182308 ◽  
Author(s):  
Ao Yu ◽  
Hui Yang ◽  
Ting Xu ◽  
Baoguo Yu ◽  
Qiuyan Yao ◽  
...  
Keyword(s):  
Neural Networks ◽  
Optical Networks ◽  
Recurrent Neural Networks ◽  
Traffic Scheduling

scholarly journals Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory

2020 ◽  
Vol 34 (04) ◽  
pp. 4115-4122
Author(s):  
Kyle Helfrich ◽  
Qiang Ye
Keyword(s):  
Neural Networks ◽  
Unit Circle ◽  
Recurrent Neural Networks ◽  
Unit Disc ◽  
Gradient Descent ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Gradient Descent Algorithm

Several variants of recurrent neural networks (RNNs) with orthogonal or unitary recurrent matrices have recently been developed to mitigate the vanishing/exploding gradient problem and to model long-term dependencies of sequences. However, with the eigenvalues of the recurrent matrix on the unit circle, the recurrent state retains all input information which may unnecessarily consume model capacity. In this paper, we address this issue by proposing an architecture that expands upon an orthogonal/unitary RNN with a state that is generated by a recurrent matrix with eigenvalues in the unit disc. Any input to this state dissipates in time and is replaced with new inputs, simulating short-term memory. A gradient descent algorithm is derived for learning such a recurrent matrix. The resulting method, called the Eigenvalue Normalized RNN (ENRNN), is shown to be highly competitive in several experiments.

A Combined Wavelet Transform and Recurrent Neural Networks Scheme for Identification of Hydrocarbon Reservoir Systems From Well Testing Signals

2020 ◽  
Vol 143 (1) ◽  
Author(s):  
Mehrafarin Moghimihanjani ◽  
Behzad Vaferi
Keyword(s):  
Neural Networks ◽  
Wavelet Transform ◽  
Recurrent Neural Networks ◽  
Predictive Accuracy ◽  
Hydrocarbon Reservoirs ◽  
Real Field ◽  
Well Testing ◽  
Hydrocarbon Reservoir ◽  
Reservoir Systems

Abstract Oil and gas are likely the most important sources for producing heat and energy in both domestic and industrial applications. Hydrocarbon reservoirs that contain these fuels are required to be characterized to exploit the maximum amount of their fluids. Well testing analysis is a valuable tool for the characterization of hydrocarbon reservoirs. Handling and analysis of long-term and noise-contaminated well testing signals using the traditional methods is a challenging task. Therefore, in this study, a novel paradigm that combines wavelet transform (WT) and recurrent neural networks (RNN) is proposed for analyzing the long-term well testing signals. The WT not only reduces the dimension of the pressure derivative (PD) signals during feature extraction but it efficiently removes noisy data. The RNN identifies reservoir type and its boundary condition from the extracted features by WT. Results confirmed that the five-level decomposition of the PD signals by the Bior 1.1 filter provides the best features for classification. A two-layer RNN model with nine hidden neurons correctly detects 3202 out of 3298 hydrocarbon reservoir systems. Performance of the proposed approach is checked using smooth, noisy, and real field well testing signals. Moreover, a comparison is done among predictive accuracy of WT-RNN, traditional RNN, conventional multilayer perceptron (MLP) neural networks, and couple WT-MLP approaches. The results confirm that the coupled WT-RNN paradigm is superior to the other considered smart machines.

Sign in / Sign up

Export Citation Format

Share Document