scholarly journals Data‐Driven Super‐Parameterization Using Deep Learning: Experimentation With Multiscale Lorenz 96 Systems and Transfer Learning

2020 ◽  
Vol 12 (11) ◽  
Author(s):  
Ashesh Chattopadhyay ◽  
Adam Subel ◽  
Pedram Hassanzadeh
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Data Driven ◽  
Lorenz 96

scholarly journals Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM

2020 ◽  
Author(s):  
Ashesh Chattopadhyay ◽  
Pedram Hassanzadeh ◽  
Devika Subramanian
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Data Driven ◽  
Small Scale ◽  
Reservoir Computing ◽  
Trajectory Data ◽  
Short Term ◽  
Learning Methods ◽  
Multi Scale ◽  
Lorenz 96

Abstract. In this paper, the performance of three deep learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multi-scale spatio-temporal Lorenz 96 system is examined. The methods are: echo state network (a type of reservoir computing, RC-ESN), deep feed-forward artificial neural network (ANN), and recurrent neural network with long short-term memory (RNN-LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC-ESN substantially outperforms ANN and RNN-LSTM for short-term prediction, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps, equivalent to several Lyapunov timescales. The RNN-LSTM and ANN show some prediction skills as well; RNN-LSTM bests ANN. Furthermore, even after losing the trajectory, data predicted by RC-ESN and RNN-LSTM have probability density functions (PDFs) that closely match the true PDF, even at the tails. The PDF of the data predicted using ANN, however, deviates from the true PDF. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems such as weather/climate are discussed.

scholarly journals A Novel Hybrid Signal Decomposition Technique for Transfer Learning Based Industrial Fault Diagnosis

2021 ◽  
Vol 5 (4) ◽  
pp. 37-53
Author(s):  
Zurana Mehrin Ruhi ◽  
Sigma Jahan ◽  
Jia Uddin
Keyword(s):  
Deep Learning ◽  
Fault Diagnosis ◽  
Transfer Learning ◽  
Industrial Revolution ◽  
State Of The Art ◽  
Signal Decomposition ◽  
Data Driven ◽  
Detection Model ◽  
Mode Decomposition ◽  
To Come

In the fourth industrial revolution, data-driven intelligent fault diagnosis for industrial purposes serves a crucial role. In contemporary times, although deep learning is a popular approach for fault diagnosis, it requires massive amounts of labelled samples for training, which is arduous to come by in the real world. Our contribution to introduce a novel comprehensive intelligent fault detection model using the Case Western Reserve University dataset is divided into two steps. Firstly, a new hybrid signal decomposition methodology is developed comprising Empirical Mode Decomposition and Variational Mode Decomposition to leverage signal information from both processes for effective feature extraction. Secondly, transfer learning with DenseNet121 is employed to alleviate the constraints of deep learning models. Finally, our proposed novel technique surpassed not only previous outcomes but also generated state-of-the-art outcomes represented via the F1 score.

A new dam structural response estimation paradigm powered by deep learning and transfer learning techniques

2021 ◽  
pp. 147592172110097
Author(s):  
Yangtao Li ◽  
Tengfei Bao ◽  
Zhixin Gao ◽  
Xiaosong Shu ◽  
Kang Zhang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Transfer Learning ◽  
Health Monitoring ◽  
Structural Response ◽  
Data Driven ◽  
Structural Health ◽  
Learning Techniques

With the rapid development of information and communication techniques, dam structural health assessment based on data collected from structural health monitoring systems has become a trend. This allows for applying data-driven methods for dam safety analysis. However, data-driven models in most related literature are statistical and shallow machine learning models, which cannot capture the time series patterns or learn from long-term dependencies of dam structural response time series. Furthermore, the effectiveness and applicability of these models are only validated in a small data set and part of monitoring points in a dam structural health monitoring system. To address the problems, this article proposes a new modeling paradigm based on various deep learning and transfer learning techniques. The paradigm utilizes one-dimensional convolutional neural networks to extract the inherent features from dam structural response–related environmental quantity monitoring data. Then bidirectional gated recurrent unit with a self-attention mechanism is used to learn from long-term dependencies, and transfer learning is utilized to transfer knowledge learned from the typical monitoring point to the others. The proposed paradigm integrates the powerful modeling capability of deep learning networks and the flexible transferability of transfer learning. Rather than traditional models that rely on experience for feature selection, the proposed deep learning–based paradigm directly utilizes environmental monitoring time series as inputs to accurately estimate dam structural response changes. A high arch dam in long-term service is selected as the case study, and three monitoring items, including dam displacement, crack opening displacement, and seepage are used as the research objects. The experimental results show that the proposed paradigm outperforms conventional and shallow machine learning–based methods in all 41 tested monitoring points, which indicates that the proposed paradigm is capable of dealing with dam structural response estimation with high accuracy and robustness.

Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals Deep Transfer Learning Based Intrusion Detection System for Electric Vehicular Networks

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4736
Author(s):  
Sk. Tanzir Mehedi ◽  
Adnan Anwar ◽  
Ziaur Rahman ◽  
Kawsar Ahmed
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Intrusion Detection ◽  
Real Time ◽  
Transfer Learning ◽  
Security Requirements ◽  
Detection Accuracy ◽  
Area Network ◽  
Complex Data ◽  
Network Intrusion

The Controller Area Network (CAN) bus works as an important protocol in the real-time In-Vehicle Network (IVN) systems for its simple, suitable, and robust architecture. The risk of IVN devices has still been insecure and vulnerable due to the complex data-intensive architectures which greatly increase the accessibility to unauthorized networks and the possibility of various types of cyberattacks. Therefore, the detection of cyberattacks in IVN devices has become a growing interest. With the rapid development of IVNs and evolving threat types, the traditional machine learning-based IDS has to update to cope with the security requirements of the current environment. Nowadays, the progression of deep learning, deep transfer learning, and its impactful outcome in several areas has guided as an effective solution for network intrusion detection. This manuscript proposes a deep transfer learning-based IDS model for IVN along with improved performance in comparison to several other existing models. The unique contributions include effective attribute selection which is best suited to identify malicious CAN messages and accurately detect the normal and abnormal activities, designing a deep transfer learning-based LeNet model, and evaluating considering real-world data. To this end, an extensive experimental performance evaluation has been conducted. The architecture along with empirical analyses shows that the proposed IDS greatly improves the detection accuracy over the mainstream machine learning, deep learning, and benchmark deep transfer learning models and has demonstrated better performance for real-time IVN security.

Sign in / Sign up

Export Citation Format

Share Document