scholarly journals Using Deep Learning Methods to Monitor Non-Observable States in a Building

2020 ◽  
Vol 1 ◽  
pp. 6
Author(s):  
Kristoffer Tangrand ◽  
Bernt Bremdal
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Training Data ◽  
Sensor Data ◽  
University Campus ◽  
Soft Sensors ◽  
Ongoing Research ◽  
Human Inspection

This paper presents results from ongoing research with a goal to use a combination of time series from non-intrusive soft sensors and deep recurrent neural networks to predict room usage at a university campus. Training data was created by collecting measurements from sensors measuring room CO2, humidity, temperature, light, motion and sound, while the labels was created manually by human inspection. Results include analyses of relationships between different sensor data sequences and recommendations for a prototype predictive model using deep recurrent neural networks.

scholarly journals Study of the impact of the COVID-19 pandemic on international air transportation

2021 ◽  
Vol 29 (1) ◽  
pp. 22-35
Author(s):  
Eugeny Yu. Shchetinin
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Stock Prices ◽  
Short Term Memory ◽  
Air Transportation ◽  
Time Series Forecasting ◽  
The Cost ◽  
The Impact

Time Series Forecasting has always been a very important area of research in many domains because many different types of data are stored as time series. Given the growing availability of data and computing power in the recent years, Deep Learning has become a fundamental part of the new generation of Time Series Forecasting models, obtaining excellent results.As different time series problems are studied in many different fields, a large number of new architectures have been developed in recent years. This has also been simplified by the growing availability of open source frameworks, which make the development of new custom network components easier and faster.In this paper three different Deep Learning Architecture for Time Series Forecasting are presented: Recurrent Neural Networks (RNNs), that are the most classical and used architecture for Time Series Forecasting problems; Long Short-Term Memory (LSTM), that are an evolution of RNNs developed in order to overcome the vanishing gradient problem; Gated Recurrent Unit (GRU), that are another evolution of RNNs, similar to LSTM.The article is devoted to modeling and forecasting the cost of international air transportation in a pandemic using deep learning methods. The author builds time series models of the American Airlines (AAL) stock prices for a selected period using LSTM, GRU, RNN recurrent neural networks models and compare the accuracy forecast results.

scholarly journals Predicting Remaining Useful Life using Time Series Embeddings based on Recurrent Neural Networks

2020 ◽  
Vol 9 (1) ◽  
Author(s):  
Narendhar Gugulothu ◽  
Vishnu TV ◽  
Pankaj Malhotra ◽  
Lovekesh Vig ◽  
Puneet Agarwal ◽  
...  
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Recurrent Neural Networks ◽  
Missing Values ◽  
Multivariate Time Series ◽  
Remaining Useful Life ◽  
Sensor Data ◽  
Novel Approach ◽  
Useful Life ◽  
Robust To Noise

We consider the problem of estimating the remaining useful life (RUL) of a system or a machine from sensor data. Many approaches for RUL estimation based on sensor data make assumptions about how machines degrade. Additionally, sensor data from machines is noisy and often suffers from missing values in many practical settings. We propose Embed-RUL: a novel approach for RUL estimation from sensor data that does not rely on any degradation-trend assumptions, is robust to noise, and handles missing values. Embed-RUL utilizes a sequence-to-sequence model based on Recurrent Neural Networks (RNNs) to generate embeddings for multivariate time series subsequences. The embeddings for normal and degraded machines tend to be different, and are therefore found to be useful for RUL estimation. We show that the embeddings capture the overall pattern in the time series while filtering out the noise, so that the embeddings of two machines with similar operational behavior are close to each other, even when their sensor readings have significant and varying levels of noise content. We perform experiments on publicly available turbofan engine dataset and a proprietary real-world dataset, and demonstrate that Embed-RUL outperforms the previously reported state-of-the-art (Malhotra, TV, et al., 2016) on several metrics.

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sequence Similarity ◽  
Learning Models ◽  
Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

A review of irregular time series data handling with gated recurrent neural networks

2021 ◽  
Vol 441 ◽  
pp. 161-178
Author(s):  
Philip B. Weerakody ◽  
Kok Wai Wong ◽  
Guanjin Wang ◽  
Wendell Ela
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Recurrent Neural Networks ◽  
Time Series Data ◽  
Series Data ◽  
Data Handling

scholarly journals Improving Web QoE Monitoring for Encrypted Network Traffic through Time Series Modeling

2021 ◽  
Vol 48 (4) ◽  
pp. 37-40
Author(s):  
Nikolas Wehner ◽  
Michael Seufert ◽  
Joshua Schuler ◽  
Sarah Wassermann ◽  
Pedro Casas ◽  
...  
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Recurrent Neural Networks ◽  
Quality Of Experience ◽  
Unique Feature ◽  
Time Series Modeling ◽  
Web Browsing ◽  
Large Dataset ◽  
Efficient Approximation

This paper addresses the problem of Quality of Experience (QoE) monitoring for web browsing. In particular, the inference of common Web QoE metrics such as Speed Index (SI) is investigated. Based on a large dataset collected with open web-measurement platforms on different device-types, a unique feature set is designed and used to estimate the RUMSI - an efficient approximation to SI, with machinelearning based regression and classification approaches. Results indicate that it is possible to estimate the RUMSI accurately, and that in particular, recurrent neural networks are highly suitable for the task, as they capture the network dynamics more precisely.

scholarly journals Laplacian networks: bounding indicator function smoothness for neural networks robustness

2021 ◽  
Vol 10 ◽  
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Supervised Learning ◽  
Indicator Function ◽  
Training Data ◽  
Theoretical Justification ◽  
The Past ◽  
Noisy Examples

For the past few years, deep learning (DL) robustness (i.e. the ability to maintain the same decision when inputs are subject to perturbations) has become a question of paramount importance, in particular in settings where misclassification can have dramatic consequences. To address this question, authors have proposed different approaches, such as adding regularizers or training using noisy examples. In this paper we introduce a regularizer based on the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DL architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness on classical supervised learning vision datasets for various types of perturbations. We also show it can be combined with existing methods to increase overall robustness.

scholarly journals Patch-Based Change Detection Method for SAR Images with Label Updating Strategy

2021 ◽  
Vol 13 (7) ◽  
pp. 1236
Author(s):  
Yuanjun Shu ◽  
Wei Li ◽  
Menglong Yang ◽  
Peng Cheng ◽  
Songchen Han
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Change Detection ◽  
Detection Method ◽  
Training Data ◽  
Synthetic Aperture ◽  
Learning Method ◽  
Sar Images ◽  
Two Stage ◽  
Second Stage

Convolutional neural networks (CNNs) have been widely used in change detection of synthetic aperture radar (SAR) images and have been proven to have better precision than traditional methods. A two-stage patch-based deep learning method with a label updating strategy is proposed in this paper. The initial label and mask are generated at the pre-classification stage. Then a two-stage updating strategy is applied to gradually recover changed areas. At the first stage, diversity of training data is gradually restored. The output of the designed CNN network is further processed to generate a new label and a new mask for the following learning iteration. As the diversity of data is ensured after the first stage, pixels within uncertain areas can be easily classified at the second stage. Experiment results on several representative datasets show the effectiveness of our proposed method compared with several existing competitive methods.

DYNAMICAL RECURRENT NEURAL NETWORKS — TOWARDS ENVIRONMENTAL TIME SERIES PREDICTION

1995 ◽  
Vol 06 (02) ◽  
pp. 145-170 ◽  
Author(s):  
ALEX AUSSEM ◽  
FIONN MURTAGH ◽  
MARC SARAZIN
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Recurrent Neural Networks ◽  
Time Series Prediction ◽  
Predictive Ability ◽  
Complex Problem ◽  
Internal Variables ◽  
Internal Dynamic ◽  
K Nearest Neighbors ◽  
Fuzzy Coding

Dynamical Recurrent Neural Networks (DRNN) (Aussem 1995a) are a class of fully recurrent networks obtained by modeling synapses as autoregressive filters. By virtue of their internal dynamic, these networks approximate the underlying law governing the time series by a system of nonlinear difference equations of internal variables. They therefore provide history-sensitive forecasts without having to be explicitly fed with external memory. The model is trained by a local and recursive error propagation algorithm called temporal-recurrent-backpropagation. The efficiency of the procedure benefits from the exponential decay of the gradient terms backpropagated through the adjoint network. We assess the predictive ability of the DRNN model with meteorological and astronomical time series recorded around the candidate observation sites for the future VLT telescope. The hope is that reliable environmental forecasts provided with the model will allow the modern telescopes to be preset, a few hours in advance, in the most suited instrumental mode. In this perspective, the model is first appraised on precipitation measurements with traditional nonlinear AR and ARMA techniques using feedforward networks. Then we tackle a complex problem, namely the prediction of astronomical seeing, known to be a very erratic time series. A fuzzy coding approach is used to reduce the complexity of the underlying laws governing the seeing. Then, a fuzzy correspondence analysis is carried out to explore the internal relationships in the data. Based on a carefully selected set of meteorological variables at the same time-point, a nonlinear multiple regression, termed nowcasting (Murtagh et al. 1993, 1995), is carried out on the fuzzily coded seeing records. The DRNN is shown to outperform the fuzzy k-nearest neighbors method.

Sign in / Sign up

Export Citation Format

Share Document