FilterNet: A Many-to-Many Deep Learning Architecture for Time Series Classification

We present and benchmark FilterNet, a flexible deep learning architecture for time series classification tasks, such as activity recognition via multichannel sensor data. It adapts popular CNN and CNN-LSTM motifs which have excelled in activity recognition benchmarks, implementing them in a many-to-many architecture to markedly improve frame-by-frame accuracy, event segmentation accuracy, model size, and computational efficiency. We propose several model variants, evaluate them alongside other published models using the Opportunity benchmark dataset, demonstrate the effect of model ensembling and of altering key parameters, and quantify the quality of the models’ segmentation of discrete events. We also offer recommendations for use and suggest potential model extensions. FilterNet advances the state of the art in all measured accuracy and speed metrics on the benchmarked dataset, and it can be extensively customized for other applications.

Download Full-text

FilterNet: A Many-to-Many Deep Learning Architecture for Time Series Classification

10.20944/preprints202002.0318.v2 ◽

2020 ◽

Author(s):

Robert D. Chambers ◽

Nathanael C. Yoder

Keyword(s):

Time Series ◽

Deep Learning ◽

Activity Recognition ◽

Sensor Data ◽

Time Series Classification ◽

Discrete Events ◽

Segmentation Accuracy ◽

Model Size ◽

Classification Tasks

We present and benchmark FilterNet, a flexible deep learning architecture for time series classification tasks, such as activity recognition via multichannel sensor data. It adapts popular CNN and CNN-LSTM motifs which have excelled in activity recognition benchmarks, implementing them in a many-to-many architecture to markedly improve frame-by-frame accuracy, event segmentation accuracy, model size, and computational efficiency. We propose several model variants, evaluate them alongside other published models using the Opportunity benchmark dataset, demonstrate the effect of model ensembling and of altering key parameters, and quantify the quality of the models’ segmentation of discrete events. We also offer recommendations for use and suggest potential model extensions. FilterNet advances the state of the art in all measured accuracy and speed metrics on the benchmarked dataset, and it can be extensively customized for other applications.

Download Full-text

Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning

Sensors ◽

10.3390/s19071716 ◽

2019 ◽

Vol 19 (7) ◽

pp. 1716 ◽

Cited By ~ 20

Author(s):

Seungeun Chung ◽

Jiyoun Lim ◽

Kyoung Ju Noh ◽

Gague Kim ◽

Hyuntae Jeong

Keyword(s):

Deep Learning ◽

Data Acquisition ◽

Sensor Fusion ◽

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Sampling Rate ◽

Human Activity Recognition ◽

Sensor Data ◽

Activity Data

In this paper, we perform a systematic study about the on-body sensor positioning and data acquisition details for Human Activity Recognition (HAR) systems. We build a testbed that consists of eight body-worn Inertial Measurement Units (IMU) sensors and an Android mobile device for activity data collection. We develop a Long Short-Term Memory (LSTM) network framework to support training of a deep learning model on human activity data, which is acquired in both real-world and controlled environments. From the experiment results, we identify that activity data with sampling rate as low as 10 Hz from four sensors at both sides of wrists, right ankle, and waist is sufficient in recognizing Activities of Daily Living (ADLs) including eating and driving activity. We adopt a two-level ensemble model to combine class-probabilities of multiple sensor modalities, and demonstrate that a classifier-level sensor fusion technique can improve the classification performance. By analyzing the accuracy of each sensor on different types of activity, we elaborate custom weights for multimodal sensor fusion that reflect the characteristic of individual activities.

Download Full-text

Sensor-Based Human Activity Recognition with Spatio-Temporal Deep Learning

Sensors ◽

10.3390/s21062141 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2141

Author(s):

Ohoud Nafea ◽

Wadood Abdul ◽

Ghulam Muhammad ◽

Mansour Alsulaiman

Keyword(s):

Deep Learning ◽

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Human Activity Recognition ◽

Sensor Data ◽

Effective Selection ◽

Spatio Temporal ◽

High Level ◽

Improved Accuracy

Human activity recognition (HAR) remains a challenging yet crucial problem to address in computer vision. HAR is primarily intended to be used with other technologies, such as the Internet of Things, to assist in healthcare and eldercare. With the development of deep learning, automatic high-level feature extraction has become a possibility and has been used to optimize HAR performance. Furthermore, deep-learning techniques have been applied in various fields for sensor-based HAR. This study introduces a new methodology using convolution neural networks (CNN) with varying kernel dimensions along with bi-directional long short-term memory (BiLSTM) to capture features at various resolutions. The novelty of this research lies in the effective selection of the optimal video representation and in the effective extraction of spatial and temporal features from sensor data using traditional CNN and BiLSTM. Wireless sensor data mining (WISDM) and UCI datasets are used for this proposed methodology in which data are collected through diverse methods, including accelerometers, sensors, and gyroscopes. The results indicate that the proposed scheme is efficient in improving HAR. It was thus found that unlike other available methods, the proposed method improved accuracy, attaining a higher score in the WISDM dataset compared to the UCI dataset (98.53% vs. 97.05%).

Download Full-text

Joint Learning of Temporal Models to Handle Imbalanced Data for Human Activity Recognition

Applied Sciences ◽

10.3390/app10155293 ◽

2020 ◽

Vol 10 (15) ◽

pp. 5293 ◽

Cited By ~ 1

Author(s):

Rebeen Ali Hamad ◽

Longzhi Yang ◽

Wai Lok Woo ◽

Bo Wei

Keyword(s):

Deep Learning ◽

Activity Recognition ◽

Human Activity ◽

Smart Home ◽

Short Term Memory ◽

Recognition System ◽

Human Activity Recognition ◽

Sensor Data ◽

Temporal Models ◽

Wide Range

Human activity recognition has become essential to a wide range of applications, such as smart home monitoring, health-care, surveillance. However, it is challenging to deliver a sufficiently robust human activity recognition system from raw sensor data with noise in a smart environment setting. Moreover, imbalanced human activity datasets with less frequent activities create extra challenges for accurate activity recognition. Deep learning algorithms have achieved promising results on balanced datasets, but their performance on imbalanced datasets without explicit algorithm design cannot be promised. Therefore, we aim to realise an activity recognition system using multi-modal sensors to address the issue of class imbalance in deep learning and improve recognition accuracy. This paper proposes a joint diverse temporal learning framework using Long Short Term Memory and one-dimensional Convolutional Neural Network models to improve human activity recognition, especially for less represented activities. We extensively evaluate the proposed method for Activities of Daily Living recognition using binary sensors dataset. A comparative study on five smart home datasets demonstrate that our proposed approach outperforms the existing individual temporal models and their hybridization. Furthermore, this is particularly the case for minority classes in addition to reasonable improvement on the majority classes of human activities.

Download Full-text

Multivariate Time Series Classification of Sensor Data from an Industrial Drying Hopper: A Deep Learning Approach

10.33915/etd.8309 ◽

2021 ◽

Author(s):

Md Mushfiqur Rahman

Keyword(s):

Time Series ◽

Deep Learning ◽

Multivariate Time Series ◽

Sensor Data ◽

Learning Approach ◽

Time Series Classification

Download Full-text

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

International Journal of Sensors Wireless Communications and Control ◽

10.2174/2210327908666180727123657 ◽

2019 ◽

Vol 9 (1) ◽

pp. 16-31

Author(s):

Kyungkoo Jun

Keyword(s):

Fourier Transform ◽

Deep Learning ◽

Short Term Memory ◽

Window Size ◽

Sensor Data ◽

Data Sets ◽

Data Set ◽

Proposed Model ◽

Testing Data ◽

Labeling Scheme

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

Download Full-text

Similarity Embedding Networks for Robust Human Activity Recognition

ACM Transactions on Knowledge Discovery from Data ◽

10.1145/3448021 ◽

2021 ◽

Vol 15 (6) ◽

pp. 1-17

Author(s):

Chenglin Li ◽

Carrie Lu Tong ◽

Di Niu ◽

Bei Jiang ◽

Xiao Zuo ◽

...

Keyword(s):

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Real Space ◽

Human Activity Recognition ◽

Sensor Data ◽

Activity Data ◽

Extensive Evaluation ◽

Sensor Signals ◽

Public Datasets

Deep learning models for human activity recognition (HAR) based on sensor data have been heavily studied recently. However, the generalization ability of deep models on complex real-world HAR data is limited by the availability of high-quality labeled activity data, which are hard to obtain. In this article, we design a similarity embedding neural network that maps input sensor signals onto real vectors through carefully designed convolutional and Long Short-Term Memory (LSTM) layers. The embedding network is trained with a pairwise similarity loss, encouraging the clustering of samples from the same class in the embedded real space, and can be effectively trained on a small dataset and even on a noisy dataset with mislabeled samples. Based on the learned embeddings, we further propose both nonparametric and parametric approaches for activity recognition. Extensive evaluation based on two public datasets has shown that the proposed similarity embedding network significantly outperforms state-of-the-art deep models on HAR classification tasks, is robust to mislabeled samples in the training set, and can also be used to effectively denoise a noisy dataset.

Download Full-text

The Deep Learning LSTM and MTD Models Best Predict Acute Respiratory Infection among Under-Five-Year Old Children in Somaliland

Symmetry ◽

10.3390/sym13071156 ◽

2021 ◽

Vol 13 (7) ◽

pp. 1156

Author(s):

Mohamed Yusuf Hassan

Keyword(s):

Time Series ◽

Deep Learning ◽

Respiratory Infection ◽

Acute Respiratory Infection ◽

Short Term Memory ◽

Mean Deviation ◽

Sarima Model ◽

Under Five ◽

Competing Models ◽

Long Short Term Memory

The most effective techniques for predicting time series patterns include machine learning and classical time series methods. The aim of this study is to search for the best artificial intelligence and classical forecasting techniques that can predict the spread of acute respiratory infection (ARI) and pneumonia among under-five-year old children in Somaliland. The techniques used in the study include seasonal autoregressive integrated moving averages (SARIMA), mixture transitions distribution (MTD), and long short term memory (LSTM) deep learning. The data used in the study were monthly observations collected from five regions in Somaliland from 2011–2014. Prediction results from the three best competing models are compared by using root mean square error (RMSE) and absolute mean deviation (MAD) accuracy measures. Results have shown that the deep learning LSTM and MTD models slightly outperformed the classical SARIMA model in predicting ARI values.

Download Full-text

Prognostics and RUL Estimations of SAC305, SAC105 and SnAg Solders Under Temperature and Vibration Using Long Short-Term Memory (LSTM) Deep Learning

10.1115/ipack2021-74066 ◽

2021 ◽

Author(s):

Pradeep Lall ◽

Tony Thomas ◽

Ken Blecker

Keyword(s):

Time Series ◽

Deep Learning ◽

Short Term Memory ◽

Remaining Useful Life ◽

Operating Conditions ◽

Short Term ◽

Term Memory ◽

Rectangular Pattern ◽

Useful Life ◽

Long Short Term Memory

Abstract Prognostics and Remaining Useful Life (RUL) estimations of complex systems are essential to operational safety, increased efficiency, and help to schedule maintenance proactively. Modeling the remaining useful life of a system with many complexities is possible with the rapid development in the field of deep learning as a computational technique for failure prediction. Deep learning can adapt to multivariate parameters complex and nonlinear behavior, which is difficult using traditional time-series models for forecasting and prediction purposes. In this paper, a deep learning approach based on Long Short-Term Memory (LSTM) network is used to predict the remaining useful life of the PCB at different conditions of temperature and vibration. This technique can identify the different underlying patterns in the time series that can predict the RUL. This study involves feature vector identification and RUL estimations for SAC305, SAC105, and Tin Lead solder PCBs under different vibration levels and temperature conditions. The acceleration levels of vibration are fixed at 5g and 10g, while the temperature levels are 55°C and 100°C. The test board is a multilayer FR4 configuration with JEDEC standard dimensions consists of twelve packages arranged in a rectangular pattern. Strain signals are acquired from the backside of the PCB at symmetric locations to identify the failure of all the packages during vibration. The strain signals are resistance values that are acquired simultaneously during the experiment until the failure of most of the packages on the board. The feature vectors are identified from statistical analysis on the strain signals frequency and instantaneous frequency components. The principal component analysis is used as a data reduction technique to identify the different patterns produced from the four strain signals with failures of the packages during vibration. LSTM deep learning method is used to model the RUL of the packages at different individual operating conditions of vibration for all three solder materials involved in this study. A combined model for RUL prediction for a material that can take care of the changes in the operating conditions is also modeled for each material.

Download Full-text