scholarly journals Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network

Sensors ◽  
2018 ◽  
Vol 18 (9) ◽  
pp. 2892 ◽  
Author(s):  
Odongo Steven Eyobu ◽  
Dong Han
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Activity Recognition ◽  
Human Activity ◽  
Human Activities ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sensor Data ◽  
Motion Data ◽  
Feature Representations

Wearable inertial measurement unit (IMU) sensors are powerful enablers for acquisition of motion data. Specifically, in human activity recognition (HAR), IMU sensor data collected from human motion are categorically combined to formulate datasets that can be used for learning human activities. However, successful learning of human activities from motion data involves the design and use of proper feature representations of IMU sensor data and suitable classifiers. Furthermore, the scarcity of labelled data is an impeding factor in the process of understanding the performance capabilities of data-driven learning models. To tackle these challenges, two primary contributions are in this article: first; by using raw IMU sensor data, a spectrogram-based feature extraction approach is proposed. Second, an ensemble of data augmentations in feature space is proposed to take care of the data scarcity problem. Performance tests were conducted on a deep long term short term memory (LSTM) neural network architecture to explore the influence of feature representations and the augmentations on activity recognition accuracy. The proposed feature extraction approach combined with the data augmentation ensemble produces state-of-the-art accuracy results in HAR. A performance evaluation of each augmentation approach is performed to show the influence on classification accuracy. Finally, in addition to using our own dataset, the proposed data augmentation technique is evaluated against the University of California, Irvine (UCI) public online HAR dataset and yields state-of-the-art accuracy results at various learning rates.

scholarly journals Human Activity Recognition for Indoor Localization Using Smartphone Inertial Sensors

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6316
Author(s):  
Dinis Moreira ◽  
Marília Barandas ◽  
Tiago Rocha ◽  
Pedro Alves ◽  
Ricardo Santos ◽  
...  
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Human Activities ◽  
Short Term Memory ◽  
Indoor Localization ◽  
Inertial Sensor ◽  
Human Activity Recognition ◽  
Location Based Services ◽  
Sensor Data ◽  
Indoor Environments

With the fast increase in the demand for location-based services and the proliferation of smartphones, the topic of indoor localization is attracting great interest. In indoor environments, users’ performed activities carry useful semantic information. These activities can then be used by indoor localization systems to confirm users’ current relative locations in a building. In this paper, we propose a deep-learning model based on a Convolutional Long Short-Term Memory (ConvLSTM) network to classify human activities within the indoor localization scenario using smartphone inertial sensor data. Results show that the proposed human activity recognition (HAR) model accurately identifies nine types of activities: not moving, walking, running, going up in an elevator, going down in an elevator, walking upstairs, walking downstairs, or going up and down a ramp. Moreover, predicted human activities were integrated within an existing indoor positioning system and evaluated in a multi-story building across several testing routes, with an average positioning error of 2.4 m. The results show that the inclusion of human activity information can reduce the overall localization error of the system and actively contribute to the better identification of floor transitions within a building. The conducted experiments demonstrated promising results and verified the effectiveness of using human activity-related information for indoor localization.

scholarly journals A Method for Sensor-Based Activity Recognition in Missing Data Scenario

Sensors ◽  
2020 ◽  
Vol 20 (14) ◽  
pp. 3811
Author(s):  
Tahera Hossain ◽  
Md. Atiqur Rahman Ahad ◽  
Sozo Inoue
Keyword(s):  
Machine Learning ◽  
Missing Data ◽  
Activity Recognition ◽  
Human Activity ◽  
Human Activities ◽  
Real Life ◽  
Sensor Data ◽  
Quality Data ◽  
Support Vector ◽  
Random Pattern

Sensor-based human activity recognition has various applications in the arena of healthcare, elderly smart-home, sports, etc. There are numerous works in this field—to recognize various human activities from sensor data. However, those works are based on data patterns that are clean data and have almost no missing data, which is a genuine concern for real-life healthcare centers. Therefore, to address this problem, we explored the sensor-based activity recognition when some partial data were lost in a random pattern. In this paper, we propose a novel method to improve activity recognition while having missing data without any data recovery. For the missing data pattern, we considered data to be missing in a random pattern, which is a realistic missing pattern for sensor data collection. Initially, we created different percentages of random missing data only in the test data, while the training was performed on good quality data. In our proposed approach, we explicitly induce different percentages of missing data randomly in the raw sensor data to train the model with missing data. Learning with missing data reinforces the model to regulate missing data during the classification of various activities that have missing data in the test module. This approach demonstrates the plausibility of the machine learning model, as it can learn and predict from an identical domain. We exploited several time-series statistical features to extricate better features in order to comprehend various human activities. We explored both support vector machine and random forest as machine learning models for activity classification. We developed a synthetic dataset to empirically evaluate the performance and show that the method can effectively improve the recognition accuracy from 80.8% to 97.5%. Afterward, we tested our approach with activities from two challenging benchmark datasets: the human activity sensing consortium (HASC) dataset and single chest-mounted accelerometer dataset. We examined the method for different missing percentages, varied window sizes, and diverse window sliding widths. Our explorations demonstrated improved recognition performances even in the presence of missing data. The achieved results provide persuasive findings on sensor-based activity recognition in the presence of missing data.

scholarly journals New Sensor Data Structuring for Deeper Feature Extraction in Human Activity Recognition

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2814
Author(s):  
Tsige Tadesse Alemayoh ◽  
Jae Hoon Lee ◽  
Shingo Okamoto
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Real Time ◽  
Activity Recognition ◽  
Time Domain ◽  
Sensor Data ◽  
Series Data ◽  
Smart Devices ◽  
Time Activity ◽  
Data Structuring

For the effective application of thriving human-assistive technologies in healthcare services and human–robot collaborative tasks, computing devices must be aware of human movements. Developing a reliable real-time activity recognition method for the continuous and smooth operation of such smart devices is imperative. To achieve this, light and intelligent methods that use ubiquitous sensors are pivotal. In this study, with the correlation of time series data in mind, a new method of data structuring for deeper feature extraction is introduced herein. The activity data were collected using a smartphone with the help of an exclusively developed iOS application. Data from eight activities were shaped into single and double-channels to extract deep temporal and spatial features of the signals. In addition to the time domain, raw data were represented via the Fourier and wavelet domains. Among the several neural network models used to fit the deep-learning classification of the activities, a convolutional neural network with a double-channeled time-domain input performed well. This method was further evaluated using other public datasets, and better performance was obtained. The practicability of the trained model was finally tested on a computer and a smartphone in real-time, where it demonstrated promising results.

scholarly journals MSTCN: A multiscale temporal convolutional network for user independent human activity recognition

F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 1261
Author(s):  
Sarmela Raja Sekaran ◽  
Pang Ying Han ◽  
Goh Fan Ling ◽  
Ooi Shih Yin
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Activity Recognition ◽  
Human Activity ◽  
Short Term Memory ◽  
Feature Engineering ◽  
Short Term ◽  
Convolutional Network ◽  
Term Memory ◽  
Temporal Features

Background: In recent years, human activity recognition (HAR) has been an active research topic due to its widespread application in various fields such as healthcare, sports, patient monitoring, etc. HAR approaches can be categorised as handcrafted feature methods (HCF) and deep learning methods (DL). HCF involves complex data pre-processing and manual feature extraction in which the models may be exposed to high bias and crucial implicit pattern loss. Hence, DL approaches are introduced due to their exceptional recognition performance. Convolutional Neural Network (CNN) extracts spatial features while preserving localisation. However, it hardly captures temporal features. Recurrent Neural Network (RNN) learns temporal features, but it is susceptible to gradient vanishing and suffers from short-term memory problems. Unlike RNN, Long-Short Term Memory network has a relatively longer-term dependency. However, it consumes higher computation and memory because it computes and stores partial results at each level. Methods: This work proposes a novel multiscale temporal convolutional network (MSTCN) based on the Inception model with a temporal convolutional architecture. Unlike HCF methods, MSTCN requires minimal pre-processing and no manual feature engineering. Further, multiple separable convolutions with different-sized kernels are used in MSTCN for multiscale feature extraction. Dilations are applied to each separable convolution to enlarge the receptive fields without increasing the model parameters. Moreover, residual connections are utilised to prevent information loss and gradient vanishing. These features enable MSTCN to possess a longer effective history while maintaining a relatively low in-network computation. Results: The performance of MSTCN is evaluated on UCI and WISDM datasets using subject independent protocol with no overlapping subjects between the training and testing sets. MSTCN achieves F1 scores of 0.9752 on UCI and 0.9470 on WISDM. Conclusion: The proposed MSTCN dominates the other state-of-the-art methods by acquiring high recognition accuracies without requiring any manual feature engineering.

scholarly journals Dilated causal convolution with multi-head self attention for sensor human activity recognition

2021 ◽  
Author(s):  
Rebeen Ali Hamad ◽  
Masashi Kimura ◽  
Longzhi Yang ◽  
Wai Lok Woo ◽  
Bo Wei
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Activity Recognition ◽  
Human Activity ◽  
Network Architecture ◽  
Short Term Memory ◽  
Human Activity Recognition ◽  
Sensor Data ◽  
One Dimensional ◽  
Temporal Models

AbstractSystems of sensor human activity recognition are becoming increasingly popular in diverse fields such as healthcare and security. Yet, developing such systems poses inherent challenges due to the variations and complexity of human behaviors during the performance of physical activities. Recurrent neural networks, particularly long short-term memory have achieved promising results on numerous sequential learning problems, including sensor human activity recognition. However, parallelization is inhibited in recurrent networks due to sequential operation and computation that lead to slow training, occupying more memory and hard convergence. One-dimensional convolutional neural network processes input temporal sequential batches independently that lead to effectively executed operations in parallel. Despite that, a one-dimensional Convolutional Neural Network is not sensitive to the order of the time steps which is crucial for accurate and robust systems of sensor human activity recognition. To address this problem, we propose a network architecture based on dilated causal convolution and multi-head self-attention mechanisms that entirely dispense recurrent architectures to make efficient computation and maintain the ordering of the time steps. The proposed method is evaluated for human activities using smart home binary sensors data and wearable sensor data. Results of conducted extensive experiments on eight public and benchmark HAR data sets show that the proposed network outperforms the state-of-the-art models based on recurrent settings and temporal models.

scholarly journals An Efficient and Lightweight Deep Learning Model for Human Activity Recognition Using Smartphones

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3845
Author(s):  
Ankita ◽  
Shalli Rani ◽  
Himanshi Babbar ◽  
Sonya Coleman ◽  
Aman Singh ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Activity Recognition ◽  
Human Activity ◽  
Human Activities ◽  
Human Activity Recognition ◽  
Alternative Form ◽  
Accelerometer Data ◽  
Time Step ◽  
Proposed Model

Traditional pattern recognition approaches have gained a lot of popularity. However, these are largely dependent upon manual feature extraction, which makes the generalized model obscure. The sequences of accelerometer data recorded can be classified by specialized smartphones into well known movements that can be done with human activity recognition. With the high success and wide adaptation of deep learning approaches for the recognition of human activities, these techniques are widely used in wearable devices and smartphones to recognize the human activities. In this paper, convolutional layers are combined with long short-term memory (LSTM), along with the deep learning neural network for human activities recognition (HAR). The proposed model extracts the features in an automated way and categorizes them with some model attributes. In general, LSTM is alternative form of recurrent neural network (RNN) which is famous for temporal sequences’ processing. In the proposed architecture, a dataset of UCI-HAR for Samsung Galaxy S2 is used for various human activities. The CNN classifier, which should be taken single, and LSTM models should be taken in series and take the feed data. For each input, the CNN model is applied, and each input image’s output is transferred to the LSTM classifier as a time step. The number of filter maps for mapping of the various portions of image is the most important hyperparameter used. Transformation on the basis of observations takes place by using Gaussian standardization. CNN-LSTM, a proposed model, is an efficient and lightweight model that has shown high robustness and better activity detection capability than traditional algorithms by providing the accuracy of 97.89%.

Contemporary Human Activity Recognition Based Predictions by Sensors Using Random Forest Classifier

2021 ◽  
Vol 18 (4) ◽  
pp. 1243-1250
Author(s):  
Sujatha Jamuna Anand ◽  
S. Magesh ◽  
I. Arockiamary
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Human Activities ◽  
Human Activity Recognition ◽  
Sensor Data ◽  
Learning Techniques ◽  
Machine Learning Approach ◽  
Research Task ◽  
Using Data ◽  
Medical And Healthcare

The task of recognizing human activities directs extensive divergence of various functions and applications. Despite analysing the intricate activity it endures demanding requirements in contemporary field of research. A subject performs a definite task at a particular time by determining the activity by using sensor data. In this research task we appraise a unique way by using data with supervised learning techniques by placing sensors on the human body by contingent upon classification process at different stages. The State-of-art machine learning approach random forests are widely discussed in terms of covering practical and theoretical aspects of body sensing. The eventual target is the superior rate of accurate predictions effecting Human Activity Recognition further effective for behavioural monitoring, medical and healthcare sectors. Classification processes are deployed for pairs of activities that are distracted often and this work attempts to analyse the essential sensors for the improved prediction. The results shows the best accuracy scores and the remaining of our findings we expose the outline, exhibiting the degree of distraction between features of ranking and human activities which renders back to sensor ranking.

Sign in / Sign up

Export Citation Format

Share Document