scholarly journals A Framework of Combining Short-Term Spatial/Frequency Feature Extraction and Long-Term IndRNN for Activity Recognition

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6984
Author(s):  
Beidi Zhao ◽  
Shuai Li ◽  
Yanbo Gao ◽  
Chuankun Li ◽  
Wanqing Li
Keyword(s):  
Feature Extraction ◽  
Spatial Frequency ◽  
Activity Recognition ◽  
Sensor Data ◽  
Short Term ◽  
Temporal Features ◽  
Frequency Domains ◽  
Smartphone Sensors ◽  
Frequency Feature

Smartphone-sensors-based human activity recognition is attracting increasing interest due to the popularization of smartphones. It is a difficult long-range temporal recognition problem, especially with large intraclass distances such as carrying smartphones at different locations and small interclass distances such as taking a train or subway. To address this problem, we propose a new framework of combining short-term spatial/frequency feature extraction and a long-term independently recurrent neural network (IndRNN) for activity recognition. Considering the periodic characteristics of the sensor data, short-term temporal features are first extracted in the spatial and frequency domains. Then, the IndRNN, which can capture long-term patterns, is used to further obtain the long-term features for classification. Given the large differences when the smartphone is carried at different locations, a group-based location recognition is first developed to pinpoint the location of the smartphone. The Sussex-Huawei Locomotion (SHL) dataset from the SHL Challenge is used for evaluation. An earlier version of the proposed method won the second place award in the SHL Challenge 2020 (first place if not considering the multiple models fusion approach). The proposed method is further improved in this paper and achieves 80.72% accuracy, better than the existing methods using a single model.

scholarly journals MSTCN: A multiscale temporal convolutional network for user independent human activity recognition

F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 1261
Author(s):  
Sarmela Raja Sekaran ◽  
Pang Ying Han ◽  
Goh Fan Ling ◽  
Ooi Shih Yin
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Activity Recognition ◽  
Human Activity ◽  
Short Term Memory ◽  
Feature Engineering ◽  
Short Term ◽  
Convolutional Network ◽  
Term Memory ◽  
Temporal Features

Background: In recent years, human activity recognition (HAR) has been an active research topic due to its widespread application in various fields such as healthcare, sports, patient monitoring, etc. HAR approaches can be categorised as handcrafted feature methods (HCF) and deep learning methods (DL). HCF involves complex data pre-processing and manual feature extraction in which the models may be exposed to high bias and crucial implicit pattern loss. Hence, DL approaches are introduced due to their exceptional recognition performance. Convolutional Neural Network (CNN) extracts spatial features while preserving localisation. However, it hardly captures temporal features. Recurrent Neural Network (RNN) learns temporal features, but it is susceptible to gradient vanishing and suffers from short-term memory problems. Unlike RNN, Long-Short Term Memory network has a relatively longer-term dependency. However, it consumes higher computation and memory because it computes and stores partial results at each level. Methods: This work proposes a novel multiscale temporal convolutional network (MSTCN) based on the Inception model with a temporal convolutional architecture. Unlike HCF methods, MSTCN requires minimal pre-processing and no manual feature engineering. Further, multiple separable convolutions with different-sized kernels are used in MSTCN for multiscale feature extraction. Dilations are applied to each separable convolution to enlarge the receptive fields without increasing the model parameters. Moreover, residual connections are utilised to prevent information loss and gradient vanishing. These features enable MSTCN to possess a longer effective history while maintaining a relatively low in-network computation. Results: The performance of MSTCN is evaluated on UCI and WISDM datasets using subject independent protocol with no overlapping subjects between the training and testing sets. MSTCN achieves F1 scores of 0.9752 on UCI and 0.9470 on WISDM. Conclusion: The proposed MSTCN dominates the other state-of-the-art methods by acquiring high recognition accuracies without requiring any manual feature engineering.

Topological Gaussian ARTs with Short-Term and Long-Term Memory for Map Building and Fuzzy Motion Planning

2016 ◽  
Vol 6 (2) ◽  
pp. 63-87 ◽  
Author(s):  
Chin Wei Hong ◽  
Loo Chu Kiong ◽  
Kubota Naoyuki
Keyword(s):  
Motion Planning ◽  
Short Term Memory ◽  
Sensor Data ◽  
Long Term Memory ◽  
Map Building ◽  
Short Term ◽  
Topological Map ◽  
Term Memory ◽  
Input Layer

This paper proposes a cognitive architecture for building a topological map incrementally inspired by beta oscillations during place cell learning in hippocampus. The proposed architecture consists of two layer: the short-term memory layer and the long-term memory layer. The short-term memory layer emulates the entorhinal and the ? is the orientation system; the long-term memory layer emulates the hippocampus. Nodes in the topological map represent place cells (robot location), links connect nodes and store robot action (i.e. adjacent angle between connected nodes). The proposed method is formed by multiple Gaussian Adaptive Resonance Theory to receive data from various sensors for the map building. It consists of input layer and memory layer. The input layer obtains sensor data and incrementally categorizes the acquired information as topological nodes temporarily (short-term memory). In the long-term memory layer, the categorized information will be associated with robot actions to form the topological map (long-term memory). The advantages of the proposed method are: 1) it is a cognitive model that does not require human defined information and advanced knowledge to implement in a natural environment; 2) it can generate the map by processing various sensors data simultaneously in continuous space that is important for real world implementation; and 3) it is an incremental and unsupervised learning approach. Thus, the authors combine their Topological Gaussian ARTs method (TGARTs) with fuzzy motion planning to constitute a basis for mobile robot navigation in environment with slightly changes. Finally, the proposed approach was verified with several simulations using standardized benchmark datasets and real robot implementation.

A Logic-Based Approach to Activity Recognition

2013 ◽  
pp. 1-13
Author(s):  
Alexander Artikis ◽  
Marek Sergot ◽  
Georgios Paliouras
Keyword(s):  
Activity Recognition ◽  
Human Activities ◽  
Video Content ◽  
Short Term ◽  
Event Calculus ◽  
Video Frames ◽  
Detailed Evaluation ◽  
Spatio Temporal ◽  
Complex Activities

The authors have been developing a system for recognising human activities given a symbolic representation of video content. The input of the system is a stream of time-stamped short-term activities detected on video frames. The output of the system is a set of recognised long-term activities, which are pre-defined spatio-temporal combinations of short-term activities. The constraints on the short-term activities that, if satisfied, lead to the recognition of a long-term activity, are expressed using a dialect of the Event Calculus. The authors illustrate the expressiveness of the dialect by showing the representation of several typical complex activities. Furthermore, they present a detailed evaluation of the system through experimentation on a benchmark dataset of surveillance videos.

scholarly journals New Sensor Data Structuring for Deeper Feature Extraction in Human Activity Recognition

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2814
Author(s):  
Tsige Tadesse Alemayoh ◽  
Jae Hoon Lee ◽  
Shingo Okamoto
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Real Time ◽  
Activity Recognition ◽  
Time Domain ◽  
Sensor Data ◽  
Series Data ◽  
Smart Devices ◽  
Time Activity ◽  
Data Structuring

For the effective application of thriving human-assistive technologies in healthcare services and human–robot collaborative tasks, computing devices must be aware of human movements. Developing a reliable real-time activity recognition method for the continuous and smooth operation of such smart devices is imperative. To achieve this, light and intelligent methods that use ubiquitous sensors are pivotal. In this study, with the correlation of time series data in mind, a new method of data structuring for deeper feature extraction is introduced herein. The activity data were collected using a smartphone with the help of an exclusively developed iOS application. Data from eight activities were shaped into single and double-channels to extract deep temporal and spatial features of the signals. In addition to the time domain, raw data were represented via the Fourier and wavelet domains. Among the several neural network models used to fit the deep-learning classification of the activities, a convolutional neural network with a double-channeled time-domain input performed well. This method was further evaluated using other public datasets, and better performance was obtained. The practicability of the trained model was finally tested on a computer and a smartphone in real-time, where it demonstrated promising results.

scholarly journals Rank Pooling Approach for Wearable Sensor-Based ADLs Recognition

Sensors ◽  
2020 ◽  
Vol 20 (12) ◽  
pp. 3463 ◽  
Author(s):  
Muhammad Adeel Nisar ◽  
Kimiaki Shirahama ◽  
Frédéric Li ◽  
Xinyu Huang ◽  
Marcin Grzegorzek
Keyword(s):  
Hierarchical Model ◽  
Daily Living ◽  
The Other ◽  
Sensor Data ◽  
Wearable Sensor ◽  
Short Term ◽  
Large Dataset ◽  
Sensory Data ◽  
Atomic Actions

This paper addresses wearable-based recognition of Activities of Daily Living (ADLs) which are composed of several repetitive and concurrent short movements having temporal dependencies. It is improbable to directly use sensor data to recognize these long-term composite activities because two examples (data sequences) of the same ADL result in largely diverse sensory data. However, they may be similar in terms of more semantic and meaningful short-term atomic actions. Therefore, we propose a two-level hierarchical model for recognition of ADLs. Firstly, atomic activities are detected and their probabilistic scores are generated at the lower level. Secondly, we deal with the temporal transitions of atomic activities using a temporal pooling method, rank pooling. This enables us to encode the ordering of probabilistic scores for atomic activities at the higher level of our model. Rank pooling leads to a 5–13% improvement in results as compared to the other popularly used techniques. We also produce a large dataset of 61 atomic and 7 composite activities for our experiments.

scholarly journals AddGraph: Anomaly Detection in Dynamic Graph Using Attention-based Temporal GCN

2019 ◽  
Author(s):  
Li Zheng ◽  
Zhenpeng Li ◽  
Jian Li ◽  
Zhao Li ◽  
Jun Gao
Keyword(s):  
Anomaly Detection ◽  
Dynamic Graphs ◽  
Dynamic Graph ◽  
Short Term ◽  
Convolutional Network ◽  
Attention Model ◽  
Temporal Features ◽  
Structural Content ◽  
Real World Datasets

Anomaly detection in dynamic graphs becomes very critical in many different application scenarios, e.g., recommender systems, while it also raises huge challenges due to the high flexible nature of anomaly and lack of sufficient labelled data. It is better to learn the anomaly patterns by considering all possible features including the structural, content and temporal features, rather than utilizing heuristic rules over the partial features. In this paper, we propose AddGraph, a general end-to-end anomalous edge detection framework using an extended temporal GCN (Graph Convolutional Network) with an attention model, which can capture both long-term patterns and the short-term patterns in dynamic graphs. In order to cope with insufficient explicit labelled data, we employ the negative sampling and margin loss in training of AddGraph in a semi-supervised fashion. We conduct extensive experiments on real-world datasets, and illustrate that AddGraph can outperform the state-of-the-art competitors in anomaly detection significantly.

scholarly journals A Novel Framework for the Holistic Monitoring and Analysis of Human Behaviour

Proceedings ◽  
2019 ◽  
Vol 31 (1) ◽  
pp. 43
Author(s):  
Kostas Konsolakis ◽  
Hermie Hermens ◽  
Oresti Banos
Keyword(s):  
Holistic Approach ◽  
Human Behaviour ◽  
Sensor Data ◽  
Social Emotional ◽  
Short Term ◽  
Current Trends ◽  
Technological Advances ◽  
Cognitive Behaviour ◽  
Behavioural Aspects

Recent technological advances have enabled the continuous and unobtrusive monitoring of human behaviour. However, most of the existing studies focus on detecting human behaviour under the limitation of one behavioural aspect, such as physical behaviour and not addressing human behaviour in a broad sense. For this reason, we propose a novel framework that will serve as the principal generator of knowledge on the user’s behaviour. The proposed framework moves beyond the current trends in automatic behaviour analysis by detecting and inferring human behaviour automatically, based on multimodal sensor data. In particular, the framework analyses human behaviour in a holistic approach, focusing on different behavioural aspects at the same time; namely physical, social, emotional and cognitive behaviour. Furthermore, the suggested framework investigates user’s behaviour over different periods, introducing the concept of short-term and long-term behaviours and how these change over time.

Sign in / Sign up

Export Citation Format

Share Document