scholarly journals Lattice Long Short-Term Memory for Human Action Recognition

2017 ◽  
Author(s):  
Lin Sun ◽  
Kui Jia ◽  
Kevin Chen ◽  
Dit Yan Yeung ◽  
Bertram E. Shi ◽  
...  
Keyword(s):  
Action Recognition ◽  
Short Term Memory ◽  
Human Action Recognition ◽  
Human Action ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

scholarly journals Convolutional Long Short-Term Memory Hybrid Networks for Skeletal Based Human Action Recognition

2020 ◽  
Vol 9 (3) ◽  
pp. 955-961
Keyword(s):  
Time Series ◽  
Action Recognition ◽  
Short Term Memory ◽  
Human Action Recognition ◽  
Human Action ◽  
Short Term ◽  
Term Memory ◽  
Temporal Modeling ◽  
Long Short Term Memory ◽  
Spatio Temporal

The objective is to develop a time series image representation of the skeletal action data and use it for recognition through a convolutional long short-term deep learning framework. Consequently, Kinect captured human skeletal data is transformed into a Joint Change Distance Image (JCDI) descriptor which maps the time changes in the joints. Subsequently, JCDIs are decoded spatially well with a Convolutional (CNN). Temporal decomposition is executed on long short term memory (LSTM) with data changes along x , y and z position vectors of the skeleton. We propose a combination of CNN and LSTM which maps the spatio temporal information to generate a generalized time series features for recognition. Finally, scores are fused from spatially vibrant CNNs and temporally sound LSTMs for action recognition. Publicly available action datasets such as NTU RGBD, MSR Action, UTKinect and G3D were used as test inputs for experimentation. The results showed a better performance due to spatio temporal modeling at both the representation and the recognition stages when compared to other state-of-the-arts

scholarly journals Human Action Recognition in Videos using Convolution Long Short-Term Memory Network with Spatio-Temporal Networks

2021 ◽  
Vol 5 (1) ◽  
pp. 25-33
Author(s):  
Ashok Sarabu ◽  
Ajit Kumar Santra
Keyword(s):  
Neural Networks ◽  
Action Recognition ◽  
Data Augmentation ◽  
Short Term Memory ◽  
Human Action Recognition ◽  
Human Action ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Spatio Temporal

Two-stream convolutional networks plays an essential role as a powerful feature extractor in human action recognition in videos. Recent studies have shown the importance of two-stream Convolutional Neural Networks (CNN) to recognize human action recognition. Recurrent Neural Networks (RNN) has achieved the best performance in video activity recognition combining CNN. Encouraged by CNN's results with RNN, we present a two-stream network with two CNNs and Convolution Long-Short Term Memory (CLSTM). First, we extricate Spatio-temporal features using two CNNs using pre-trained ImageNet models. Second, the results of two CNNs from step one are combined and fed as input to the CLSTM to get the overall classification score. We also explored the various fusion function performance that combines two CNNs and the effects of feature mapping at different layers. And, conclude the best fusion function along with layer number. To avoid the problem of overfitting, we adopt the data augmentation techniques. Our proposed model demonstrates a substantial improvement compared to the current two-stream methods on the benchmark datasets with 70.4% on HMDB-51 and 95.4% on UCF-101 using the pre-trained ImageNet model. Doi: 10.28991/esj-2021-01254 Full Text: PDF

scholarly journals Action Recognition Algorithm of Spatio–Temporal Differential LSTM Based on Feature Enhancement

2021 ◽  
Vol 11 (17) ◽  
pp. 7876
Author(s):  
Kai Hu ◽  
Fei Zheng ◽  
Liguo Weng ◽  
Yiwu Ding ◽  
Junlan Jin
Keyword(s):  
Spatial Memory ◽  
Action Recognition ◽  
Short Term Memory ◽  
Human Action ◽  
Short Term ◽  
Memory State ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Spatio Temporal ◽  
Lstm Network

The Long Short-Term Memory (LSTM) network is a classic action recognition method because of its ability to extract time information. Researchers proposed many hybrid algorithms based on LSTM for human action recognition. In this paper, an improved Spatio–Temporal Differential Long Short-Term Memory (ST-D LSTM) network is proposed, an enhanced input differential feature module and a spatial memory state differential module are added to the network. Furthermore, a transmission mode of ST-D LSTM is proposed; this mode enables ST-D LSTM units to transmit the spatial memory state horizontally. Finally, these improvements are added into classical Long-term Recurrent Convolutional Networks (LRCN) to test the new network’s performance. Experimental results show that ST-D LSTM can effectively improve the accuracy of LRCN.

scholarly journals Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 411
Author(s):  
Yunkai Zhang ◽  
Yinghong Tian ◽  
Pingyi Wu ◽  
Dongfan Chen
Keyword(s):  
Autism Spectrum Disorder ◽  
Action Recognition ◽  
Short Term Memory ◽  
Autism Spectrum ◽  
Spectrum Disorder ◽  
Short Term ◽  
Term Memory ◽  
Clinical Observations ◽  
The Neural Network ◽  
Long Short Term Memory

The recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognition process of stereotyped actions, a method based on skeleton data and Long Short-Term Memory (LSTM) is proposed in this paper. In the first stage of our method, the OpenPose algorithm is used to obtain the initial skeleton data from the video of ASD children. Furthermore, four denoising methods are proposed to eliminate the noise of the initial skeleton data. In the second stage, we track multiple ASD children in the same scene by matching distance between current skeletons and previous skeletons. In the last stage, the neural network based on LSTM is proposed to classify the ASD children’s actions. The performed experiments show that our proposed method is effective for ASD children’s action recognition. Compared to the previous traditional schemes, our scheme has higher accuracy and is almost non-invasive for ASD children.

Deep historical long short-term memory network for action recognition

2020 ◽  
Vol 407 ◽  
pp. 428-438
Author(s):  
Jiaxin Cai ◽  
Junlin Hu ◽  
Xin Tang ◽  
Tzu-Yi Hung ◽  
Yap-Peng Tan
Keyword(s):  
Action Recognition ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Memory Network ◽  
Long Short Term Memory
Sign in / Sign up

Export Citation Format

Share Document