Feature fusion of triaxial acceleration signals and depth maps for human action recognition

2016 ◽  
Author(s):  
Yi Li ◽  
Jun Cheng ◽  
Wei Feng ◽  
Dapeng Tao
Keyword(s):  
Action Recognition ◽  
Feature Fusion ◽  
Human Action Recognition ◽  
Human Action ◽  
Depth Maps ◽  
Acceleration Signals

scholarly journals Using a Multilearner to Fuse Multimodal Features for Human Action Recognition

2020 ◽  
Vol 2020 ◽  
pp. 1-18
Author(s):  
Chao Tang ◽  
Huosheng Hu ◽  
Wenjian Wang ◽  
Wei Li ◽  
Hua Peng ◽  
...  
Keyword(s):  
Action Recognition ◽  
Feature Fusion ◽  
Human Action Recognition ◽  
Human Action ◽  
Human Motion ◽  
Image Features ◽  
Depth Image ◽  
Good Ability ◽  
Multimodal Features ◽  
Feature Based

The representation and selection of action features directly affect the recognition effect of human action recognition methods. Single feature is often affected by human appearance, environment, camera settings, and other factors. Aiming at the problem that the existing multimodal feature fusion methods cannot effectively measure the contribution of different features, this paper proposed a human action recognition method based on RGB-D image features, which makes full use of the multimodal information provided by RGB-D sensors to extract effective human action features. In this paper, three kinds of human action features with different modal information are proposed: RGB-HOG feature based on RGB image information, which has good geometric scale invariance; D-STIP feature based on depth image, which maintains the dynamic characteristics of human motion and has local invariance; and S-JRPF feature-based skeleton information, which has good ability to describe motion space structure. At the same time, multiple K-nearest neighbor classifiers with better generalization ability are used to integrate decision-making classification. The experimental results show that the algorithm achieves ideal recognition results on the public G3D and CAD60 datasets.

scholarly journals Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition

2019 ◽  
Vol 5 (10) ◽  
pp. 82 ◽  
Author(s):  
Mahmoud Al-Faris ◽  
John Chiverton ◽  
Yanyan Yang ◽  
David Ndzi
Keyword(s):  
Action Recognition ◽  
Spatial Information ◽  
Feature Fusion ◽  
Human Action Recognition ◽  
Human Action ◽  
Single Type ◽  
Weight Functions ◽  
Motion Model ◽  
Temporal Dimension ◽  
Depth Motion Maps

Human action recognition (HAR) is an important yet challenging task. This paper presents a novel method. First, fuzzy weight functions are used in computations of depth motion maps (DMMs). Multiple length motion information is also used. These features are referred to as fuzzy weighted multi-resolution DMMs (FWMDMMs). This formulation allows for various aspects of individual actions to be emphasized. It also helps to characterise the importance of the temporal dimension. This is important to help overcome, e.g., variations in time over which a single type of action might be performed. A deep convolutional neural network (CNN) motion model is created and trained to extract discriminative and compact features. Transfer learning is also used to extract spatial information from RGB and depth data using the AlexNet network. Different late fusion techniques are then investigated to fuse the deep motion model with the spatial network. The result is a spatial temporal HAR model. The developed approach is capable of recognising both human action and human–object interaction. Three public domain datasets are used to evaluate the proposed solution. The experimental results demonstrate the robustness of this approach compared with state-of-the art algorithms.

Sign in / Sign up

Export Citation Format

Share Document