scholarly journals Enhanced Multi-Channel Feature Synthesis for Hand Gesture Recognition Based on CNN With a Channel and Spatial Attention Mechanism

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 144610-144620
Author(s):  
Chuan Du ◽  
Lei Zhang ◽  
Xiping Sun ◽  
Junxu Wang ◽  
Jialian Sheng
Keyword(s):  
Spatial Attention ◽  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Attention Mechanism ◽  
Hand Gesture

scholarly journals Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Jinghua Li ◽  
Runze Liu ◽  
Dehui Kong ◽  
Shaofan Wang ◽  
Lichun Wang ◽  
...  
Keyword(s):  
Knowledge Transfer ◽  
Spatial Attention ◽  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Attention Mechanism ◽  
Feature Representation ◽  
Hand Gesture ◽  
Complementary Information ◽  
Adaptive Parameter ◽  
Positive Knowledge

Hand gesture recognition is a challenging topic in the field of computer vision. Multimodal hand gesture recognition based on RGB-D is with higher accuracy than that of only RGB or depth. It is not difficult to conclude that the gain originates from the complementary information existing in the two modalities. However, in reality, multimodal data are not always easy to acquire simultaneously, while unimodal RGB or depth hand gesture data are more general. Therefore, one hand gesture system is expected, in which only unimordal RGB or Depth data is supported for testing, while multimodal RGB-D data is available for training so as to attain the complementary information. Fortunately, a kind of method via multimodal training and unimodal testing has been proposed. However, unimodal feature representation and cross-modality transfer still need to be further improved. To this end, this paper proposes a new 3D-Ghost and Spatial Attention Inflated 3D ConvNet (3DGSAI) to extract high-quality features for each modality. The baseline of 3DGSAI network is Inflated 3D ConvNet (I3D), and two main improvements are proposed. One is 3D-Ghost module, and the other is the spatial attention mechanism. The 3D-Ghost module can extract richer features for hand gesture representation, and the spatial attention mechanism makes the network pay more attention to hand region. This paper also proposes an adaptive parameter for positive knowledge transfer, which ensures that the transfer always occurs from the strong modality network to the weak one. Extensive experiments on SKIG, VIVA, and NVGesture datasets demonstrate that our method is competitive with the state of the art. Especially, the performance of our method reaches 97.87% on the SKIG dataset using only RGB, which is the current best result.

Connectionist Temporal Classification Model for Dynamic Hand Gesture Recognition using RGB and Optical flow Data

2020 ◽  
Vol 17 (4) ◽  
pp. 497-506
Author(s):  
Sunil Patel ◽  
Ramji Makwana
Keyword(s):  
Neural Network ◽  
Optical Flow ◽  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Classification Model ◽  
Hand Gesture ◽  
Flow Data ◽  
Dynamic Hand Gesture Recognition ◽  
Connectionist Temporal Classification

Automatic classification of dynamic hand gesture is challenging due to the large diversity in a different class of gesture, Low resolution, and it is performed by finger. Due to a number of challenges many researchers focus on this area. Recently deep neural network can be used for implicit feature extraction and Soft Max layer is used for classification. In this paper, we propose a method based on a two-dimensional convolutional neural network that performs detection and classification of hand gesture simultaneously from multimodal Red, Green, Blue, Depth (RGBD) and Optical flow Data and passes this feature to Long-Short Term Memory (LSTM) recurrent network for frame-to-frame probability generation with Connectionist Temporal Classification (CTC) network for loss calculation. We have calculated an optical flow from Red, Green, Blue (RGB) data for getting proper motion information present in the video. CTC model is used to efficiently evaluate all possible alignment of hand gesture via dynamic programming and check consistency via frame-to-frame for the visual similarity of hand gesture in the unsegmented input stream. CTC network finds the most probable sequence of a frame for a class of gesture. The frame with the highest probability value is selected from the CTC network by max decoding. This entire CTC network is trained end-to-end with calculating CTC loss for recognition of the gesture. We have used challenging Vision for Intelligent Vehicles and Applications (VIVA) dataset for dynamic hand gesture recognition captured with RGB and Depth data. On this VIVA dataset, our proposed hand gesture recognition technique outperforms competing state-of-the-art algorithms and gets an accuracy of 86%

Feature Fusion Based Hand Gesture Recognition Method for Automotive Interfaces

2020 ◽  
Vol 29 (6) ◽  
pp. 1153-1164
Author(s):  
Qianyi Xu ◽  
Guihe Qin ◽  
Minghui Sun ◽  
Jie Yan ◽  
Huiming Jiang ◽  
...  
Keyword(s):  
Gesture Recognition ◽  
Feature Fusion ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Recognition Method

Normalized Edge Convolutional Networks for Skeleton-based Hand Gesture Recognition

2021 ◽  
pp. 108044
Author(s):  
Fangtai Guo ◽  
Zaixing He ◽  
Shuyou Zhang ◽  
Xinyue Zhao ◽  
Jinhui Fang ◽  
...  
Keyword(s):  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Convolutional Networks

Deep-Learning for Hand-Gesture Recognition with Simultaneous Thermal and Radar Sensors

2020 ◽  
Author(s):  
Sruthy Skaria ◽  
Da Huang ◽  
Akram Al-Hourani ◽  
Robin J. Evans ◽  
Margaret Lech
Keyword(s):  
Deep Learning ◽  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Radar Sensors
Sign in / Sign up

Export Citation Format

Share Document