scholarly journals Chinese Sign Language Recognition Based on DTW-Distance-Mapping Features

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Juan Cheng ◽  
Fulin Wei ◽  
Yu Liu ◽  
Chang Li ◽  
Qiang Chen ◽  
...  
Keyword(s):  
Sign Language ◽  
Feature Representation ◽  
Communication Tool ◽  
Language Recognition ◽  
Time Warping ◽  
Sign Language Recognition ◽  
Key Points ◽  
Representation Method ◽  
Distance Mapping ◽  
Dynamic Time

Sign language is an important communication tool between the deaf and the external world. As the number of the Chinese deaf accounts for 15% of the world, it is highly urgent to develop a Chinese sign language recognition (CSLR) system. Recently, a novel phonology- and radical-coded CSL, taking advantages of a limited and constant number of coded gestures, has been preliminarily verified to be feasible for practical CSLR systems. The keynote of this version of CSL is that the same coded gesture performed in different orientations has different meanings. In this paper, we mainly propose a novel two-stage feature representation method to effectively characterize the CSL gestures. First, an orientation-sensitive feature is extracted regarding the distances between the palm center and the key points of the hand contour. Second, the extracted features are transformed by a dynamic time warping- (DTW-) based feature mapping approach for better representation. Experimental results demonstrate the effectiveness of the proposed feature extraction and mapping approaches. The averaged classification accuracy of all the 39 types of CSL gestures acquired from 11 subjects exceeds 93% for all the adopted classifiers, achieving significant improvement compared to the scheme without DTW-distance-mapping.

scholarly journals Sign Language Recognition Using Wearable Electronics: Implementing k-Nearest Neighbors with Dynamic Time Warping and Convolutional Neural Network Algorithms

Sensors ◽  
2020 ◽  
Vol 20 (14) ◽  
pp. 3879 ◽  
Author(s):  
Giovanni Saggio ◽  
Pietro Cavallo ◽  
Mariachiara Ricci ◽  
Vito Errico ◽  
Jonathan Zea ◽  
...  
Keyword(s):  
Sign Language ◽  
Dynamic Time Warping ◽  
Parametric Method ◽  
Nearest Neighbors ◽  
Wearable Electronics ◽  
Language Recognition ◽  
Time Warping ◽  
Sign Language Recognition ◽  
K Nearest Neighbors ◽  
Dynamic Time

We propose a sign language recognition system based on wearable electronics and two different classification algorithms. The wearable electronics were made of a sensory glove and inertial measurement units to gather fingers, wrist, and arm/forearm movements. The classifiers were k-Nearest Neighbors with Dynamic Time Warping (that is a non-parametric method) and Convolutional Neural Networks (that is a parametric method). Ten sign-words were considered from the Italian Sign Language: cose, grazie, maestra, together with words with international meaning such as google, internet, jogging, pizza, television, twitter, and ciao. The signs were repeated one-hundred times each by seven people, five male and two females, aged 29–54 y ± 10.34 (SD). The adopted classifiers performed with an accuracy of 96.6% ± 3.4 (SD) for the k-Nearest Neighbors plus the Dynamic Time Warping and of 98.0% ± 2.0 (SD) for the Convolutional Neural Networks. Our system was made of wearable electronics among the most complete ones, and the classifiers top performed in comparison with other relevant works reported in the literature.

scholarly journals Multi-Stroke Thai Finger-Spelling Sign Language Recognition System with Deep Learning

Symmetry ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 262
Author(s):  
Thongpan Pariwat ◽  
Pusadee Seresangtakul
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Semantic Segmentation ◽  
Recognition System ◽  
Communication Tool ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Hand Segmentation ◽  
Average Accuracy ◽  
Filter Size

Sign language is a type of language for the hearing impaired that people in the general public commonly do not understand. A sign language recognition system, therefore, represents an intermediary between the two sides. As a communication tool, a multi-stroke Thai finger-spelling sign language (TFSL) recognition system featuring deep learning was developed in this study. This research uses a vision-based technique on a complex background with semantic segmentation performed with dilated convolution for hand segmentation, hand strokes separated using optical flow, and learning feature and classification done with convolution neural network (CNN). We then compared the five CNN structures that define the formats. The first format was used to set the number of filters to 64 and the size of the filter to 3 × 3 with 7 layers; the second format used 128 filters, each filter 3 × 3 in size with 7 layers; the third format used the number of filters in ascending order with 7 layers, all of which had an equal 3 × 3 filter size; the fourth format determined the number of filters in ascending order and the size of the filter based on a small size with 7 layers; the final format was a structure based on AlexNet. As a result, the average accuracy was 88.83%, 87.97%, 89.91%, 90.43%, and 92.03%, respectively. We implemented the CNN structure based on AlexNet to create models for multi-stroke TFSL recognition systems. The experiment was performed using an isolated video of 42 Thai alphabets, which are divided into three categories consisting of one stroke, two strokes, and three strokes. The results presented an 88.00% average accuracy for one stroke, 85.42% for two strokes, and 75.00% for three strokes.

scholarly journals Indonesian sign language recognition using kinect and dynamic time warping

2019 ◽  
Vol 15 (1) ◽  
pp. 495
Author(s):  
Wijayanti Nurul Khotimah ◽  
Nanik Suciati ◽  
Tiara Anggita
Keyword(s):  
Feature Extraction ◽  
Sign Language ◽  
Sequence Data ◽  
Recognition System ◽  
Classification Method ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Feature Extraction Method ◽  
Dependent Sequence ◽  
Dynamic Time

Sign Language Recognition System (SLRS) is a system to recognise sign language and then translate them into text. This system can be developed by using a sensor-based technique. Some studies have implemented various feature extraction and classification methods to recognise sign language in the different country. However, their systems were user dependent (the accuracy was high when the trained and the tested user were the same people, but it was getting worse when the tested user was different to the trained user). Therefore in this study, we proposed a feature extraction method which is invariant to a user. We used the distance between two users’ skeleton instead of using the users’ skeleton positions because the skeleton distance is independent to the user posture. Finally, forty-five features were extracted in this proposed method. Further, we classified the features by using a classification method that is suitable with sign language gestures characteristic (time-dependent sequence data). The classification method is Dynamic Time Wrapping. For the experiment, we used twenty Indonesian sign languages from different semantic groups (greetings, questions, pronouns, places, family and others) and different gesture characteristic (static gesture and dynamic gesture). Then the system was tested by a different user with the user who did the training. The result was promising, this proposed method produced high accuracy, reach 91% which shows that this proposed method is user independent.

Sign in / Sign up

Export Citation Format

Share Document