scholarly journals Movement Trajectory Recognition of Sign Language Based on Optimized Dynamic Time Warping

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1400
Author(s):  
Wenguo Li ◽  
Zhizeng Luo ◽  
Xugang Xi
Keyword(s):  
Sign Language ◽  
Dynamic Time Warping ◽  
Activation Function ◽  
Movement Trajectory ◽  
Time Warping ◽  
Translation Research ◽  
Multidimensional Time Series ◽  
The Difference ◽  
Dynamic Time ◽  
Shape Characteristics

Movement trajectory recognition is the key link of sign language (SL) translation research, which directly affects the accuracy of SL translation results. A new method is proposed for the accurate recognition of movement trajectory. First, the gesture motion information collected should be converted into a fixed coordinate system by the coordinate transformation. The SL movement trajectory is reconstructed using the adaptive Simpson algorithm to maintain the originality and integrity of the trajectory. The algorithm is then extended to multidimensional time series by using Mahalanobis distance (MD). The activation function of generalized linear regression (GLR) is modified to optimize the dynamic time warping (DTW) algorithm, which ensures that the local shape characteristics are considered for the global amplitude characteristics and avoids the problem of abnormal matching in the process of trajectory recognition. Finally, the similarity measure method is used to calculate the distance between two warped trajectories, to judge whether they are classified to the same category. Experimental results show that this method is effective for the recognition of SL movement trajectory, and the accuracy of trajectory recognition is 86.25%. The difference ratio between the inter-class features and intra-class features of the movement trajectory is 20, and the generalization ability of the algorithm can be effectively improved.

scholarly journals DESIGNING TRANSLATION TOOL: BETWEEN SIGN LANGUAGE TO SPOKEN TEXT ON KINECT TIME SERIES DATA USING DYNAMIC TIME WARPING

SINERGI ◽  
2018 ◽  
Vol 22 (2) ◽  
pp. 91
Author(s):  
Zico Pratama Putera ◽  
Mila Desi Anasanti ◽  
Bagus Priambodo
Keyword(s):  
User Interface ◽  
Sign Language ◽  
Dynamic Time Warping ◽  
User Study ◽  
Hearing Impaired ◽  
Microsoft Kinect ◽  
Series Data ◽  
Time Warping ◽  
Dynamic Gestures ◽  
Dynamic Time

The gesture is one of the most natural and expressive methods for the hearing impaired. Most researchers, however, focus on either static gestures, postures or a small group of dynamic gestures due to the complexity of dynamic gestures. We propose the Kinect Translation Tool to recognize the user's gesture. As a result, the Kinect Translation Tool can be used for bilateral communication with the deaf community. Since real-time detection of a large number of dynamic gestures is taken into account, some efficient algorithms and models are required. The dynamic time warping algorithm is used here to detect and translate the gesture. Kinect Sign Language should translate sign language into written and spoken words. Conversely, people can reply directly with their spoken word, which is converted into literal text together with the animated 3D sign language gestures. The user study, which included several prototypes of the user interface, was carried out with the observation of ten participants who had to gesture and spell the phrases in American Sign Language (ASL). The speech recognition tests for simple phrases have therefore shown good results. The system also recognized the participant's gesture very well during the test. The study suggested that a natural user interface with Microsoft Kinect could be interpreted as a sign language translator for the hearing impaired.

scholarly journals Modified Dynamic Time Warping Based on Direction Similarity for Fast Gesture Recognition

2018 ◽  
Vol 2018 ◽  
pp. 1-9 ◽  
Author(s):  
Hyo-Rim Choi ◽  
TaeYong Kim
Keyword(s):  
Gesture Recognition ◽  
Dynamic Time Warping ◽  
Spline Interpolation ◽  
Recognition System ◽  
Movement Direction ◽  
Time Warping ◽  
Suggested Algorithm ◽  
The Difference ◽  
Improved Performance ◽  
Dynamic Time

We propose a modified dynamic time warping (DTW) algorithm that compares gesture-position sequences based on the direction of the gestural movement. Standard DTW does not specifically consider the two-dimensional characteristic of the user’s movement. Therefore, in gesture recognition, the sequence comparison by standard DTW needs to be improved. The proposed gesture-recognition system compares the sequences of the input gesture’s position with gesture positions saved in the database and selects the most similar gesture by filtering out unrelated gestures. The suggested algorithm uses the cosine similarity of the movement direction at each moment to calculate the difference and reflects the characteristics of the gesture movement by using the ratio of the Euclidean distance and the proportional distance to the calculated difference. Selective spline interpolation assists in solving the issue of recognition-decline at instances of gestures. Through experiments with public databases (MSRC-12 and G3D), the suggested algorithm revealed an improved performance on both databases compared to other methods.

scholarly journals Do all aspects of learning benefit from iconicity? Evidence from motion capture

2019 ◽  
Vol 12 (1) ◽  
pp. 36-55
Author(s):  
ASHA SATO ◽  
MARIEKE SCHOUWSTRA ◽  
MOLLY FLAHERTY ◽  
SIMON KIRBY
Keyword(s):  
Language Learning ◽  
Sign Language ◽  
Motion Capture ◽  
Dynamic Time Warping ◽  
Negative Impact ◽  
Conclusive Evidence ◽  
Microsoft Kinect ◽  
Time Warping ◽  
3D Camera ◽  
Dynamic Time

abstractRecent work suggests that not all aspects of learning benefit from an iconicity advantage (Ortega, 2017). We present the results of an artificial sign language learning experiment testing the hypothesis that iconicity may help learners to learn mappings between forms and meanings, whilst having a negative impact on learning specific features of the form. We used a 3D camera (Microsoft Kinect) to capture participants’ gestures and quantify the accuracy with which they reproduce the target gestures in two conditions. In the iconic condition, participants were shown an artificial sign language consisting of congruent gesture–meaning pairs. In the arbitrary condition, the language consisted of non-congruent gesture–meaning pairs. We quantified the accuracy of participants’ gestures using dynamic time warping (Celebi et. al., 2013). Our results show that participants in the iconic condition learn mappings more successfully than participants in the arbitrary condition, but there is no difference in the accuracy with which participants reproduce the forms. While our work confirms that iconicity helps to establish form–meaning mappings, our study did not give conclusive evidence about the effect of iconicity on production; we suggest that iconicity may only have an impact on learning forms when these are complex.

scholarly journals Sign Language Recognition Using Wearable Electronics: Implementing k-Nearest Neighbors with Dynamic Time Warping and Convolutional Neural Network Algorithms

Sensors ◽  
2020 ◽  
Vol 20 (14) ◽  
pp. 3879 ◽  
Author(s):  
Giovanni Saggio ◽  
Pietro Cavallo ◽  
Mariachiara Ricci ◽  
Vito Errico ◽  
Jonathan Zea ◽  
...  
Keyword(s):  
Sign Language ◽  
Dynamic Time Warping ◽  
Parametric Method ◽  
Nearest Neighbors ◽  
Wearable Electronics ◽  
Language Recognition ◽  
Time Warping ◽  
Sign Language Recognition ◽  
K Nearest Neighbors ◽  
Dynamic Time

We propose a sign language recognition system based on wearable electronics and two different classification algorithms. The wearable electronics were made of a sensory glove and inertial measurement units to gather fingers, wrist, and arm/forearm movements. The classifiers were k-Nearest Neighbors with Dynamic Time Warping (that is a non-parametric method) and Convolutional Neural Networks (that is a parametric method). Ten sign-words were considered from the Italian Sign Language: cose, grazie, maestra, together with words with international meaning such as google, internet, jogging, pizza, television, twitter, and ciao. The signs were repeated one-hundred times each by seven people, five male and two females, aged 29–54 y ± 10.34 (SD). The adopted classifiers performed with an accuracy of 96.6% ± 3.4 (SD) for the k-Nearest Neighbors plus the Dynamic Time Warping and of 98.0% ± 2.0 (SD) for the Convolutional Neural Networks. Our system was made of wearable electronics among the most complete ones, and the classifiers top performed in comparison with other relevant works reported in the literature.

scholarly journals A first sketch: Construction of model defect priors inspired by dynamic time warping

2019 ◽  
Vol 211 ◽  
pp. 07005 ◽  
Author(s):  
Georg Schnabel ◽  
Henrik Sjöstrand
Keyword(s):  
Dynamic Time Warping ◽  
Nuclear Data ◽  
Evaluation Procedure ◽  
Time Warping ◽  
Model Defect ◽  
Nuclear Models ◽  
The Difference ◽  
Dynamic Time ◽  
Energy Dependent

Model defects are known to cause biased nuclear data evaluations if they are not taken into account in the evaluation procedure. We suggest a method to construct prior distributions for model defects for reaction models using neighboring isotopes of 56Fe as an example. A model defect is usually a function of energy and describes the difference between the model prediction and the truth. Of course, neither the truth nor the model defect are accessible. A Gaussian process (GP) enables to define a probability distribution on possible shapes of a model defect by referring to intuitively understandable concepts such as smoothness and the expected magnitude of the defect. Standard specifications of GPs impose a typical length-scale and amplitude valid for the whole energy range, which is often not justified, e.g., when the model covers both the resonance and statistical range. In this contribution, we show how a GP with energy-dependent length-scales and amplitudes can be constructed from available experimental data. The proposed construction is inspired by a technique called dynamic time warping used, e.g., for speech recognition. We demonstrate the feasibility of the data-driven determination of model defects by inferring a model defect of the nuclear models code TALYS for (n,p) reactions of isotopes with charge number between 20 and 30. The newly introduced GP parametrization besides its potential to improve evaluations for reactor relevant isotopes, such as 56Fe, may also help to better understand the performance of nuclear models in the future.

Sign in / Sign up

Export Citation Format

Share Document