Sign language recognition using dynamic time warping and hand shape distance based on histogram of oriented gradient features

2014 ◽  
Author(s):  
Pat Jangyodsuk ◽  
Christopher Conly ◽  
Vassilis Athitsos
Keyword(s):  
Sign Language ◽  
Dynamic Time Warping ◽  
Language Recognition ◽  
Time Warping ◽  
Sign Language Recognition ◽  
Histogram Of Oriented Gradient ◽  
Hand Shape ◽  
Shape Distance ◽  
Dynamic Time

scholarly journals Sign Language Recognition Using Wearable Electronics: Implementing k-Nearest Neighbors with Dynamic Time Warping and Convolutional Neural Network Algorithms

Sensors ◽  
2020 ◽  
Vol 20 (14) ◽  
pp. 3879 ◽  
Author(s):  
Giovanni Saggio ◽  
Pietro Cavallo ◽  
Mariachiara Ricci ◽  
Vito Errico ◽  
Jonathan Zea ◽  
...  
Keyword(s):  
Sign Language ◽  
Dynamic Time Warping ◽  
Parametric Method ◽  
Nearest Neighbors ◽  
Wearable Electronics ◽  
Language Recognition ◽  
Time Warping ◽  
Sign Language Recognition ◽  
K Nearest Neighbors ◽  
Dynamic Time

We propose a sign language recognition system based on wearable electronics and two different classification algorithms. The wearable electronics were made of a sensory glove and inertial measurement units to gather fingers, wrist, and arm/forearm movements. The classifiers were k-Nearest Neighbors with Dynamic Time Warping (that is a non-parametric method) and Convolutional Neural Networks (that is a parametric method). Ten sign-words were considered from the Italian Sign Language: cose, grazie, maestra, together with words with international meaning such as google, internet, jogging, pizza, television, twitter, and ciao. The signs were repeated one-hundred times each by seven people, five male and two females, aged 29–54 y ± 10.34 (SD). The adopted classifiers performed with an accuracy of 96.6% ± 3.4 (SD) for the k-Nearest Neighbors plus the Dynamic Time Warping and of 98.0% ± 2.0 (SD) for the Convolutional Neural Networks. Our system was made of wearable electronics among the most complete ones, and the classifiers top performed in comparison with other relevant works reported in the literature.

scholarly journals Chinese Sign Language Recognition Based on DTW-Distance-Mapping Features

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Juan Cheng ◽  
Fulin Wei ◽  
Yu Liu ◽  
Chang Li ◽  
Qiang Chen ◽  
...  
Keyword(s):  
Sign Language ◽  
Feature Representation ◽  
Communication Tool ◽  
Language Recognition ◽  
Time Warping ◽  
Sign Language Recognition ◽  
Key Points ◽  
Representation Method ◽  
Distance Mapping ◽  
Dynamic Time

Sign language is an important communication tool between the deaf and the external world. As the number of the Chinese deaf accounts for 15% of the world, it is highly urgent to develop a Chinese sign language recognition (CSLR) system. Recently, a novel phonology- and radical-coded CSL, taking advantages of a limited and constant number of coded gestures, has been preliminarily verified to be feasible for practical CSLR systems. The keynote of this version of CSL is that the same coded gesture performed in different orientations has different meanings. In this paper, we mainly propose a novel two-stage feature representation method to effectively characterize the CSL gestures. First, an orientation-sensitive feature is extracted regarding the distances between the palm center and the key points of the hand contour. Second, the extracted features are transformed by a dynamic time warping- (DTW-) based feature mapping approach for better representation. Experimental results demonstrate the effectiveness of the proposed feature extraction and mapping approaches. The averaged classification accuracy of all the 39 types of CSL gestures acquired from 11 subjects exceeds 93% for all the adopted classifiers, achieving significant improvement compared to the scheme without DTW-distance-mapping.

scholarly journals DESIGNING TRANSLATION TOOL: BETWEEN SIGN LANGUAGE TO SPOKEN TEXT ON KINECT TIME SERIES DATA USING DYNAMIC TIME WARPING

SINERGI ◽  
2018 ◽  
Vol 22 (2) ◽  
pp. 91
Author(s):  
Zico Pratama Putera ◽  
Mila Desi Anasanti ◽  
Bagus Priambodo
Keyword(s):  
User Interface ◽  
Sign Language ◽  
Dynamic Time Warping ◽  
User Study ◽  
Hearing Impaired ◽  
Microsoft Kinect ◽  
Series Data ◽  
Time Warping ◽  
Dynamic Gestures ◽  
Dynamic Time

The gesture is one of the most natural and expressive methods for the hearing impaired. Most researchers, however, focus on either static gestures, postures or a small group of dynamic gestures due to the complexity of dynamic gestures. We propose the Kinect Translation Tool to recognize the user's gesture. As a result, the Kinect Translation Tool can be used for bilateral communication with the deaf community. Since real-time detection of a large number of dynamic gestures is taken into account, some efficient algorithms and models are required. The dynamic time warping algorithm is used here to detect and translate the gesture. Kinect Sign Language should translate sign language into written and spoken words. Conversely, people can reply directly with their spoken word, which is converted into literal text together with the animated 3D sign language gestures. The user study, which included several prototypes of the user interface, was carried out with the observation of ten participants who had to gesture and spell the phrases in American Sign Language (ASL). The speech recognition tests for simple phrases have therefore shown good results. The system also recognized the participant's gesture very well during the test. The study suggested that a natural user interface with Microsoft Kinect could be interpreted as a sign language translator for the hearing impaired.

scholarly journals Movement Trajectory Recognition of Sign Language Based on Optimized Dynamic Time Warping

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1400
Author(s):  
Wenguo Li ◽  
Zhizeng Luo ◽  
Xugang Xi
Keyword(s):  
Sign Language ◽  
Dynamic Time Warping ◽  
Activation Function ◽  
Movement Trajectory ◽  
Time Warping ◽  
Translation Research ◽  
Multidimensional Time Series ◽  
The Difference ◽  
Dynamic Time ◽  
Shape Characteristics

Movement trajectory recognition is the key link of sign language (SL) translation research, which directly affects the accuracy of SL translation results. A new method is proposed for the accurate recognition of movement trajectory. First, the gesture motion information collected should be converted into a fixed coordinate system by the coordinate transformation. The SL movement trajectory is reconstructed using the adaptive Simpson algorithm to maintain the originality and integrity of the trajectory. The algorithm is then extended to multidimensional time series by using Mahalanobis distance (MD). The activation function of generalized linear regression (GLR) is modified to optimize the dynamic time warping (DTW) algorithm, which ensures that the local shape characteristics are considered for the global amplitude characteristics and avoids the problem of abnormal matching in the process of trajectory recognition. Finally, the similarity measure method is used to calculate the distance between two warped trajectories, to judge whether they are classified to the same category. Experimental results show that this method is effective for the recognition of SL movement trajectory, and the accuracy of trajectory recognition is 86.25%. The difference ratio between the inter-class features and intra-class features of the movement trajectory is 20, and the generalization ability of the algorithm can be effectively improved.

scholarly journals Do all aspects of learning benefit from iconicity? Evidence from motion capture

2019 ◽  
Vol 12 (1) ◽  
pp. 36-55
Author(s):  
ASHA SATO ◽  
MARIEKE SCHOUWSTRA ◽  
MOLLY FLAHERTY ◽  
SIMON KIRBY
Keyword(s):  
Language Learning ◽  
Sign Language ◽  
Motion Capture ◽  
Dynamic Time Warping ◽  
Negative Impact ◽  
Conclusive Evidence ◽  
Microsoft Kinect ◽  
Time Warping ◽  
3D Camera ◽  
Dynamic Time

abstractRecent work suggests that not all aspects of learning benefit from an iconicity advantage (Ortega, 2017). We present the results of an artificial sign language learning experiment testing the hypothesis that iconicity may help learners to learn mappings between forms and meanings, whilst having a negative impact on learning specific features of the form. We used a 3D camera (Microsoft Kinect) to capture participants’ gestures and quantify the accuracy with which they reproduce the target gestures in two conditions. In the iconic condition, participants were shown an artificial sign language consisting of congruent gesture–meaning pairs. In the arbitrary condition, the language consisted of non-congruent gesture–meaning pairs. We quantified the accuracy of participants’ gestures using dynamic time warping (Celebi et. al., 2013). Our results show that participants in the iconic condition learn mappings more successfully than participants in the arbitrary condition, but there is no difference in the accuracy with which participants reproduce the forms. While our work confirms that iconicity helps to establish form–meaning mappings, our study did not give conclusive evidence about the effect of iconicity on production; we suggest that iconicity may only have an impact on learning forms when these are complex.

Sign in / Sign up

Export Citation Format

Share Document