scholarly journals Gesture recognition of sign language alphabet with a convolutional neural network using a magnetic positioning system

ACTA IMEKO ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 97
Author(s):  
Emanuele Buchicchio ◽  
Francesco Santoni ◽  
Alessio De Angelis ◽  
Antonio Moschitta ◽  
Paolo Carbone
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Gesture Recognition ◽  
Classification Model ◽  
Positioning System ◽  
Automated Translation ◽  
Learning Techniques ◽  
3D Positioning ◽  
Gesture Tracking ◽  
Model Training

<p class="Abstract"><span lang="EN-US">Gesture recognition is a fundamental step to enable efficient communication for the deaf through the automated translation of sign language. This work proposes the usage of a high-precision magnetic positioning system for 3D positioning and orientation tracking of the fingers and hands palm. The gesture is reconstructed by the MagIK (magnetic and inverse kinematics) method and then processed by a deep learning gesture classification model trained to recognize the gestures associated with the sign language alphabet. Results confirm the limits of vision-based systems and show that the proposed method based on hand skeleton reconstruction has good generalization properties. The proposed system, which combines sensor-based gesture acquisition and deep learning techniques for gesture recognition, provides a 100% classification accuracy, signer independent, after a few hours of training using transfer learning technique on well-known ResNet CNN architecture. The proposed classification model training method can be applied to other sensor-based gesture tracking systems and other applications, regardless of the specific data acquisition technology.</span></p>

scholarly journals Gesture Recognition of Sign Language Alphabet Using a Magnetic Positioning System

2021 ◽  
Vol 11 (12) ◽  
pp. 5594
Author(s):  
Matteo Rinalduzzi ◽  
Alessio De Angelis ◽  
Francesco Santoni ◽  
Emanuele Buchicchio ◽  
Antonio Moschitta ◽  
...  
Keyword(s):  
Sign Language ◽  
Gesture Recognition ◽  
Experimental Tests ◽  
Recognition System ◽  
Positioning System ◽  
Machine Learning Classification ◽  
Automated Translation ◽  
Measured Position ◽  
Position Data ◽  
Position And Orientation

Hand gesture recognition is a crucial task for the automated translation of sign language, which enables communication for the deaf. This work proposes the usage of a magnetic positioning system for recognizing the static gestures associated with the sign language alphabet. In particular, a magnetic positioning system, which is comprised of several wearable transmitting nodes, measures the 3D position and orientation of the fingers within an operating volume of about 30 × 30 × 30 cm, where receiving nodes are placed at known positions. Measured position data are then processed by a machine learning classification algorithm. The proposed system and classification method are validated by experimental tests. Results show that the proposed approach has good generalization properties and provides a classification accuracy of approximately 97% on 24 alphabet letters. Thus, the feasibility of the proposed gesture recognition system for the task of automated translation of the sign language alphabet for fingerspelling is proven.

KArSL

2021 ◽  
Vol 20 (1) ◽  
pp. 1-19
Author(s):  
Ala Addin I. Sidig ◽  
Hamzah Luqman ◽  
Sabri Mahmoud ◽  
Mohamed Mohandes
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Markov Models ◽  
Body Language ◽  
Arab Countries ◽  
Microsoft Kinect ◽  
Classification Model ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Arabic Sign Language

Sign language is the major means of communication for the deaf community. It uses body language and gestures such as hand shapes, lib patterns, and facial expressions to convey a message. Sign language is geography-specific, as it differs from one country to another. Arabic Sign language is used in all Arab countries. The availability of a comprehensive benchmarking database for ArSL is one of the challenges of the automatic recognition of Arabic Sign language. This article introduces KArSL database for ArSL, consisting of 502 signs that cover 11 chapters of ArSL dictionary. Signs in KArSL database are performed by three professional signers, and each sign is repeated 50 times by each signer. The database is recorded using state-of-art multi-modal Microsoft Kinect V2. We also propose three approaches for sign language recognition using this database. The proposed systems are Hidden Markov Models, deep learning images’ classification model applied on an image composed of shots of the video of the sign, and attention-based deep learning captioning system. Recognition accuracies of these systems indicate their suitability for such a large number of Arabic signs. The techniques are also tested on a publicly available database. KArSL database will be made freely available for interested researchers.

scholarly journals Underwater Gesture Recognition Using Classical Computer Vision and Deep Learning Techniques

2020 ◽  
Vol 8 (1) ◽  
pp. 9-14
Author(s):  
Mygel Andrei M. Martija ◽  
Jakov Ivan S. Dumbrique ◽  
Prospero C., Jr Naval
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Gesture Recognition ◽  
Learning Techniques ◽  
Classical Computer

Sign Language Gesture Recognition Using Doppler Radar and Deep Learning

2019 ◽  
Author(s):  
Hovannes Kulhandjian ◽  
Prakshi Sharma ◽  
Michel Kulhandjian ◽  
Claude D'Amours
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Gesture Recognition ◽  
Doppler Radar

scholarly journals Think Twice: A Post-Processing Approach for the Chinese Spelling Error Correction

2021 ◽  
Vol 11 (13) ◽  
pp. 5832
Author(s):  
Wei Gou ◽  
Zheng Chen
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Error Correction ◽  
Language Processing ◽  
Classification Model ◽  
Spelling Error ◽  
Learning Method ◽  
Post Processing ◽  
Rule Based ◽  
Model Training

Chinese Spelling Error Correction is a hot subject in the field of natural language processing. Researchers have already produced many great solutions, from the initial rule-based solution to the current deep learning method. At present, SpellGCN, proposed by Alibaba’s team, achieves the best results of which character level precision over SIGHAN2013 is 98.4%. However, when we apply this algorithm to practical error correction tasks, it produces many false error correction results. We believe that this is because the corpus used for model training contains significantly more errors than the text used for model correcting. In response to this problem, we propose performing a post-processing operation on the error correction tasks. We employ the initial model’s output as a candidate character, obtain various features of the character itself and its context, and then use a classification model to filter the initial model’s false error correction results. The post-processing idea introduced in this paper can apply to most Chinese Spelling Error Correction models to improve their performance over practical error correction tasks.

Sign in / Sign up

Export Citation Format

Share Document