A Review of Sign Language Recognition Techniques

2021 ◽  
Vol 10 (6) ◽  
pp. 1-8
Keyword(s):  
Computer Vision ◽  
Sign Language ◽  
Nonverbal Communication ◽  
Sensor Technology ◽  
Deaf Community ◽  
Language Recognition ◽  
Review Of The Literature ◽  
Sign Language Recognition ◽  
Hand Gestures ◽  
Computer Based

Sign language recognition is important for natural and convenient communication between deaf community and hearing majority. Hand gestures are a form of nonverbal communication that makes up the bulk of the communication between mute individuals, as sign language constitutes largely of hand gestures. Research works based on hand gestures have adopted many different techniques, including those based on instrumented sensor technology and computer vision. In other words, the hand sign can be classified under many headings, such as posture and gesture, as well as dynamic and static, or a hybrid of the two. This paper focuses on a review of the literature on computer based sign language recognition approaches, their motivations, techniques, observed limitations and suggestion for improvement.

Indian Sign Language Recognition on PYNQ Board

2020 ◽  
Vol 13 ◽  
Author(s):  
Sukhendra Singh ◽  
G. N. Rathna ◽  
Vivek Singhal
Keyword(s):  
Sign Language ◽  
Hand Gesture ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Hand Gestures ◽  
Depth Images ◽  
Kinect Camera ◽  
Impaired People ◽  
Web Camera ◽  
Indian Sign Language

Introduction: Sign language is the only way to communicate for speech-impaired people. But this sign language is not known to normal people so this is the cause of barrier in communicating. This is the problem faced by speech impaired people. In this paper, we have presented our solution which captured hand gestures with Kinect camera and classified the hand gesture into its correct symbol. Method: We used Kinect camera not the ordinary web camera because the ordinary camera does not capture its 3d orientation or depth of an image from camera however Kinect camera can capture 3d image and this will make classification more accurate. Result: Kinect camera will produce a different image for hand gestures for ‘2’ and ‘V’ and similarly for ‘1’ and ‘I’ however, normal web camera will not be able to distinguish between these two. We used hand gesture for Indian sign language and our dataset had 46339, RGB images and 46339 depth images. 80% of the total images were used for training and the remaining 20% for testing. In total 36 hand gestures were considered to capture alphabets and alphabets from A-Z and 10 for numeric, 26 for digits from 0-9 were considered to capture alphabets and Keywords. Conclusion: Along with real-time implementation, we have also shown the comparison of the performance of the various machine learning models in which we have found out the accuracy of CNN on depth- images has given the most accurate performance than other models. All these resulted were obtained on PYNQ Z2 board.

scholarly journals Global-Local Enhancement Network for NMF-Aware Sign Language Recognition

2021 ◽  
Vol 17 (3) ◽  
pp. 1-19
Author(s):  
Hezhen Hu ◽  
Wengang Zhou ◽  
Junfu Pu ◽  
Houqiang Li
Keyword(s):  
Sign Language ◽  
Daily Life ◽  
The Other ◽  
Challenging Problem ◽  
Language Recognition ◽  
Local Enhancement ◽  
Sign Language Recognition ◽  
Vocabulary Size ◽  
Hand Gestures ◽  
Fine Grained

Sign language recognition (SLR) is a challenging problem, involving complex manual features (i.e., hand gestures) and fine-grained non-manual features (NMFs) (i.e., facial expression, mouth shapes, etc .). Although manual features are dominant, non-manual features also play an important role in the expression of a sign word. Specifically, many sign words convey different meanings due to non-manual features, even though they share the same hand gestures. This ambiguity introduces great challenges in the recognition of sign words. To tackle the above issue, we propose a simple yet effective architecture called Global-Local Enhancement Network (GLE-Net), including two mutually promoted streams toward different crucial aspects of SLR. Of the two streams, one captures the global contextual relationship, while the other stream captures the discriminative fine-grained cues. Moreover, due to the lack of datasets explicitly focusing on this kind of feature, we introduce the first non-manual-feature-aware isolated Chinese sign language dataset (NMFs-CSL) with a total vocabulary size of 1,067 sign words in daily life. Extensive experiments on NMFs-CSL and SLR500 datasets demonstrate the effectiveness of our method.

Sign Language Recognition Using Deep Learning and Computer Vision

2020 ◽  
Vol 12 (05-SPECIAL ISSUE) ◽  
pp. 964-968
Author(s):  
Sabeenian R.S ◽  
S. Sai Bharathwaj ◽  
M. Mohamed Aadhil
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Sign Language ◽  
Language Recognition ◽  
Sign Language Recognition

scholarly journals Indian Sign Language Recognition through Hybrid ConvNet-LSTM Networks

2021 ◽  
Vol 9 (1) ◽  
pp. 182-203
Author(s):  
Muthu Mariappan H ◽  
Dr Gomathi V
Keyword(s):  
Neural Network ◽  
Computer Vision ◽  
Real Time ◽  
Sign Language ◽  
Gesture Recognition ◽  
Language Translation ◽  
Video Gaming ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Indian Sign Language

Dynamic hand gesture recognition is a challenging task of Human-Computer Interaction (HCI) and Computer Vision. The potential application areas of gesture recognition include sign language translation, video gaming, video surveillance, robotics, and gesture-controlled home appliances. In the proposed research, gesture recognition is applied to recognize sign language words from real-time videos. Classifying the actions from video sequences requires both spatial and temporal features. The proposed system handles the former by the Convolutional Neural Network (CNN), which is the core of several computer vision solutions and the latter by the Recurrent Neural Network (RNN), which is more efficient in handling the sequences of movements. Thus, the real-time Indian sign language (ISL) recognition system is developed using the hybrid CNN-RNN architecture. The system is trained with the proposed CasTalk-ISL dataset. The ultimate purpose of the presented research is to deploy a real-time sign language translator to break the hurdles present in the communication between hearing-impaired people and normal people. The developed system achieves 95.99% top-1 accuracy and 99.46% top-3 accuracy on the test dataset. The obtained results outperform the existing approaches using various deep models on different datasets.

scholarly journals Template Matching Based Sign Language Recognition System for Android Devices

2020 ◽  
Vol 5 (1) ◽  
Author(s):  
Kudirat O Jimoh ◽  
Anuoluwapo O Ajayi ◽  
Ibrahim K Ogundoyin
Keyword(s):  
Feature Extraction ◽  
Sign Language ◽  
Template Matching ◽  
Principal Component ◽  
Recognition System ◽  
Generative Adversarial Networks ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Hand Gestures ◽  
Classification Rate

An android based sign language recognition system for selected English vocabularies was developed with the explicit objective to examine the specific characteristics that are responsible for gestures recognition. Also, a recognition model for the process was designed, implemented, and evaluated on 230 samples of hand gestures.  The collected samples were pre-processed and rescaled from 3024 ×4032 pixels to 245 ×350 pixels. The samples were examined for the specific characteristics using Oriented FAST and Rotated BRIEF, and the Principal Component Analysis used for feature extraction. The model was implemented in Android Studio using the template matching algorithm as its classifier. The performance of the system was evaluated using precision, recall, and accuracy as metrics. It was observed that the system obtained an average classification rate of 87%, an average precision value of 88% and 91% for the average recall rate on the test data of hand gestures.  The study, therefore, has successfully classified hand gestures for selected English vocabularies. The developed system will enhance the communication skills between hearing and hearing-impaired people, and also aid their teaching and learning processes. Future work include exploring state-of-the-art machining learning techniques such Generative Adversarial Networks (GANs) for large dataset to improve the accuracy of results. Keywords— Feature extraction; Gestures Recognition; Sign Language; Vocabulary, Android device.

scholarly journals AUTOMATIC LIP-READING OF HEARING IMPAIRED PEOPLE

2019 ◽  
Vol XLII-2/W12 ◽  
pp. 97-101
Author(s):  
D. Ivanko ◽  
D. Ryumin ◽  
A. Karpov
Keyword(s):  
Sign Language ◽  
Assistive Technologies ◽  
Hearing Impaired ◽  
Visual Speech ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Hand Gestures ◽  
Human Machine Interaction ◽  
Lip Reading ◽  
Impaired People

<p><strong>Abstract.</strong> Inability to use speech interfaces greatly limits the deaf and hearing impaired people in the possibility of human-machine interaction. To solve this problem and to increase the accuracy and reliability of the automatic Russian sign language recognition system it is proposed to use lip-reading in addition to hand gestures recognition. Deaf and hearing impaired people use sign language as the main way of communication in everyday life. Sign language is a structured form of hand gestures and lips movements involving visual motions and signs, which is used as a communication system. Since sign language includes not only hand gestures, but also lip movements that mimic vocalized pronunciation, it is of interest to investigate how accurately such a visual speech can be recognized by a lip-reading system, especially considering the fact that the visual speech of hearing impaired people is often characterized with hyper-articulation, which should potentially facilitate its recognition. For this purpose, thesaurus of Russian sign language (TheRusLan) collected in SPIIRAS in 2018–19 was used. The database consists of color optical FullHD video recordings of 13 native Russian sign language signers (11 females and 2 males) from “Pavlovsk boarding school for the hearing impaired”. Each of the signers demonstrated 164 phrases for 5 times. This work covers the initial stages of this research, including data collection, data labeling, region-of-interest detection and methods for informative features extraction. The results of this study can later be used to create assistive technologies for deaf or hearing impaired people.</p>

Sign in / Sign up

Export Citation Format

Share Document