Sign Language Interpreter using Deep Learning

Abstract: Sign Language is invaluable to hearing and speaking impaired people and is their only way of communicating among themselves. However, it has limitations with its reach as the rest of the people have no information regarding sign language interpretation. Sign language is communicated via hand gestures and visual modes and is therefore used by hearing and speaking impaired people to intercommunicate. These languages have alphabets and grammar of their own, which cannot be understood by people who have no knowledge about the specific symbols and rules. Thus, it has become essential for everyone to interpret, understand and communicate via sign language to overcome and alleviate the barriers of speech and communication. This can be tackled with the help of machine learning. This model is a Sign Language Interpreter that uses a dataset of images and interprets the sign language alphabets and sentences with 90.9% accuracy. For this paper, we have used an ASL (American Sign Language) Alphabet. We have used the CNN algorithm for this project. This paper ends with a summary of the model’s viability and its usefulness for interpretation of Sign Language. Keywords: Sign Language, Machine Learning, Interpretation model, Convoluted Neural Networks, American Sign Language

Download Full-text

An Innovative Method for Automatic American Sign Language Interpretation using Machine Learning and Leap Motion Controller

10.1109/uemcon53757.2021.9666640 ◽

2021 ◽

Author(s):

Jon Jenkins ◽

Sherif Rashad

Keyword(s):

Machine Learning ◽

American Sign Language ◽

Sign Language ◽

Leap Motion ◽

American Sign ◽

Motion Controller ◽

Leap Motion Controller ◽

Innovative Method ◽

Language Interpretation

Download Full-text

Linguistic Properties Based on American Sign Language Recognition with Artificial Neural Networks Using a Sensory Glove and Motion Tracker

Computational Intelligence and Bioinspired Systems - Lecture Notes in Computer Science ◽

10.1007/11494669_147 ◽

2005 ◽

pp. 1197-1205 ◽

Cited By ~ 2

Author(s):

Cemil Oz ◽

Ming C. Leu

Keyword(s):

Neural Networks ◽

Artificial Neural Networks ◽

American Sign Language ◽

Sign Language ◽

American Sign ◽

Language Recognition ◽

Sign Language Recognition ◽

Artificial Neural

Download Full-text

American Sign Language word recognition with a sensory glove using artificial neural networks

Engineering Applications of Artificial Intelligence ◽

10.1016/j.engappai.2011.06.015 ◽

2011 ◽

Vol 24 (7) ◽

pp. 1204-1213 ◽

Cited By ~ 90

Author(s):

Cemil Oz ◽

Ming C. Leu

Keyword(s):

Neural Networks ◽

Artificial Neural Networks ◽

Word Recognition ◽

American Sign Language ◽

Sign Language ◽

American Sign ◽

Artificial Neural

Download Full-text

Interpretation and Translation of American Sign Language for Hearing Impaired Individuals using Image Processing

International Journal of Recent Technology and Engineering - 2 ◽

10.35940/ijrte.d6966.118419 ◽

2019 ◽

Vol 8 (4) ◽

pp. 415-420

Keyword(s):

Image Processing ◽

American Sign Language ◽

Sign Language ◽

English Language ◽

Hearing Impaired ◽

Hand Gesture ◽

American Sign ◽

The Past ◽

Impaired People ◽

Live Image

The growth of technology has influenced development in various fields. Technology has helped people achieve their dreams over the past years. One such field that technology involves is aiding the hearing and speech impaired people. The obstruction between common individuals and individuals with hearing and language incapacities can be resolved by using the current technology to develop an environment such that the aforementioned easily communicate among one and other. ASL Interpreter aims to facilitate communication among the hearing and speech impaired individuals. This project mainly focuses on the development of software that can convert American Sign Language to Communicative English Language and vice-versa. This is accomplished via Image-Processing. The latter is a system that does a few activities on a picture, to acquire an improved picture or to extricate some valuable data from it. Image processing in this project is done by using MATLAB, software by MathWorks. The latter is programmed in a way that it captures the live image of the hand gesture. The captured gestures are put under the spotlight by being distinctively colored in contrast with the black background. The contrasted hand gesture will be delivered in the database as a binary equivalent of the location of each pixel and the interpreter would now link the binary value to its equivalent translation delivered in the database. This database shall be integrated into the mainframe image processing interface. The Image Processing toolbox, which is an inbuilt toolkit provided by MATLAB is used in the development of the software and Histogramic equivalents of the images are brought to the database and the extracted image will be converted to a histogram using the ‘imhist()’ function and would be compared with the same. The concluding phase of the project i.e. translation of speech to sign language is designed by matching the letter equivalent to the hand gesture in the database and displaying the result as images. The software will use a webcam to capture the hand gesture made by the user. This venture plans to facilitate the way toward learning gesture-based communication and supports hearing-impaired people to converse without trouble.

Download Full-text