scholarly journals Indian sign language interpreter using image processing and machine learning

2020 ◽  
Vol 872 ◽  
pp. 012026
Author(s):  
Shubhendu Apoorv ◽  
Sudharshan Kumar Bhowmick ◽  
R Sakthi Prabha
Keyword(s):  
Machine Learning ◽  
Image Processing ◽  
Sign Language ◽  
Sign Language Interpreter ◽  
Indian Sign Language

scholarly journals Sign Language Interpreter using Deep Learning

2021 ◽  
Vol 9 (9) ◽  
pp. 2129-2132
Author(s):  
Rachaell Nihalaani
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
American Sign Language ◽  
Sign Language ◽  
American Sign ◽  
The People ◽  
Language Interpretation ◽  
Impaired People ◽  
Sign Language Interpreter ◽  
Interpretation Model

Abstract: Sign Language is invaluable to hearing and speaking impaired people and is their only way of communicating among themselves. However, it has limitations with its reach as the rest of the people have no information regarding sign language interpretation. Sign language is communicated via hand gestures and visual modes and is therefore used by hearing and speaking impaired people to intercommunicate. These languages have alphabets and grammar of their own, which cannot be understood by people who have no knowledge about the specific symbols and rules. Thus, it has become essential for everyone to interpret, understand and communicate via sign language to overcome and alleviate the barriers of speech and communication. This can be tackled with the help of machine learning. This model is a Sign Language Interpreter that uses a dataset of images and interprets the sign language alphabets and sentences with 90.9% accuracy. For this paper, we have used an ASL (American Sign Language) Alphabet. We have used the CNN algorithm for this project. This paper ends with a summary of the model’s viability and its usefulness for interpretation of Sign Language. Keywords: Sign Language, Machine Learning, Interpretation model, Convoluted Neural Networks, American Sign Language

Conversation Engine for Deaf and Dumb

2021 ◽  
pp. 107-112
Keyword(s):  
Machine Learning ◽  
Feature Extraction ◽  
Sign Language ◽  
Supervised Machine Learning ◽  
Deaf People ◽  
Hand Gestures ◽  
Sign Languages ◽  
Normal People ◽  
Machine Learning Model ◽  
Indian Sign Language

There are a lot of people who have many disabilities in our world out of which,people who are deaf and dumb cannot convey there messages to the normal people. Conversation becomes very difficult for this people. Deaf people cannot understand and hear what normal people is going to convey ,similarly dumb people need to convey their message using sign languages where normal people cannot understand unless he/she knows or understands the sign language. This brings to a need of an application which can be useful for having conversation between deaf,dumb and normal people. Here we are using hand gestures of Indian sign language (ISL) which contain all the alphabets and 0-9 digit gestures. The dataset of alphabets and digits is created by us.After dataset building we extracted the features using bagof- words and image preprocessing.With the feature extraction, histograms are been generated which maps alphabets to images. Finally, these features are fed to the supervised machine learning model to predict the gesture/sign. We did also use CNN model for training the model.

scholarly journals Recognition of Russian and Indian Sign Languages Based on Machine Learning

2021 ◽  
pp. 53-74
Author(s):  
Mikhail G. Grif ◽  
◽  
R. Elakkiya ◽  
Alexey L. Prikhodko ◽  
Maxim А. Bakaev ◽  
...  
Keyword(s):  
Machine Learning ◽  
Sign Language ◽  
Network Architecture ◽  
Head Tilt ◽  
Recognition System ◽  
Recurrent Network ◽  
The Body ◽  
Sign Languages ◽  
Trained Neural Network ◽  
Indian Sign Language

In the paper, we consider recognition of sign languages (SL) with a particular focus on Russian and Indian SLs. The proposed recognition system includes five components: configuration, orientation, localization, movement and non-manual markers. The analysis uses methods of recognition of individual gestures and continuous sign speech for Indian and Russian sign languages (RSL). To recognize individual gestures, the RSL Dataset was developed, which includes more than 35,000 files for over 1000 signs. Each sign was performed with 5 repetitions and at least by 5 deaf native speakers of the Russian Sign Language from Siberia. To isolate epenthesis for continuous RSL, 312 sentences with 5 repetitions were selected and recorded on video. Five types of movements were distinguished, namely, "No gesture", "There is a gesture", "Initial movement", "Transitional movement", "Final movement". The markup of sentences for highlighting epenthesis was carried out on the Supervisely.ly platform. A recurrent network architecture (LSTM) was built, implemented using the TensorFlow Keras machine learning library. The accuracy of correct recognition of epenthesis was 95 %. The work on a similar dataset for the recognition of both individual gestures and continuous Indian sign language (ISL) is continuing. To recognize hand gestures, the mediapipe holistic library module was used. It contains a group of trained neural network algorithms that allow obtaining the coordinates of the key points of the body, palms and face of a person in the image. The accuracy of 85 % was achieved for the verification data. In the future, it is necessary to significantly increase the amount of labeled data. To recognize non-manual components, a number of rules have been developed for certain movements in the face. These rules include positions for the eyes, eyelids, mouth, tongue, and head tilt.

scholarly journals Duplex Sign Language Communicator

2021 ◽  
Vol 9 (VI) ◽  
pp. 958-961
Author(s):  
Prof. Prema Sahane
Keyword(s):  
Machine Learning ◽  
Sign Language ◽  
Language Processing ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Data Sets ◽  
Duplex System ◽  
Indian Sign Language ◽  
Country People ◽  
Readable Text

In this paper we are introducing a sign language converter which works as a duplex system as it can convert text to sign language as well as it can do a real time video to text conversion. It is basically a system that can be used by all people who know sign language as well as who are not familiar with it. The main aim of this system is to involve the specially abled people as much as possible to interact with others. Our system uses the basic NLP i.e. the Natural language Processing and algorithms like CNN classifier to make the implementation of this translator. Along with that this system focuses on the Indian Sign Language so that it can be used by our country people. The finger gestures are captured by the camera and using various machine learning algorithms the system will automatically translate the signs to the readable text, similarly in sign to text conversion, based on the data sets and various Machine learning algorithms the text will be converted to sign language.

scholarly journals Non-Verbal (Sign Language) To Verbal Language Translator Using Convolutional Neural Network

2022 ◽  
Vol 10 (1) ◽  
pp. 269-273
Author(s):  
Poonam Yerpude
Keyword(s):  
Neural Network ◽  
Image Processing ◽  
Convolutional Neural Network ◽  
Sign Language ◽  
The Body ◽  
Image Processing Technique ◽  
Hand Gestures ◽  
Verbal Language ◽  
Architecture Model ◽  
Indian Sign Language

Abstract: Communication is very imperative for daily life. Normal people use verbal language for communication while people with disabilities use sign language for communication. Sign language is a way of communicating by using the hand gestures and parts of the body instead of speaking and listening. As not all people are familiar with sign language, there lies a language barrier. There has been much research in this field to remove this barrier. There are mainly 2 ways in which we can convert the sign language into speech or text to close the gap, i.e. , Sensor based technique,and Image processing. In this paper we will have a look at the Image processing technique, for which we will be using the Convolutional Neural Network (CNN). So, we have built a sign detector, which will recognise the sign numbers from 1 to 10. It can be easily extended to recognise other hand gestures including alphabets (A- Z) and expressions. We are creating this model based on Indian Sign Language(ISL). Keywords: Multi Level Perceptron (MLP), Convolutional Neural Network (CNN), Indian Sign Language(ISL), Region of interest(ROI), Artificial Neural Network(ANN), VGG 16(CNN vision architecture model), SGD(Stochastic Gradient Descent).

Sign in / Sign up

Export Citation Format

Share Document