Bangla Sign Language (BdSL) Alphabets and Numerals Classification Using a Deep Learning Model

A real-time Bangla Sign Language interpreter can enable more than 200 k hearing and speech-impaired people to the mainstream workforce in Bangladesh. Bangla Sign Language (BdSL) recognition and detection is a challenging topic in computer vision and deep learning research because sign language recognition accuracy may vary on the skin tone, hand orientation, and background. This research has used deep machine learning models for accurate and reliable BdSL Alphabets and Numerals using two well-suited and robust datasets. The dataset prepared in this study comprises of the largest image database for BdSL Alphabets and Numerals in order to reduce inter-class similarity while dealing with diverse image data, which comprises various backgrounds and skin tones. The papers compared classification with and without background images to determine the best working model for BdSL Alphabets and Numerals interpretation. The CNN model trained with the images that had a background was found to be more effective than without background. The hand detection portion in the segmentation approach must be more accurate in the hand detection process to boost the overall accuracy in the sign recognition. It was found that ResNet18 performed best with 99.99% accuracy, precision, F1 score, sensitivity, and 100% specificity, which outperforms the works in the literature for BdSL Alphabets and Numerals recognition. This dataset is made publicly available for researchers to support and encourage further research on Bangla Sign Language Interpretation so that the hearing and speech-impaired individuals can benefit from this research.

Download Full-text

Technological Aids for Deaf and Mute in Modern World

Recent Patents on Engineering ◽

10.2174/1872212114999201116214802 ◽

2020 ◽

Vol 14 ◽

Author(s):

Vasu Mehra ◽

Dhiraj Pandey ◽

Aayush Rastogi ◽

Aditya Singh ◽

Harsh Preet Singh

Keyword(s):

Sign Language ◽

Gesture Recognition ◽

Recognition System ◽

Modern World ◽

Language Recognition ◽

Sign Language Recognition ◽

Background Elimination ◽

Sign Recognition ◽

Technological Advances ◽

Better Than

Background:: People suffering from hearing and speaking disabilities have a few ways of communicating with other people. One of these is to communicate through the use of sign language. Objective:: Developing a system for sign language recognition becomes essential for deaf as well as a mute person. The recognition system acts as a translator between a disabled and an able person. This eliminates the hindrances in exchange of ideas. Most of the existing systems are very poorly designed with limited support for the needs of their day to day facilities. Methods:: The proposed system embedded with gesture recognition capability has been introduced here which extracts signs from a video sequence and displays them on screen. On the other hand, a speech to text as well as text to speech system is also introduced to further facilitate the grieved people. To get the best out of human computer relationship, the proposed solution consists of various cutting-edge technologies and Machine Learning based sign recognition models which have been trained by using Tensor Flow and Keras library. Result:: The proposed architecture works better than several gesture recognition techniques like background elimination and conversion to HSV because of sharply defined image provided to the model for classification. The results of testing indicate reliable recognition systems with high accuracy that includes most of the essential and necessary features for any deaf and dumb person in his/her day to day tasks. Conclusion:: It’s the need of current technological advances to develop reliable solutions which can be deployed to assist deaf and dumb people to adjust to normal life. Instead of focusing on a standalone technology, a plethora of them have been introduced in this proposed work. Proposed Sign Recognition System is based on feature extraction and classification. The trained model helps in identification of different gestures.

Download Full-text

Deep Learning Methods for Indian Sign Language Recognition

2020 IEEE 10th International Conference on Consumer Electronics (ICCE-Berlin) ◽

10.1109/icce-berlin50680.2020.9352194 ◽

2020 ◽

Author(s):

Pratik Likhar ◽

Neel Kamal Bhagat ◽

Rathna G N

Keyword(s):

Deep Learning ◽

Sign Language ◽

Language Recognition ◽

Sign Language Recognition ◽

Learning Methods ◽

Indian Sign Language

Download Full-text

Bangla Sign Language Recognition and Sentence Building Using Deep Learning

2020 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) ◽

10.1109/csde50874.2020.9411523 ◽

2020 ◽

Author(s):

Safayet Anowar Shurid ◽

Khandaker Habibul Amin ◽

Md. Shahnawaz Mirbahar ◽

Dolan Karmaker ◽

Mohammad Tanvir Mahtab ◽

...

Keyword(s):

Deep Learning ◽

Sign Language ◽

Language Recognition ◽

Sign Language Recognition

Download Full-text

KArSL

ACM Transactions on Asian and Low-Resource Language Information Processing ◽

10.1145/3423420 ◽

2021 ◽

Vol 20 (1) ◽

pp. 1-19

Author(s):

Ala Addin I. Sidig ◽

Hamzah Luqman ◽

Sabri Mahmoud ◽

Mohamed Mohandes

Keyword(s):

Deep Learning ◽

Sign Language ◽

Markov Models ◽

Body Language ◽

Arab Countries ◽

Microsoft Kinect ◽

Classification Model ◽

Language Recognition ◽

Sign Language Recognition ◽

Arabic Sign Language

Sign language is the major means of communication for the deaf community. It uses body language and gestures such as hand shapes, lib patterns, and facial expressions to convey a message. Sign language is geography-specific, as it differs from one country to another. Arabic Sign language is used in all Arab countries. The availability of a comprehensive benchmarking database for ArSL is one of the challenges of the automatic recognition of Arabic Sign language. This article introduces KArSL database for ArSL, consisting of 502 signs that cover 11 chapters of ArSL dictionary. Signs in KArSL database are performed by three professional signers, and each sign is repeated 50 times by each signer. The database is recorded using state-of-art multi-modal Microsoft Kinect V2. We also propose three approaches for sign language recognition using this database. The proposed systems are Hidden Markov Models, deep learning images’ classification model applied on an image composed of shots of the video of the sign, and attention-based deep learning captioning system. Recognition accuracies of these systems indicate their suitability for such a large number of Arabic signs. The techniques are also tested on a publicly available database. KArSL database will be made freely available for interested researchers.

Download Full-text