Towards an Arabic Sign Language (ArSL) corpus for deaf drivers

Sign language is a common language that deaf people around the world use to communicate with others. However, normal people are generally not familiar with sign language (SL) and they do not need to learn their language to communicate with them in everyday life. Several technologies offer possibilities for overcoming these barriers to assisting deaf people and facilitating their active lives, including natural language processing (NLP), text understanding, machine translation, and sign language simulation. In this paper, we mainly focus on the problem faced by the deaf community in Saudi Arabia as an important member of the society that needs assistance in communicating with others, especially in the field of work as a driver. Therefore, this community needs a system that facilitates the mechanism of communication with the users using NLP that allows translating Arabic Sign Language (ArSL) into voice and vice versa. Thus, this paper aims to purplish our created dataset dictionary and ArSL corpus videos that were done in our previous work. Furthermore, we illustrate our corpus, data determination (deaf driver terminologies), dataset creation and processing in order to implement the proposed future system. Therefore, the evaluation of the dataset will be presented and simulated using two methods. First, using the evaluation of four expert signers, where the result was 10.23% WER. The second method, using Cohen’s Kappa in order to evaluate the corpus of ArSL videos that was made by three signers from different regions of Saudi Arabia. We found that the agreement between signer 2 and signer 3 is 61%, which is a good agreement. In our future direction, we will use the ArSL video corpus of signer 2 and signer 3 to implement ML techniques for our deaf driver system.

Download Full-text

MEDIA INTERAKTIF PENGENALAN BAHASA ISYARAT BISINDO

Jurnal Teknologi Informasi dan Pendidikan ◽

10.24036/tip.v13i1.293 ◽

2020 ◽

Vol 13 (1) ◽

pp. 35-44

Author(s):

Bayu Ramadhani Fajri ◽

Agariadne Dwinggo Samala ◽

Fadhli Ranuharja

Keyword(s):

Sign Language ◽

Group Discussion ◽

Interactive Media ◽

Smartphone Application ◽

Deaf People ◽

Community Needs ◽

The People ◽

Data Collection Technique ◽

Level Of Selection ◽

The Right

Sign language is the communication that used by the deaf people. Sign language is also a form of accessibility for deaf people to be able to socialize with people around them. Deaf socialization with normal people in the society is not optimal because most of the people does not yet know sign language. Based on these problems, this study aims to develop an interactive media introduction to BISINDO sign language based on augmented reality for the community in supporting a society that is more inclusive of the deaf. This study uses a Research and Development (R&D) approach. The sample in this study was deaf people who participated in the Gerkatin Padang community, and people from various professions in the city of Padang. The data collection technique was carried out using a questionnaire and a focused group discussion technique. Based on the results of the collection of 100 respondents' perceptions data from the public towards sign language obtained: 1) the level of getting to know the community towards sign language is still low at 57%, 2) the level of skills using sign language is still low at 83%, 3) the importance level of sign language according the community is 81%, 4) The level of community needs to learn Sign Language is obtained by 88%, and 5) The level of selection of the right media for learning Sign Language, 74% of respondents choose media in the form of a smartphone application. The validity of the interactive media prototype introduction BISINDO sign language developed in terms of substance that is the suitability of sign language in the application obtained a validator rating of 89.33%, then in terms of the media obtained a validator value of 89.67%. It can be concluded that the interactive media prototype in the form of a smartphone application that was developed very well in terms of substance and media so that it is very feasible or valid to be used in the introduction of BISINDO sign language.

Download Full-text

A Systematic Mapping of Translation-Enabling Technologies for Sign Languages

Electronics ◽

10.3390/electronics8091047 ◽

2019 ◽

Vol 8 (9) ◽

pp. 1047 ◽

Cited By ~ 2

Author(s):

Luis Naranjo-Zeledón ◽

Jesús Peral ◽

Antonio Ferrández ◽

Mario Chacón-Rivas

Keyword(s):

Software Engineering ◽

Sign Language ◽

Language Processing ◽

Information Technologies ◽

Deaf People ◽

Systematic Mapping ◽

Scientific Publications ◽

Sign Languages ◽

European Continent ◽

Enabling Technologies

Sign languages (SL) are the first language for most deaf people. Consequently, bidirectional communication among deaf and non-deaf people has always been a challenging issue. Sign language usage has increased due to inclusion policies and general public agreement, which must then become evident in information technologies, in the many facets that comprise sign language understanding and its computational treatment. In this study, we conduct a thorough systematic mapping of translation-enabling technologies for sign languages. This mapping has considered the most recommended guidelines for systematic reviews, i.e., those pertaining software engineering, since there is a need to account for interdisciplinary areas of accessibility, human computer interaction, natural language processing, and education, all of them part of ACM (Association for Computing Machinery) computing classification system directly related to software engineering. An ongoing development of a software tool called SYMPLE (SYstematic Mapping and Parallel Loading Engine) facilitated the querying and construction of a base set of candidate studies. A great diversity of topics has been studied over the last 25 years or so, but this systematic mapping allows for comfortable visualization of predominant areas, venues, top authors, and different measures of concentration and dispersion. The systematic review clearly shows a large number of classifications and subclassifications interspersed over time. This is an area of study in which there is much interest, with a basically steady level of scientific publications over the last decade, concentrated mainly in the European continent. The publications by country, nevertheless, usually favor their local sign language.

Download Full-text

Arabic Sign Language Recognition System Using 2D Hands and Body Skeleton Data

IEEE Access ◽

10.1109/access.2021.3069714 ◽

2021 ◽

Vol 9 ◽

pp. 59612-59627

Author(s):

Mohamed A. Bencherif ◽

Mohammed Algabri ◽

Mohamed A. Mekhtiche ◽

Mohammed Faisal ◽

Mansour Alsulaiman ◽

...

Keyword(s):

Sign Language ◽

Recognition System ◽

Language Recognition ◽

Sign Language Recognition ◽

Arabic Sign Language

Download Full-text

3D Avatar Approach for Continuous Sign Movement Using Speech/Text

Applied Sciences ◽

10.3390/app11083439 ◽

2021 ◽

Vol 11 (8) ◽

pp. 3439

Author(s):

Debashis Das Chakladar ◽

Pradeep Kumar ◽

Shubham Mandal ◽

Partha Pratim Roy ◽

Masakazu Iwamura ◽

...

Keyword(s):

Language Learning ◽

Sign Language ◽

Language Processing ◽

Hearing Impaired ◽

Learning System ◽

English Sentence ◽

Impaired People ◽

Indian Sign Language ◽

Impaired Person ◽

3D Avatar

Sign language is a visual language for communication used by hearing-impaired people with the help of hand and finger movements. Indian Sign Language (ISL) is a well-developed and standard way of communication for hearing-impaired people living in India. However, other people who use spoken language always face difficulty while communicating with a hearing-impaired person due to lack of sign language knowledge. In this study, we have developed a 3D avatar-based sign language learning system that converts the input speech/text into corresponding sign movements for ISL. The system consists of three modules. Initially, the input speech is converted into an English sentence. Then, that English sentence is converted into the corresponding ISL sentence using the Natural Language Processing (NLP) technique. Finally, the motion of the 3D avatar is defined based on the ISL sentence. The translation module achieves a 10.50 SER (Sign Error Rate) score.

Download Full-text

KArSL

ACM Transactions on Asian and Low-Resource Language Information Processing ◽

10.1145/3423420 ◽

2021 ◽

Vol 20 (1) ◽

pp. 1-19

Author(s):

Ala Addin I. Sidig ◽

Hamzah Luqman ◽

Sabri Mahmoud ◽

Mohamed Mohandes

Keyword(s):

Deep Learning ◽

Sign Language ◽

Markov Models ◽

Body Language ◽

Arab Countries ◽

Microsoft Kinect ◽

Classification Model ◽

Language Recognition ◽

Sign Language Recognition ◽

Arabic Sign Language

Sign language is the major means of communication for the deaf community. It uses body language and gestures such as hand shapes, lib patterns, and facial expressions to convey a message. Sign language is geography-specific, as it differs from one country to another. Arabic Sign language is used in all Arab countries. The availability of a comprehensive benchmarking database for ArSL is one of the challenges of the automatic recognition of Arabic Sign language. This article introduces KArSL database for ArSL, consisting of 502 signs that cover 11 chapters of ArSL dictionary. Signs in KArSL database are performed by three professional signers, and each sign is repeated 50 times by each signer. The database is recorded using state-of-art multi-modal Microsoft Kinect V2. We also propose three approaches for sign language recognition using this database. The proposed systems are Hidden Markov Models, deep learning images’ classification model applied on an image composed of shots of the video of the sign, and attention-based deep learning captioning system. Recognition accuracies of these systems indicate their suitability for such a large number of Arabic signs. The techniques are also tested on a publicly available database. KArSL database will be made freely available for interested researchers.

Download Full-text

Smart, Comfortable Wearable System for Recognizing Arabic Sign Language in Real-time using IMUs and Features-based Fusion

Expert Systems with Applications ◽

10.1016/j.eswa.2021.115448 ◽

2021 ◽

pp. 115448

Author(s):

Aziz Qaroush ◽

Sara Yassin ◽

Ali Al-Nubani ◽

Ameer Alqam

Keyword(s):

Real Time ◽

Sign Language ◽

Wearable System ◽

Arabic Sign Language

Download Full-text

Edge-based recognizer for Arabic sign language alphabet (ArS2V-Arabic sign to voice)

2010 International Computer Engineering Conference (ICENCO) ◽

10.1109/icenco.2010.5720438 ◽

2010 ◽

Cited By ~ 11

Author(s):

Elsayed E. Hemayed ◽

Allam S. Hassanien

Keyword(s):

Sign Language ◽

Arabic Sign Language ◽

Edge Based

Download Full-text

Mask Region-Based Convolutional Neural Networks (R-CNN) for Sinhala Sign Language to Text Conversion

10.5121/csit.2021.111418 ◽

2021 ◽

Author(s):

R. D. Rusiru Sewwantha ◽

T. N. D. S. Ginige

Keyword(s):

Natural Language ◽

Object Detection ◽

American Sign Language ◽

Sign Language ◽

Language Processing ◽

Sri Lankan ◽

American Sign ◽

Disabled People ◽

Communication Difficulties ◽

Communication Gap

Sign Language is the use of various gestures and symbols for communication. It is mainly used by disabled people with communication difficulties due to their speech or hearing impediments. Due to the lack of knowledge on sign language, natural language speakers like us, are not able to communicate with such people. As a result, a communication gap is created between sign language users and natural language speakers. It should also be noted that sign language differs from country to country. With American sign language being the most commonly used, in Sri Lanka, we use Sri Lankan/Sinhala sign language. In this research, the authors propose a mobile solution using a Region Based Convolutional Neural Network for object detection to reduce the communication gap between the sign users and language speakers by identifying and interpreting Sinhala sign language to Sinhala text using Natural Language Processing (NLP). The system is able to identify and interpret still gesture signs in real-time using the trained model. The proposed solution uses object detection for the identification of the signs.

Download Full-text