scholarly journals Towards Hybrid Multimodal Manual and Non-Manual Arabic Sign Language Recognition: mArSL Database and Pilot Study

Electronics ◽  
2021 ◽  
Vol 10 (14) ◽  
pp. 1739
Author(s):  
Hamzah Luqman ◽  
El-Sayed M. El-Alfy
Keyword(s):  
Sign Language ◽  
Recognition System ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Sign Languages ◽  
Temporal Features ◽  
Learning Techniques ◽  
Recognition Systems ◽  
Arabic Sign Language ◽  
Independent Mode

Sign languages are the main visual communication medium between hard-hearing people and their societies. Similar to spoken languages, they are not universal and vary from region to region, but they are relatively under-resourced. Arabic sign language (ArSL) is one of these languages that has attracted increasing attention in the research community. However, most of the existing and available works on sign language recognition systems focus on manual gestures, ignoring other non-manual information needed for other language signals such as facial expressions. One of the main challenges of not considering these modalities is the lack of suitable datasets. In this paper, we propose a new multi-modality ArSL dataset that integrates various types of modalities. It consists of 6748 video samples of fifty signs performed by four signers and collected using Kinect V2 sensors. This dataset will be freely available for researchers to develop and benchmark their techniques for further advancement of the field. In addition, we evaluated the fusion of spatial and temporal features of different modalities, manual and non-manual, for sign language recognition using the state-of-the-art deep learning techniques. This fusion boosted the accuracy of the recognition system at the signer-independent mode by 3.6% compared with manual gestures.

scholarly journals Arabic Sign Language Recognition System for Alphabets Using Machine Learning Techniques

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Gamal Tharwat ◽  
Abdelmoty M. Ahmed ◽  
Belgacem Bouallegue
Keyword(s):  
Machine Learning ◽  
Sign Language ◽  
Nearest Neighbor ◽  
Recognition System ◽  
Machine Learning Techniques ◽  
Language Recognition ◽  
K Nearest Neighbor ◽  
Sign Language Recognition ◽  
Learning Techniques ◽  
Arabic Sign Language

In recent years, the role of pattern recognition in systems based on human computer interaction (HCI) has spread in terms of computer vision applications and machine learning, and one of the most important of these applications is to recognize the hand gestures used in dealing with deaf people, in particular to recognize the dashed letters in surahs of the Quran. In this paper, we suggest an Arabic Alphabet Sign Language Recognition System (AArSLRS) using the vision-based approach. The proposed system consists of four stages: the stage of data processing, preprocessing of data, feature extraction, and classification. The system deals with three types of datasets: data dealing with bare hands and a dark background, data dealing with bare hands, but with a light background, and data dealing with hands wearing dark colored gloves. AArSLRS begins with obtaining an image of the alphabet gestures, then revealing the hand from the image and isolating it from the background using one of the proposed methods, after which the hand features are extracted according to the selection method used to extract them. Regarding the classification process in this system, we have used supervised learning techniques for the classification of 28-letter Arabic alphabet using 9240 images. We focused on the classification for 14 alphabetic letters that represent the first Quran surahs in the Quranic sign language (QSL). AArSLRS achieved an accuracy of 99.5% for the K-Nearest Neighbor (KNN) classifier.

scholarly journals Arabic Sign Language Recognition System Using 2D Hands and Body Skeleton Data

IEEE Access ◽  
2021 ◽  
Vol 9 ◽  
pp. 59612-59627
Author(s):  
Mohamed A. Bencherif ◽  
Mohammed Algabri ◽  
Mohamed A. Mekhtiche ◽  
Mohammed Faisal ◽  
Mansour Alsulaiman ◽  
...  
Keyword(s):  
Sign Language ◽  
Recognition System ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Arabic Sign Language

scholarly journals Sign Language Recognition-A Survey of Techniques

2020 ◽  
Vol 9 (2) ◽  
pp. 854-857
Keyword(s):  
Sign Language ◽  
Recognition System ◽  
Spoken Language ◽  
Hearing Impaired ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Sign Languages ◽  
Data Glove ◽  
Depth Sensors ◽  
Impaired People

Sign language is the only method of communication for the hearing and speech impaired people around the world. Most of the speech and hearing-impaired people understand single sign language. Thus, there is an increasing demand for sign language interpreters. For regular people learning sign language is difficult, and for speech and hearing-impaired person, learning spoken language is impossible. There is a lot of research being done in the domain of automatic sign language recognition. Different methods such as, computer vision, data glove, depth sensors can be used to train a computer to interpret sign language. The interpretation is being done from sign to text, text to sign, speech to sign and sign to speech. Different countries use different sign languages, the signers of different sign languages are unable to communicate with each other. Analyzing the characteristic features of gestures provides insights about the sign language, some common features in sign languages gestures will help in designing a sign language recognition system. This type of system will help in reducing the communication gap between sign language users and spoken language users.

scholarly journals Vision-based Pakistani Sign Language Recognition Using Bag-of-Words and Support Vector Machines

2022 ◽  
Author(s):  
Muhammad Shaheer Mirza ◽  
Sheikh Muhammad Munaf ◽  
Shahid Ali ◽  
Fahad Azim ◽  
Saad Jawaid Khan
Keyword(s):  
Sign Language ◽  
Recognition System ◽  
Support Vector ◽  
Bag Of Words ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Sign Languages ◽  
Vector Machines ◽  
Rgb Images ◽  
Native Signers

Abstract In order to perform their daily activities, a person is required to communicating with others. This can be a major obstacle for the deaf population of the world, who communicate using sign languages (SL). Pakistani Sign Language (PSL) is used by more than 250,000 deaf Pakistanis. Developing a SL recognition system would greatly facilitate these people. This study aimed to collect data of static and dynamic PSL alphabets and to develop a vision-based system for their recognition using Bag-of-Words (BoW) and Support Vector Machine (SVM) techniques. A total of 5,120 images for 36 static PSL alphabet signs and 353 videos with 45,224 frames for 3 dynamic PSL alphabet signs were collected from 10 native signers of PSL. The developed system used the collected data as input, resized the data to various scales and converted the RGB images into grayscale. The resized grayscale images were segmented using Thresholding technique and features were extracted using Speeded Up Robust Feature (SURF). The obtained SURF descriptors were clustered using K-means clustering. A BoW was obtained by computing the Euclidean distance between the SURF descriptors and the clustered data. The codebooks were divided into training and testing using 5-fold cross validation. The highest overall classification accuracy for static PSL signs was 97.80% at 750×750 image dimensions and 500 Bags. For dynamic PSL signs a 96.53% accuracy was obtained at 480×270 video resolution and 200 Bags.

Sign in / Sign up

Export Citation Format

Share Document