scholarly journals AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Feng Wen ◽  
Zixuan Zhang ◽  
Tianyiyi He ◽  
Chengkuo Lee
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Learning Model ◽  
Virtual Space ◽  
New Order ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Sentence Recognition ◽  
Bidirectional Communication ◽  
Deep Learning Model

AbstractSign language recognition, especially the sentence recognition, is of great significance for lowering the communication barrier between the hearing/speech impaired and the non-signers. The general glove solutions, which are employed to detect motions of our dexterous hands, only achieve recognizing discrete single gestures (i.e., numbers, letters, or words) instead of sentences, far from satisfying the meet of the signers’ daily communication. Here, we propose an artificial intelligence enabled sign language recognition and communication system comprising sensing gloves, deep learning block, and virtual reality interface. Non-segmentation and segmentation assisted deep learning model achieves the recognition of 50 words and 20 sentences. Significantly, the segmentation approach splits entire sentence signals into word units. Then the deep learning model recognizes all word elements and reversely reconstructs and recognizes sentences. Furthermore, new/never-seen sentences created by new-order word elements recombination can be recognized with an average correct rate of 86.67%. Finally, the sign language recognition results are projected into virtual space and translated into text and audio, allowing the remote and bidirectional communication between signers and non-signers.

scholarly journals Arabic Sign Language Recognition and Generating Arabic Speech Using Convolutional Neural Network

2020 ◽  
Vol 2020 ◽  
pp. 1-9 ◽  
Author(s):  
M. M. Kamruzzaman
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Arabic Language ◽  
Learning Model ◽  
Leap Motion ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Research Activities ◽  
Arabic Sign Language ◽  
Deep Learning Model

Sign language encompasses the movement of the arms and hands as a means of communication for people with hearing disabilities. An automated sign recognition system requires two main courses of action: the detection of particular features and the categorization of particular input data. In the past, many approaches for classifying and detecting sign languages have been put forward for improving system performance. However, the recent progress in the computer vision field has geared us towards the further exploration of hand signs/gestures’ recognition with the aid of deep neural networks. The Arabic sign language has witnessed unprecedented research activities to recognize hand signs and gestures using the deep learning model. A vision-based system by applying CNN for the recognition of Arabic hand sign-based letters and translating them into Arabic speech is proposed in this paper. The proposed system will automatically detect hand sign letters and speaks out the result with the Arabic language with a deep learning model. This system gives 90% accuracy to recognize the Arabic hand sign-based letters which assures it as a highly dependable system. The accuracy can be further improved by using more advanced hand gestures recognizing devices such as Leap Motion or Xbox Kinect. After recognizing the Arabic hand sign-based letters, the outcome will be fed to the text into the speech engine which produces the audio of the Arabic language as an output.

scholarly journals Bangla Sign Language Recognition and Sentence Building Using Deep Learning

2020 ◽  
Author(s):  
Safayet Anowar Shurid ◽  
Khandaker Habibul Amin ◽  
Md. Shahnawaz Mirbahar ◽  
Dolan Karmaker ◽  
Mohammad Tanvir Mahtab ◽  
...  
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Language Recognition ◽  
Sign Language Recognition

KArSL

2021 ◽  
Vol 20 (1) ◽  
pp. 1-19
Author(s):  
Ala Addin I. Sidig ◽  
Hamzah Luqman ◽  
Sabri Mahmoud ◽  
Mohamed Mohandes
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Markov Models ◽  
Body Language ◽  
Arab Countries ◽  
Microsoft Kinect ◽  
Classification Model ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Arabic Sign Language

Sign language is the major means of communication for the deaf community. It uses body language and gestures such as hand shapes, lib patterns, and facial expressions to convey a message. Sign language is geography-specific, as it differs from one country to another. Arabic Sign language is used in all Arab countries. The availability of a comprehensive benchmarking database for ArSL is one of the challenges of the automatic recognition of Arabic Sign language. This article introduces KArSL database for ArSL, consisting of 502 signs that cover 11 chapters of ArSL dictionary. Signs in KArSL database are performed by three professional signers, and each sign is repeated 50 times by each signer. The database is recorded using state-of-art multi-modal Microsoft Kinect V2. We also propose three approaches for sign language recognition using this database. The proposed systems are Hidden Markov Models, deep learning images’ classification model applied on an image composed of shots of the video of the sign, and attention-based deep learning captioning system. Recognition accuracies of these systems indicate their suitability for such a large number of Arabic signs. The techniques are also tested on a publicly available database. KArSL database will be made freely available for interested researchers.

Sign Language Recognition Using Deep Learning and Computer Vision

2020 ◽  
Vol 12 (05-SPECIAL ISSUE) ◽  
pp. 964-968
Author(s):  
Sabeenian R.S ◽  
S. Sai Bharathwaj ◽  
M. Mohamed Aadhil
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Sign Language ◽  
Language Recognition ◽  
Sign Language Recognition

scholarly journals Sensor Fusion of Motion-Based Sign Language Interpretation with Deep Learning

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6256
Author(s):  
Boon Giin Lee ◽  
Teak-Wei Chong ◽  
Wan-Young Chung
Keyword(s):  
Deep Learning ◽  
American Sign Language ◽  
Sign Language ◽  
Sensor Fusion ◽  
Recognition Rate ◽  
Recognition System ◽  
Hearing Impaired ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Impaired People

Sign language was designed to allow hearing-impaired people to interact with others. Nonetheless, knowledge of sign language is uncommon in society, which leads to a communication barrier with the hearing-impaired community. Many studies of sign language recognition utilizing computer vision (CV) have been conducted worldwide to reduce such barriers. However, this approach is restricted by the visual angle and highly affected by environmental factors. In addition, CV usually involves the use of machine learning, which requires collaboration of a team of experts and utilization of high-cost hardware utilities; this increases the application cost in real-world situations. Thus, this study aims to design and implement a smart wearable American Sign Language (ASL) interpretation system using deep learning, which applies sensor fusion that “fuses” six inertial measurement units (IMUs). The IMUs are attached to all fingertips and the back of the hand to recognize sign language gestures; thus, the proposed method is not restricted by the field of view. The study reveals that this model achieves an average recognition rate of 99.81% for dynamic ASL gestures. Moreover, the proposed ASL recognition system can be further integrated with ICT and IoT technology to provide a feasible solution to assist hearing-impaired people in communicating with others and improve their quality of life.

Sign in / Sign up

Export Citation Format

Share Document