scholarly journals Real-time glove and android application for visual and audible Arabic sign language translation

2019 ◽  
Vol 163 ◽  
pp. 450-459
Author(s):  
Nema Salem ◽  
Saja Alharbi ◽  
Raghdah Khezendar ◽  
Hedaih Alshami
Keyword(s):  
Real Time ◽  
Sign Language ◽  
Language Translation ◽  
Android Application ◽  
Arabic Sign Language

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

2021 ◽  
Vol 5 (2) ◽  
pp. 1-30
Author(s):  
HyeonJung Park ◽  
Youngki Lee ◽  
JeongGil Ko
Keyword(s):  
Real Time ◽  
Sign Language ◽  
Data Augmentation ◽  
Language Translation ◽  
Mobile Platforms ◽  
Depth Cameras ◽  
Language Data ◽  
In The Wild ◽  
Environmental Robustness ◽  
Cloud Servers

In this work we present SUGO, a depth video-based system for translating sign language to text using a smartphone's front camera. While exploiting depth-only videos offer benefits such as being less privacy-invasive compared to using RGB videos, it introduces new challenges which include dealing with low video resolutions and the sensors' sensitiveness towards user motion. We overcome these challenges by diversifying our sign language video dataset to be robust to various usage scenarios via data augmentation and design a set of schemes to emphasize human gestures from the input images for effective sign detection. The inference engine of SUGO is based on a 3-dimensional convolutional neural network (3DCNN) to classify a sequence of video frames as a pre-trained word. Furthermore, the overall operations are designed to be light-weight so that sign language translation takes place in real-time using only the resources available on a smartphone, with no help from cloud servers nor external sensing components. Specifically, to train and test SUGO, we collect sign language data from 20 individuals for 50 Korean Sign Language words, summing up to a dataset of ~5,000 sign gestures and collect additional in-the-wild data to evaluate the performance of SUGO in real-world usage scenarios with different lighting conditions and daily activities. Comprehensively, our extensive evaluations show that SUGO can properly classify sign words with an accuracy of up to 91% and also suggest that the system is suitable (in terms of resource usage, latency, and environmental robustness) to enable a fully mobile solution for sign language translation.

scholarly journals Translation System from Arabic Text to Arabic Sign Language

2018 ◽  
Vol 3 (2) ◽  
pp. 57-70
Author(s):  
Nadia Aouiti ◽  
Mohamed Jemni
Keyword(s):  
Natural Language ◽  
Real Time ◽  
Sign Language ◽  
Virtual World ◽  
Web Application ◽  
Arabic Language ◽  
Translation System ◽  
Arabic Text ◽  
Arabic Sign Language ◽  
Ongoing Project

This research paper presents our ongoing project aiming at translating in real time an Arabic text to Arabic Sign Language (ArSL). This project is a part of a Web application [1] based on the technology of the avatar (animation in the virtual world). The input of the system is a text in natural language. The output is a real-time and online interpretation in sign language [2]. Our work focuses on the Arabic language as the text in the input, which needs many treatments due to the particularity of this language. Our solution starts from the linguistic treatment of the Arabic sentence, passing through the definition and the generation of Arabic Annotation Gloss system and coming finally to the generation of an animated sentence using the avatar technology.

scholarly journals Indian Sign Language Recognition through Hybrid ConvNet-LSTM Networks

2021 ◽  
Vol 9 (1) ◽  
pp. 182-203
Author(s):  
Muthu Mariappan H ◽  
Dr Gomathi V
Keyword(s):  
Neural Network ◽  
Computer Vision ◽  
Real Time ◽  
Sign Language ◽  
Gesture Recognition ◽  
Language Translation ◽  
Video Gaming ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Indian Sign Language

Dynamic hand gesture recognition is a challenging task of Human-Computer Interaction (HCI) and Computer Vision. The potential application areas of gesture recognition include sign language translation, video gaming, video surveillance, robotics, and gesture-controlled home appliances. In the proposed research, gesture recognition is applied to recognize sign language words from real-time videos. Classifying the actions from video sequences requires both spatial and temporal features. The proposed system handles the former by the Convolutional Neural Network (CNN), which is the core of several computer vision solutions and the latter by the Recurrent Neural Network (RNN), which is more efficient in handling the sequences of movements. Thus, the real-time Indian sign language (ISL) recognition system is developed using the hybrid CNN-RNN architecture. The system is trained with the proposed CasTalk-ISL dataset. The ultimate purpose of the presented research is to deploy a real-time sign language translator to break the hurdles present in the communication between hearing-impaired people and normal people. The developed system achieves 95.99% top-1 accuracy and 99.46% top-3 accuracy on the test dataset. The obtained results outperform the existing approaches using various deep models on different datasets.

scholarly journals Intelligent Arabic text to Arabic Sign Language Translation for Easy Deaf Communication

2014 ◽  
Vol 92 (8) ◽  
pp. 22-29 ◽  
Author(s):  
A. E.E.ElAlfi ◽  
M. M. R. El Basuony ◽  
S. M. El Atawy
Keyword(s):  
Sign Language ◽  
Language Translation ◽  
Arabic Text ◽  
Arabic Sign Language
Sign in / Sign up

Export Citation Format

Share Document