scholarly journals Sign language recognition using Kinect sensor based on color stream and skeleton points

2021 ◽  
Vol 47 (2) ◽  
pp. 769-778
Author(s):  
Isack Bulugu
Keyword(s):  
Sign Language ◽  
Recognition Accuracy ◽  
Learning Algorithm ◽  
Recognition System ◽  
Kinect Sensor ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Sign Recognition ◽  
Average Accuracy ◽  
High Recognition Accuracy

This paper presents a sign language recognition system based on color stream and skeleton points. Several approaches have been established to address sign language recognition problems. However, most of the previous approaches still have poor recognition accuracy. The proposed approach uses Kinect sensor based on color stream and skeleton points from the depth stream to improved recognition accuracy. Techniques within this approach use hand trajectories and hand shapes in combating sign recognition challenges. Therefore, for a particular sign a representative feature vector is extracted, which consists of hand trajectories and hand shapes. A sparse dictionary learning algorithm, Label Consistent K-SVD (LC-KSVD) is applied to obtain a discriminative dictionary. Based on that, the system was further developed to a new classification approach for better results. The proposed system was fairly evaluated based on 21 sign words including one-handed signs and two-handed signs. It was observed that the proposed system gets high recognition accuracy of 98.25%, and obtained an average accuracy of 95.34% for signer independent recognition. Keywords: Sign language, Color stream, Skeleton points, Kinect sensor, Discriminative dictionary.

Technological Aids for Deaf and Mute in Modern World

2020 ◽  
Vol 14 ◽  
Author(s):  
Vasu Mehra ◽  
Dhiraj Pandey ◽  
Aayush Rastogi ◽  
Aditya Singh ◽  
Harsh Preet Singh
Keyword(s):  
Sign Language ◽  
Gesture Recognition ◽  
Recognition System ◽  
Modern World ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Background Elimination ◽  
Sign Recognition ◽  
Technological Advances ◽  
Better Than

Background:: People suffering from hearing and speaking disabilities have a few ways of communicating with other people. One of these is to communicate through the use of sign language. Objective:: Developing a system for sign language recognition becomes essential for deaf as well as a mute person. The recognition system acts as a translator between a disabled and an able person. This eliminates the hindrances in exchange of ideas. Most of the existing systems are very poorly designed with limited support for the needs of their day to day facilities. Methods:: The proposed system embedded with gesture recognition capability has been introduced here which extracts signs from a video sequence and displays them on screen. On the other hand, a speech to text as well as text to speech system is also introduced to further facilitate the grieved people. To get the best out of human computer relationship, the proposed solution consists of various cutting-edge technologies and Machine Learning based sign recognition models which have been trained by using Tensor Flow and Keras library. Result:: The proposed architecture works better than several gesture recognition techniques like background elimination and conversion to HSV because of sharply defined image provided to the model for classification. The results of testing indicate reliable recognition systems with high accuracy that includes most of the essential and necessary features for any deaf and dumb person in his/her day to day tasks. Conclusion:: It’s the need of current technological advances to develop reliable solutions which can be deployed to assist deaf and dumb people to adjust to normal life. Instead of focusing on a standalone technology, a plethora of them have been introduced in this proposed work. Proposed Sign Recognition System is based on feature extraction and classification. The trained model helps in identification of different gestures.

scholarly journals Multi-Stroke Thai Finger-Spelling Sign Language Recognition System with Deep Learning

Symmetry ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 262
Author(s):  
Thongpan Pariwat ◽  
Pusadee Seresangtakul
Keyword(s):  
Deep Learning ◽  
Sign Language ◽  
Semantic Segmentation ◽  
Recognition System ◽  
Communication Tool ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Hand Segmentation ◽  
Average Accuracy ◽  
Filter Size

Sign language is a type of language for the hearing impaired that people in the general public commonly do not understand. A sign language recognition system, therefore, represents an intermediary between the two sides. As a communication tool, a multi-stroke Thai finger-spelling sign language (TFSL) recognition system featuring deep learning was developed in this study. This research uses a vision-based technique on a complex background with semantic segmentation performed with dilated convolution for hand segmentation, hand strokes separated using optical flow, and learning feature and classification done with convolution neural network (CNN). We then compared the five CNN structures that define the formats. The first format was used to set the number of filters to 64 and the size of the filter to 3 × 3 with 7 layers; the second format used 128 filters, each filter 3 × 3 in size with 7 layers; the third format used the number of filters in ascending order with 7 layers, all of which had an equal 3 × 3 filter size; the fourth format determined the number of filters in ascending order and the size of the filter based on a small size with 7 layers; the final format was a structure based on AlexNet. As a result, the average accuracy was 88.83%, 87.97%, 89.91%, 90.43%, and 92.03%, respectively. We implemented the CNN structure based on AlexNet to create models for multi-stroke TFSL recognition systems. The experiment was performed using an isolated video of 42 Thai alphabets, which are divided into three categories consisting of one stroke, two strokes, and three strokes. The results presented an 88.00% average accuracy for one stroke, 85.42% for two strokes, and 75.00% for three strokes.

INDIAN SIGN LANGUAGE RECOGNITION SYSTEM USING OPENPOSE

2019 ◽  
Vol 7 (2) ◽  
pp. 43
Author(s):  
MALHOTRA POOJA ◽  
K. MANIAR CHIRAG ◽  
V. SANKPAL NIKHIL ◽  
R. THAKKAR HARDIK ◽  
◽  
...  
Keyword(s):  
Sign Language ◽  
Recognition System ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Indian Sign Language

scholarly journals Arabic Sign Language Recognition System Using 2D Hands and Body Skeleton Data

IEEE Access ◽  
2021 ◽  
Vol 9 ◽  
pp. 59612-59627
Author(s):  
Mohamed A. Bencherif ◽  
Mohammed Algabri ◽  
Mohamed A. Mekhtiche ◽  
Mohammed Faisal ◽  
Mansour Alsulaiman ◽  
...  
Keyword(s):  
Sign Language ◽  
Recognition System ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Arabic Sign Language

Assistive Technology based Solution for Hearing Impairment using Smartphone

2022 ◽  
Vol 10 (1) ◽  
pp. 0-0
Keyword(s):  
Gesture Recognition ◽  
Video Sequence ◽  
Recognition System ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Background Elimination ◽  
Sign Recognition ◽  
Limited Support ◽  
Recognition Capability ◽  
Better Than

Developing a system for sign language recognition becomes essential for the deaf as well as a mute person. The recognition system acts as a translator between a disabled and an able person. This eliminates the hindrances in the exchange of ideas. Most of the existing systems are very poorly designed with limited support for the needs of their day to day facilities. The proposed system embedded with gesture recognition capability has been introduced here which extracts signs from a video sequence and displays them on screen. On the other hand, a speech to text as well as text to speech system is also introduced to further facilitate the grieved people. To get the best out of a human-computer relationship, the proposed solution consists of various cutting-edge technologies and Machine Learning based sign recognition models that have been trained by using TensorFlow and Keras library. The proposed architecture works better than several gesture recognition techniques like background elimination and conversion to HSV

scholarly journals Sign Language Recognition Using Two-Stream Convolutional Neural Networks with Wi-Fi Signals

2020 ◽  
Vol 10 (24) ◽  
pp. 9005
Author(s):  
Chien-Cheng Lee ◽  
Zhongjian Gao
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Sign Language ◽  
Recognition Accuracy ◽  
Signal Interference ◽  
Stream Networks ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Stream Network ◽  
Value Decomposition

Sign language is an important way for deaf people to understand and communicate with others. Many researchers use Wi-Fi signals to recognize hand and finger gestures in a non-invasive manner. However, Wi-Fi signals usually contain signal interference, background noise, and mixed multipath noise. In this study, Wi-Fi Channel State Information (CSI) is preprocessed by singular value decomposition (SVD) to obtain the essential signals. Sign language includes the positional relationship of gestures in space and the changes of actions over time. We propose a novel dual-output two-stream convolutional neural network. It not only combines the spatial-stream network and the motion-stream network, but also effectively alleviates the backpropagation problem of the two-stream convolutional neural network (CNN) and improves its recognition accuracy. After the two stream networks are fused, an attention mechanism is applied to select the important features learned by the two-stream networks. Our method has been validated by the public dataset SignFi and adopted five-fold cross-validation. Experimental results show that SVD preprocessing can improve the performance of our dual-output two-stream network. For home, lab, and lab + home environment, the average recognition accuracy rates are 99.13%, 96.79%, and 97.08%, respectively. Compared with other methods, our method has good performance and better generalization capability.

Hear Sign Language: A Real-time End-to-End Sign Language Recognition System

2020 ◽  
pp. 1-1
Author(s):  
Zhibo Wang ◽  
Tengda Zhao ◽  
Jinxin Ma ◽  
Hongkai Chen ◽  
Kaixin Liu ◽  
...  
Keyword(s):  
Real Time ◽  
Sign Language ◽  
Recognition System ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
End To End
Sign in / Sign up

Export Citation Format

Share Document