Sign To Text Conversion- Helping Aid

The ancient way of sign language is most natural forms of communication. The recognition of sign is place a key role in research field. The development and improvement on this kind of work need more and more new techniques to analyze the accurate results. Many people don't know it and interpreters are hard to come by, we developed a real-time technique for finger spelling-based American Sign Language using neural networks. In our technique, the hand is first sent through a filter, and then it is passed through a classifier, which analyses the class of hand movements. For each alphabet the proposed model has a 96 percent accuracy rate. This model mainly implemented for Dumb and Deaf people for communication.

Download Full-text

Web Based Recognition and Translation of American Sign Language with CNN and RNN

International Journal of Online and Biomedical Engineering (iJOE) ◽

10.3991/ijoe.v17i01.18585 ◽

2021 ◽

Vol 17 (01) ◽

pp. 34

Author(s):

Dhanashree Shyam Bendarkar ◽

Pratiksha Appasaheb Somase ◽

Preety Kalyansingh Rebari ◽

Renuka Ramkrishna Paturkar ◽

Arjumand Masood Khan

Keyword(s):

American Sign Language ◽

Sign Language ◽

Gesture Recognition ◽

Short Term Memory ◽

Hand Movements ◽

American Sign ◽

Static Gesture Recognition ◽

Dynamic Gestures ◽

Spatio Temporal ◽

Dynamic Gesture Recognition

Individuals with hearing hindrance utilize gesture based communication to exchange their thoughts. Generally hand movements are used by them to communicate among themselves. But there are certain limitations when they communicate with other people who cannot understand these hand movements. There is a need to have a mechanism that can act as a translator between these people to communicate. It would be easier for these people to interact if there exists direct infrastructure that is able to convert signs to text and voice messages. As of late, numerous such frameworks for gesture based communication acknowledgment have been developed. But most of them are made either for static gesture recognition or dynamic gesture recognition. As sentences are generated using combinations of static and dynamic gestures, it would be simpler for hearing debilitated individuals if such computerized frameworks can detect both the static and dynamic motions together. We have proposed a design and architecture of American Sign Language (ASL) recognition with convolutional neural networks (CNN). This paper utilizes a pretrained VGG-16 architecture for static gesture recognition and for dynamic gesture recognition, spatiotemporal features were learnt with the complex architecture, called deep learning. It contains a bidirectional convolutional Long Short Term Memory network (ConvLSTM) and 3D convolutional neural network (3DCNN) and this architecture is responsible to extract 2D spatio temporal features.

Download Full-text

Signer-Independent Sign Language Recognition with Adversarial Neural Networks

International Journal of Machine Learning and Computing ◽

10.18178/ijmlc.2021.11.2.1024 ◽

2021 ◽

Vol 11 (2) ◽

pp. 121-129

Author(s):

Pedro M. Ferreira ◽

◽

Diogo Pernes ◽

Ana Rebelo ◽

Jaime S. Cardoso

Keyword(s):

Sign Language ◽

Research Field ◽

Training Procedure ◽

Language Recognition ◽

Sign Language Recognition ◽

Learning Stage ◽

Sign Recognition ◽

Proposed Model ◽

Adversarial Training ◽

Latent Representations

Sign Language Recognition (SLR) has become an appealing topic in modern societies because such technology can ideally be used to bridge the gap between deaf and hearing people. Although important steps have been made towards the development of real-world SLR systems, signer-independent SLR is still one of the bottleneck problems of this research field. In this regard, we propose a deep neural network along with an adversarial training objective, specifically designed to address the signer-independent problem. Specifically, the proposed model consists of an encoder, mapping from input images to latent representations, and two classifiers operating on these underlying representations: (i) the sign-classifier, for predicting the class/sign labels, and (ii) the signer-classifier, for predicting their signer identities. During the learning stage, the encoder is simultaneously trained to help the sign-classifier as much as possible while trying to fool the signer-classifier. This adversarial training procedure allows learning signer-invariant latent representations that are in fact highly discriminative for sign recognition. Experimental results demonstrate the effectiveness of the proposed model and its capability of dealing with the large inter-signer variations.

Download Full-text

American Sign Language to Text - Speech using Background Subtraction using Running Averages

International Journal of Engineering and Advanced Technology - Regular Issue ◽

10.35940/ijeat.b3621.129219 ◽

2019 ◽

Vol 9 (2) ◽

pp. 2150-2156

Keyword(s):

Feature Extraction ◽

American Sign Language ◽

Sign Language ◽

Web Applications ◽

Background Model ◽

Absolute Difference ◽

American Sign ◽

Current Frame ◽

Proposed Model ◽

Standard Techniques

This Paper Proposes A System Which Converts American Sign Language Hand Gestures Into Text Cum Speech And Helps To Bridge The Communication Gap Between DeafMute People And Rest Of The Society. Any System For This Purpose Generally Has Four Modules: Segmentation, Feature Extraction, Classification And Text-To-Speech. This Paper Focuses On An Improved Method For The Segmentation And The Feature Extraction Processes To Get More Better Resultswhile Using The Standard Techniques On The Other Two Modules. Proposed Algorithm Captures Initial 30 Frames Of The Live Video From The Web Cam Of The System To Construct The Background Model. It Then Finds The Absolute Difference Between The Current Frame And The Background Model In Order To Get The Foreground. Various Features Are Extracted To Classify The Gestures Like Contour, Convexity Hull Etc.. Proposed Algorithm Has Been Tested Under Low And Normal Room Light Conditions. The Overall Performance Of The Proposed Model Will Be Very High And Will Produce Far More Better Resultsdue To Improved Proposed Algorithms For The Initial Two Modules In Comparison To Other Standard Techniques Used Like Hsv, Ycbcr The Above System Can Be Incorporated Into Simple Web Applications, Mobile Applications And Many Other Applications Translating Gestures In The Conversations In Real Time.

Download Full-text

Real-Time Sign Language Detection and Recognition

International Journal for Research in Applied Science and Engineering Technology ◽

10.22214/ijraset.2021.39103 ◽

2021 ◽

Vol 9 (11) ◽

pp. 1944-1948

Author(s):

Sarthak Sharma

Keyword(s):

Neural Networks ◽

American Sign Language ◽

Real Time ◽

Sign Language ◽

American Sign ◽

Hand Gestures ◽

Natural Form ◽

To Come ◽

Language Detection ◽

Detection And Recognition

Abstract: Sign language is one of the oldest and most natural form of language for communication, but since most people do not know sign language and interpreters are very difficult to come by we have come up with a real time method using neural networks for fingerspelling based American sign language. In our method, the hand is first passed through a filter and after the filter is applied the hand is passed through a classifier which predicts the class of the hand gestures.

Download Full-text