scholarly journals Hybrid FAST-SIFT-CNN (HFSC) approach for Vision-Based Indian Sign Language Recognition

Indian Sign Language (ISL) is the conventional means of communication for the deaf-mute community in the Indian subcontinent. Accurate feature extraction is one of the prime challenges in automatic gesture recognition of ISL gestures. In this paper, a hybrid approach, namely HFSC, integrating FAST and SIFT with CNN has been proposed for automatic and accurate recognition of ISL's static and single-hand gestures. Features from accelerated segment test (FAST) and scale-invariant feature transform (SIFT) provides the basic framework for feature extraction while CNN is used for classification. The performance of HFSC is compared with existing sign language recognition approaches by testing on standard benchmark (MNIST, Jochen-Trisech, and NUS hand posture-II) datasets. The HFSC algorithm's efficiency has been shown by comparing it with CNN and SIFT_CNN for a uniform dataset with an accuracy of 97.89%. Furthermore, the Computational results of the HFSC on complex background dataset achieve comparable accuracy of 95%.

Communication is one of the basic requirements for living in the world. Deaf and dumb people convey through Sign Language but normal people have difficulty to understand their language. In order to provide a source of medium between normal and differently abled people, Sign Language Recognition System (SLR) is a solution . American Sign Language (ASL) has attracted many researchers’ attention but Indian Sign Language Recognition (ISLR) is significantly different from ASL due to different phonetic, grammar and hand movement. Designing a system for Indian Sign Language Recognition becomes a difficult task. ISLR system uses Indian Sign Language (ISL) dataset for recognition but suffers from problem of scaling, object orientation and lack of optimal feature set. In this paper to address these issues, Scale-Invariant Feature Transform (SIFT) as a descriptor is used. It extracts the features that train the Feed Forward Back Propagation Neural Network (FFBPNN) and optimize it using Artificial Bee Colony (ABC) according to the fitness function. The dataset has been collected for alphabet from the video by extracting frames and for numbers it has been created manually from deaf and dumb students of NGO “Sarthak”. It has been shown through simulation results that there has been significant improvement in accurately identifying alphabets and numbers with an average accuracy of 99.43%


2019 ◽  
Vol 7 (2) ◽  
pp. 43
Author(s):  
MALHOTRA POOJA ◽  
K. MANIAR CHIRAG ◽  
V. SANKPAL NIKHIL ◽  
R. THAKKAR HARDIK ◽  
◽  
...  

Sign in / Sign up

Export Citation Format

Share Document