American Sign Language Recognition Based on MobileNetV2

In today's digital world, rapid technological advancements continue to lessen the burden of tasks for individuals. Among these tasks is communication across perceived language barriers. Indeed, increased attention has been drawn to American Sign Language (ASL) recognition in recent years. Camera-based and motion detection-based methods have been researched extensively; however, there remains a divide in communication between ASL users and non-users. Therefore, this research team proposes the use of a novel wireless sensor (Frequency-Modulated Continuous-Wave Radar) to help bridge the gap in communication. In short, this device sends out signals that detect the user's body positioning in space. These signals then reflect off the body and back to the sensor, developing thousands of cloud points per second, indicating where the body is positioned in space. These cloud points can then be examined for movement over multiple consecutive time frames using a cell division algorithm, ultimately showing how the body moves through space as it completes a single gesture or sentence. At the end of the project, 95% accuracy was achieved in one-object prediction as well as 80% accuracy on cross-object prediction with 30% other objects' data introduced on 19 commonly used gestures. There are 30 samples for each gesture per person from three persons.

Download Full-text

A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion

Advances in Intelligent Systems and Computing - Advances in Computational Intelligence Systems ◽

10.1007/978-3-030-29933-0_9 ◽

2019 ◽

pp. 107-118

Author(s):

Qing Gao ◽

Uchenna Emeoha Ogenyi ◽

Jinguo Liu ◽

Zhaojie Ju ◽

Honghai Liu

Keyword(s):

Data Fusion ◽

American Sign Language ◽

Sign Language ◽

American Sign ◽

Language Recognition ◽

Sign Language Recognition ◽

Multimodal Data ◽

Multimodal Data Fusion

Download Full-text

Comparing ANN, SVM, and HMM based Machine Learning Methods for American Sign Language Recognition using Wearable Motion Sensors

2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC) ◽

10.1109/ccwc.2019.8666491 ◽

2019 ◽

Cited By ~ 4

Author(s):

Rabeet Fatmi ◽

Sherif Rashad ◽

Ryan Integlia

Keyword(s):

Machine Learning ◽

American Sign Language ◽

Sign Language ◽

Motion Sensors ◽

American Sign ◽

Language Recognition ◽

Sign Language Recognition ◽

Learning Methods ◽

Machine Learning Methods ◽

Wearable Motion Sensors

Download Full-text

American Sign Language Recognition Using Leap Motion Controller with Machine Learning Approach

Sensors ◽

10.3390/s18103554 ◽

2018 ◽

Vol 18 (10) ◽

pp. 3554 ◽

Cited By ~ 18

Author(s):

Teak-Wei Chong ◽

Boon-Giin Lee

Keyword(s):

American Sign Language ◽

Sign Language ◽

Recognition System ◽

Support Vector ◽

Leap Motion ◽

American Sign ◽

Language Recognition ◽

Motion Controller ◽

Sign Language Recognition ◽

Leap Motion Controller

Sign language is intentionally designed to allow deaf and dumb communities to convey messages and to connect with society. Unfortunately, learning and practicing sign language is not common among society; hence, this study developed a sign language recognition prototype using the Leap Motion Controller (LMC). Many existing studies have proposed methods for incomplete sign language recognition, whereas this study aimed for full American Sign Language (ASL) recognition, which consists of 26 letters and 10 digits. Most of the ASL letters are static (no movement), but certain ASL letters are dynamic (they require certain movements). Thus, this study also aimed to extract features from finger and hand motions to differentiate between the static and dynamic gestures. The experimental results revealed that the sign language recognition rates for the 26 letters using a support vector machine (SVM) and a deep neural network (DNN) are 80.30% and 93.81%, respectively. Meanwhile, the recognition rates for a combination of 26 letters and 10 digits are slightly lower, approximately 72.79% for the SVM and 88.79% for the DNN. As a result, the sign language recognition system has great potential for reducing the gap between deaf and dumb communities and others. The proposed prototype could also serve as an interpreter for the deaf and dumb in everyday life in service sectors, such as at the bank or post office.

Download Full-text