Isolated Sign Language Recognition with Multi-Scale Spatial-Temporal Graph Convolutional Networks

2021 ◽  
Author(s):  
Manuel Vazquez-Enriquez ◽  
Jose L. Alba-Castro ◽  
Laura Docio-Fernandez ◽  
Eduardo Rodriguez-Banga
Keyword(s):  
Sign Language ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Convolutional Networks ◽  
Multi Scale ◽  
Temporal Graph

scholarly journals An Attention-Enhanced Multi-Scale and Dual Sign Language Recognition Network Based on a Graph Convolution Network

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1120
Author(s):  
Lu Meng ◽  
Ronghui Li
Keyword(s):  
Sign Language ◽  
Attention Mechanism ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Accuracy Rate ◽  
Long Distance ◽  
Convolutional Network ◽  
Attention Network ◽  
Multi Scale ◽  
Human Finger

Sign language is the most important way of communication for hearing-impaired people. Research on sign language recognition can help normal people understand sign language. We reviewed the classic methods of sign language recognition, and the recognition accuracy is not high enough because of redundant information, human finger occlusion, motion blurring, the diversified signing styles of different people, and so on. To overcome these shortcomings, we propose a multi-scale and dual sign language recognition Network (SLR-Net) based on a graph convolutional network (GCN). The original input data was RGB videos. We first extracted the skeleton data from them and then used the skeleton data for sign language recognition. SLR-Net is mainly composed of three sub-modules: multi-scale attention network (MSA), multi-scale spatiotemporal attention network (MSSTA) and attention enhanced temporal convolution network (ATCN). MSA allows the GCN to learn the dependencies between long-distance vertices; MSSTA can directly learn the spatiotemporal features; ATCN allows the GCN network to better learn the long temporal dependencies. The three different attention mechanisms, multi-scale attention mechanism, spatiotemporal attention mechanism, and temporal attention mechanism, are proposed to further improve the robustness and accuracy. Besides, a keyframe extraction algorithm is proposed, which can greatly improve efficiency by sacrificing a little accuracy. Experimental results showed that our method can reach 98.08% accuracy rate in the CSL-500 dataset with a 500-word vocabulary. Even on the challenging dataset DEVISIGN-L with a 2000-word vocabulary, it also reached a 64.57% accuracy rate, outperforming other state-of-the-art sign language recognition methods.

INDIAN SIGN LANGUAGE RECOGNITION SYSTEM USING OPENPOSE

2019 ◽  
Vol 7 (2) ◽  
pp. 43
Author(s):  
MALHOTRA POOJA ◽  
K. MANIAR CHIRAG ◽  
V. SANKPAL NIKHIL ◽  
R. THAKKAR HARDIK ◽  
◽  
...  
Keyword(s):  
Sign Language ◽  
Recognition System ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Indian Sign Language

STATIC DEVNAGARI SIGN LANGUAGE RECOGNITION

2016 ◽  
Vol 3 (3) ◽  
pp. 13
Author(s):  
VERMA VERSHA ◽  
PATIL SANDEEP B. ◽  
◽  
Keyword(s):  
Sign Language ◽  
Language Recognition ◽  
Sign Language Recognition
Sign in / Sign up

Export Citation Format

Share Document