scholarly journals A Combined Semantic and Motion Capture Database for Real-Time Sign Language Synthesis

2009 ◽  
pp. 432-438 ◽  
Author(s):  
Charly Awad ◽  
Nicolas Courty ◽  
Kyle Duarte ◽  
Thibaut Le Naour ◽  
Sylvie Gibet
Keyword(s):  
Real Time ◽  
Sign Language ◽  
Motion Capture ◽  
Sign Language Synthesis

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

2021 ◽  
Vol 5 (2) ◽  
pp. 1-30
Author(s):  
HyeonJung Park ◽  
Youngki Lee ◽  
JeongGil Ko
Keyword(s):  
Real Time ◽  
Sign Language ◽  
Data Augmentation ◽  
Language Translation ◽  
Mobile Platforms ◽  
Depth Cameras ◽  
Language Data ◽  
In The Wild ◽  
Environmental Robustness ◽  
Cloud Servers

In this work we present SUGO, a depth video-based system for translating sign language to text using a smartphone's front camera. While exploiting depth-only videos offer benefits such as being less privacy-invasive compared to using RGB videos, it introduces new challenges which include dealing with low video resolutions and the sensors' sensitiveness towards user motion. We overcome these challenges by diversifying our sign language video dataset to be robust to various usage scenarios via data augmentation and design a set of schemes to emphasize human gestures from the input images for effective sign detection. The inference engine of SUGO is based on a 3-dimensional convolutional neural network (3DCNN) to classify a sequence of video frames as a pre-trained word. Furthermore, the overall operations are designed to be light-weight so that sign language translation takes place in real-time using only the resources available on a smartphone, with no help from cloud servers nor external sensing components. Specifically, to train and test SUGO, we collect sign language data from 20 individuals for 50 Korean Sign Language words, summing up to a dataset of ~5,000 sign gestures and collect additional in-the-wild data to evaluate the performance of SUGO in real-world usage scenarios with different lighting conditions and daily activities. Comprehensively, our extensive evaluations show that SUGO can properly classify sign words with an accuracy of up to 91% and also suggest that the system is suitable (in terms of resource usage, latency, and environmental robustness) to enable a fully mobile solution for sign language translation.

Research of Sign Language Synthesis Based on VRML

2013 ◽  
Vol 347-350 ◽  
pp. 2631-2635
Author(s):  
Shi Cai Yu ◽  
Rong Lu
Keyword(s):  
Sign Language ◽  
Human Body ◽  
Normal Hearing ◽  
Computer Assisted Instruction ◽  
Virtual Human ◽  
Computer Assisted ◽  
Smoothing Algorithm ◽  
Human Body Modeling ◽  
Sign Language Synthesis ◽  
Assisted Instruction

Sign language is to help the deaf and normal hearing people natural communication and computer assisted instruction. Through the analysis of language features, and proposed one kind based on the VRML human body modeling and virtual human based on context of gesture smoothing algorithm, thus the sign language synthesis research and implementation.

Using Real-Time Feedback with Motion Capture to Measure Upper Extremity Workspace: a Preliminary Evaluation

2021 ◽  
Author(s):  
R. Tyler ◽  
Stephanie Russo ◽  
Ross Chafetz ◽  
Spencer Warshauer ◽  
Emily Nice ◽  
...  
Keyword(s):  
Real Time ◽  
Upper Extremity ◽  
Motion Capture ◽  
Preliminary Evaluation
Sign in / Sign up

Export Citation Format

Share Document