Hand Gesture Classification Using Early Fusion Based Multimodal Deep Learning

2021 ◽  
Vol 70 (11) ◽  
pp. 1714-1721
Author(s):  
Ik-Jin Kim ◽  
Su-Yeol Kim ◽  
Yong-Chan Lee ◽  
Yun-Jung Lee
Sensors ◽  
2020 ◽  
Vol 20 (10) ◽  
pp. 2972
Author(s):  
Qinghua Gao ◽  
Shuo Jiang ◽  
Peter B. Shull

Hand gesture classification and finger angle estimation are both critical for intuitive human–computer interaction. However, most approaches study them in isolation. We thus propose a dual-output deep learning model to enable simultaneous hand gesture classification and finger angle estimation. Data augmentation and deep learning were used to detect spatial-temporal features via a wristband with ten modified barometric sensors. Ten subjects performed experimental testing by flexing/extending each finger at the metacarpophalangeal joint while the proposed model was used to classify each hand gesture and estimate continuous finger angles simultaneously. A data glove was worn to record ground-truth finger angles. Overall hand gesture classification accuracy was 97.5% and finger angle estimation R 2 was 0.922, both of which were significantly higher than shallow existing learning approaches used in isolation. The proposed method could be used in applications related to the human–computer interaction and in control environments with both discrete and continuous variables.


Sensors ◽  
2019 ◽  
Vol 19 (8) ◽  
pp. 1952 ◽  
Author(s):  
Wentao Sun ◽  
Huaxin Liu ◽  
Rongyu Tang ◽  
Yiran Lang ◽  
Jiping He ◽  
...  

Conventional pattern-recognition algorithms for surface electromyography (sEMG)-based hand-gesture classification have difficulties in capturing the complexity and variability of sEMG. The deep structures of deep learning enable the method to learn high-level features of data to improve both accuracy and robustness of a classification. However, the features learned through deep learning are incomprehensible, and this issue has precluded the use of deep learning in clinical applications where model comprehension is required. In this paper, a generative flow model (GFM), which is a recent flourishing branch of deep learning, is used with a SoftMax classifier for hand-gesture classification. The proposed approach achieves 63 . 86 ± 5 . 12 % accuracy in classifying 53 different hand gestures from the NinaPro database 5. The distribution of all 53 hand gestures is modelled by the GFM, and each dimension of the feature learned by the GFM is comprehensible using the reverse flow of the GFM. Moreover, the feature appears to be related to muscle synergy to some extent.


Author(s):  
Sruthy Skaria ◽  
Da Huang ◽  
Akram Al-Hourani ◽  
Robin J. Evans ◽  
Margaret Lech

2022 ◽  
pp. 108053
Author(s):  
Noemi Gozzi ◽  
Lorenzo Malandri ◽  
Fabio Mercorio ◽  
Alessandra Pedrocchi

Sign in / Sign up

Export Citation Format

Share Document