scholarly journals Semi-Supervised Joint Learning for Hand Gesture Recognition from a Single Color Image

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 1007
Author(s):  
Chi Xu ◽  
Yunkai Jiang ◽  
Jun Zhou ◽  
Yi Liu
Keyword(s):  
Gesture Recognition ◽  
Pose Estimation ◽  
Color Image ◽  
Recognition Performance ◽  
Recognition Task ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Estimation Task ◽  
Hand Pose Estimation ◽  
Hand Pose

Hand gesture recognition and hand pose estimation are two closely correlated tasks. In this paper, we propose a deep-learning based approach which jointly learns an intermediate level shared feature for these two tasks, so that the hand gesture recognition task can be benefited from the hand pose estimation task. In the training process, a semi-supervised training scheme is designed to solve the problem of lacking proper annotation. Our approach detects the foreground hand, recognizes the hand gesture, and estimates the corresponding 3D hand pose simultaneously. To evaluate the hand gesture recognition performance of the state-of-the-arts, we propose a challenging hand gesture recognition dataset collected in unconstrained environments. Experimental results show that, the gesture recognition accuracy of ours is significantly boosted by leveraging the knowledge learned from the hand pose estimation task.

scholarly journals Robust hand gesture recognition using multiple shape-oriented visual cues

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Samy Bakheet ◽  
Ayoub Al-Hamadi
Keyword(s):  
Real Time ◽  
Gesture Recognition ◽  
Pose Estimation ◽  
Depth Map ◽  
Hand Gesture Recognition ◽  
Support Vector ◽  
Hand Gesture ◽  
Hand Pose Estimation ◽  
Time Operation ◽  
Hand Pose

AbstractRobust vision-based hand pose estimation is highly sought but still remains a challenging task, due to its inherent difficulty partially caused by self-occlusion among hand fingers. In this paper, an innovative framework for real-time static hand gesture recognition is introduced, based on an optimized shape representation build from multiple shape cues. The framework incorporates a specific module for hand pose estimation based on depth map data, where the hand silhouette is first extracted from the extremely detailed and accurate depth map captured by a time-of-flight (ToF) depth sensor. A hybrid multi-modal descriptor that integrates multiple affine-invariant boundary-based and region-based features is created from the hand silhouette to obtain a reliable and representative description of individual gestures. Finally, an ensemble of one-vs.-all support vector machines (SVMs) is independently trained on each of these learned feature representations to perform gesture classification. When evaluated on a publicly available dataset incorporating a relatively large and diverse collection of egocentric hand gestures, the approach yields encouraging results that agree very favorably with those reported in the literature, while maintaining real-time operation.

scholarly journals An Online Robot Teaching Method using Static Hand Gestures and Poses

2021 ◽  
Author(s):  
Digang Sun ◽  
Ping Zhang ◽  
Mingxuan Chen ◽  
Jiaxin Chen
Keyword(s):  
Gesture Recognition ◽  
Pose Estimation ◽  
Hand Gesture Recognition ◽  
Human Robot Interaction ◽  
Hand Gesture ◽  
Depth Information ◽  
Hand Gestures ◽  
Teaching Mode ◽  
Hand Pose Estimation ◽  
Hand Pose

With an increasing number of robots are employed in manufacturing, a human-robot interaction method that can teach robots in a natural, accurate, and rapid manner is needed. In this paper, we propose a novel human-robot interface based on the combination of static hand gestures and hand poses. In our proposed interface, the pointing direction of the index finger and the orientation of the whole hand are extracted to indicate the moving direction and orientation of the robot in a fast-teaching mode. A set of hand gestures are designed according to their usage in humans' daily life and recognized to control the position and orientation of the robot in a fine-teaching mode. We employ the feature extraction ability of the hand pose estimation network via transfer learning and utilize attention mechanisms to improve the performance of the hand gesture recognition network. The inputs of hand pose estimation and hand gesture recognition networks are monocular RGB images, making our method independent of depth information input and applicable to more scenarios. In the regular shape reconstruction experiments on the UR3 robot, the mean error of the reconstructed shape is less than 1 mm, which demonstrates the effectiveness and efficiency of our method.

scholarly journals An Online Robot Teaching Method using Static Hand Gestures and Poses

2021 ◽  
Author(s):  
Digang Sun ◽  
Ping Zhang ◽  
Mingxuan Chen ◽  
Jiaxin Chen
Keyword(s):  
Gesture Recognition ◽  
Pose Estimation ◽  
Hand Gesture Recognition ◽  
Human Robot Interaction ◽  
Hand Gesture ◽  
Depth Information ◽  
Hand Gestures ◽  
Teaching Mode ◽  
Hand Pose Estimation ◽  
Hand Pose

With an increasing number of robots are employed in manufacturing, a human-robot interaction method that can teach robots in a natural, accurate, and rapid manner is needed. In this paper, we propose a novel human-robot interface based on the combination of static hand gestures and hand poses. In our proposed interface, the pointing direction of the index finger and the orientation of the whole hand are extracted to indicate the moving direction and orientation of the robot in a fast-teaching mode. A set of hand gestures are designed according to their usage in humans' daily life and recognized to control the position and orientation of the robot in a fine-teaching mode. We employ the feature extraction ability of the hand pose estimation network via transfer learning and utilize attention mechanisms to improve the performance of the hand gesture recognition network. The inputs of hand pose estimation and hand gesture recognition networks are monocular RGB images, making our method independent of depth information input and applicable to more scenarios. In the regular shape reconstruction experiments on the UR3 robot, the mean error of the reconstructed shape is less than 1 mm, which demonstrates the effectiveness and efficiency of our method.

scholarly journals Hand gesture recognition based on free-form contours and probabilistic inference

2012 ◽  
Vol 22 (2) ◽  
pp. 437-448 ◽  
Author(s):  
Włodzimierz Kasprzak ◽  
Artur Wilkowski ◽  
Karol Czapnik
Keyword(s):  
Gesture Recognition ◽  
Color Image ◽  
Vision System ◽  
Probabilistic Inference ◽  
Hand Gesture Recognition ◽  
Free Form ◽  
Hand Gesture ◽  
Inference Methods ◽  
Viterbi Search ◽  
Hand Pose

Hand gesture recognition based on free-form contours and probabilistic inference A computer vision system is described that captures color image sequences, detects and recognizes static hand poses (i.e., "letters") and interprets pose sequences in terms of gestures (i.e., "words"). The hand object is detected with a double-active contour-based method. A tracking of the hand pose in a short sequence allows detecting "modified poses", like diacritic letters in national alphabets. The static hand pose set corresponds to hand signs of a thumb alphabet. Finally, by tracking hand poses in a longer image sequence, the pose sequence is interpreted in terms of gestures. Dynamic Bayesian models and their inference methods (particle filter and Viterbi search) are applied at this stage, allowing a bi-driven control of the entire system.

HAND GESTURE RECOGNITION IN REAL-TIME

2021 ◽  
Vol 1 (10) ◽  
pp. 346-350
Author(s):  
Priyanshi Gupta ◽  
Amita Goel ◽  
Nidhi Sengar ◽  
Vashudha Bahl
Keyword(s):  
Machine Learning ◽  
Gesture Recognition ◽  
Research Field ◽  
Hand Gesture Recognition ◽  
Machine Learning Algorithms ◽  
Hand Gesture ◽  
Normal People ◽  
Human Gestures ◽  
Dynamic Gestures ◽  
Hand Pose

Hand gesture is language through which normal people can communicate with deaf and dumb people. Hand gesture recognition detects the hand pose and converts it to the corresponding alphabet or sentence. In past years it received great attention from society because of its application. It uses machine learning algorithms. Hand gesture recognition is a great application of human computer interaction. An emerging research field that is based on human centered computing aims to understand human gestures and integrate users and their social context with computer systems. One of the unique and challenging applications in this framework is to collect information about human dynamic gestures. Keywords: Tensor Flow, Machine learning, React js, handmark model, media pipeline

scholarly journals putEMG—A Surface Electromyography Hand Gesture Recognition Dataset

Sensors ◽  
2019 ◽  
Vol 19 (16) ◽  
pp. 3548 ◽  
Author(s):  
Piotr Kaczmarek ◽  
Tomasz Mańkowski ◽  
Jakub Tomczyński
Keyword(s):  
Gesture Recognition ◽  
Motion Tracking ◽  
Recognition Performance ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Svm Classifier ◽  
Classification Methods ◽  
Hand Gestures ◽  
Semg Signal ◽  
Feature Sets

In this paper, we present a putEMG dataset intended for the evaluation of hand gesture recognition methods based on sEMG signal. The dataset was acquired for 44 able-bodied subjects and include 8 gestures (3 full hand gestures, 4 pinches and idle). It consists of uninterrupted recordings of 24 sEMG channels from the subject’s forearm, RGB video stream and depth camera images used for hand motion tracking. Moreover, exemplary processing scripts are also published. The putEMG dataset is available under a Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). The dataset was validated regarding sEMG amplitudes and gesture recognition performance. The classification was performed using state-of-the-art classifiers and feature sets. An accuracy of 90% was achieved for SVM classifier utilising RMS feature and for LDA classifier using Hudgin’s and Du’s feature sets. Analysis of performance for particular gestures showed that LDA/Du combination has significantly higher accuracy for full hand gestures, while SVM/RMS performs better for pinch gestures. The presented dataset can be used as a benchmark for various classification methods, the evaluation of electrode localisation concepts, or the development of classification methods invariant to user-specific features or electrode displacement.

Sign in / Sign up

Export Citation Format

Share Document