scholarly journals Skeleton-Based Dynamic Hand Gesture Recognition Using an Enhanced Network with One-Shot Learning

2020 ◽  
Vol 10 (11) ◽  
pp. 3680 ◽  
Author(s):  
Chunyong Ma ◽  
Shengsheng Zhang ◽  
Anni Wang ◽  
Yongyang Qi ◽  
Ge Chen
Keyword(s):  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Experimental Comparison ◽  
Hand Gesture ◽  
Hand Gestures ◽  
The Public ◽  
Temporal Features ◽  
Motion Features ◽  
Dynamic Hand Gesture Recognition ◽  
Spatio Temporal

Dynamic hand gesture recognition based on one-shot learning requires full assimilation of the motion features from a few annotated data. However, how to effectively extract the spatio-temporal features of the hand gestures remains a challenging issue. This paper proposes a skeleton-based dynamic hand gesture recognition using an enhanced network (GREN) based on one-shot learning by improving the memory-augmented neural network, which can rapidly assimilate the motion features of dynamic hand gestures. Besides, the network effectively combines and stores the shared features between dissimilar classes, which lowers the prediction error caused by the unnecessary hyper-parameters updating, and improves the recognition accuracy with the increase of categories. In this paper, the public dynamic hand gesture database (DHGD) is used for the experimental comparison of the state-of-the-art performance of the GREN network, and although only 30% of the dataset was used for training, the accuracy of skeleton-based dynamic hand gesture recognition reached 82.29% based on one-shot learning. Experiments with the Microsoft Research Asia (MSRA) hand gesture dataset verified the robustness of the GREN network. The experimental results demonstrate that the GREN network is feasible for skeleton-based dynamic hand gesture recognition based on one-shot learning.

scholarly journals Dynamic Hand Gesture Recognition Based on a Leap Motion Controller and Two-Layer Bidirectional Recurrent Neural Network

Sensors ◽  
2020 ◽  
Vol 20 (7) ◽  
pp. 2106 ◽  
Author(s):  
Linchu Yang ◽  
Ji’an Chen ◽  
Weihang Zhu
Keyword(s):  
Neural Network ◽  
Gesture Recognition ◽  
Recurrent Neural Network ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Leap Motion ◽  
Motion Controller ◽  
Hand Gestures ◽  
Leap Motion Controller ◽  
Dynamic Hand Gesture Recognition

Dynamic hand gesture recognition is one of the most significant tools for human–computer interaction. In order to improve the accuracy of the dynamic hand gesture recognition, in this paper, a two-layer Bidirectional Recurrent Neural Network for the recognition of dynamic hand gestures from a Leap Motion Controller (LMC) is proposed. In addition, based on LMC, an efficient way to capture the dynamic hand gestures is identified. Dynamic hand gestures are represented by sets of feature vectors from the LMC. The proposed system has been tested on the American Sign Language (ASL) datasets with 360 samples and 480 samples, and the Handicraft-Gesture dataset, respectively. On the ASL dataset with 360 samples, the system achieves accuracies of 100% and 96.3% on the training and testing sets. On the ASL dataset with 480 samples, the system achieves accuracies of 100% and 95.2%. On the Handicraft-Gesture dataset, the system achieves accuracies of 100% and 96.7%. In addition, 5-fold, 10-fold, and Leave-One-Out cross-validation are performed on these datasets. The accuracies are 93.33%, 94.1%, and 98.33% (360 samples), 93.75%, 93.5%, and 98.13% (480 samples), and 88.66%, 90%, and 92% on ASL and Handicraft-Gesture datasets, respectively. The developed system demonstrates similar or better performance compared to other approaches in the literature.

scholarly journals MFA-Net: Motion Feature Augmented Network for Dynamic Hand Gesture Recognition from Skeletal Data

Sensors ◽  
2019 ◽  
Vol 19 (2) ◽  
pp. 239 ◽  
Author(s):  
Xinghao Chen ◽  
Guijin Wang ◽  
Hengkai Guo ◽  
Cairong Zhang ◽  
Hang Wang ◽  
...  
Keyword(s):  
Gesture Recognition ◽  
Classification Performance ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Motion Feature ◽  
Variational Autoencoder ◽  
Comparable Performance ◽  
Motion Features ◽  
Dynamic Hand Gesture Recognition ◽  
Global Movements

Dynamic hand gesture recognition has attracted increasing attention because of its importance for human–computer interaction. In this paper, we propose a novel motion feature augmented network (MFA-Net) for dynamic hand gesture recognition from skeletal data. MFA-Net exploits motion features of finger and global movements to augment features of deep network for gesture recognition. To describe finger articulated movements, finger motion features are extracted from the hand skeleton sequence via a variational autoencoder. Global motion features are utilized to represent the global movements of hand skeleton. These motion features along with the skeleton sequence are then fed into three branches of a recurrent neural network (RNN), which augment the motion features for RNN and improve the classification performance. The proposed MFA-Net is evaluated on two challenging skeleton-based dynamic hand gesture datasets, including DHG-14/28 dataset and SHREC’17 dataset. Experimental results demonstrate that our proposed method achieves comparable performance on DHG-14/28 dataset and better performance on SHREC’17 dataset when compared with start-of-the-art methods.

HUMAN-COMPUTER INTERACTION USING DYNAMIC HAND GESTURE RECOGNITION TO CONVENIENTLY CONTROL THE SYSTEM

2021 ◽  
Vol 5 (9) ◽  
Author(s):  
Smit Parikh ◽  
Srikar Banka ◽  
Isha Lautrey ◽  
Isha Gupta ◽  
Prof Dhanalekshmi Yedurkar
Keyword(s):  
Human Computer Interaction ◽  
Gesture Recognition ◽  
Recognition System ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Hand Gestures ◽  
Basic Functions ◽  
Dynamic Hand Gesture Recognition ◽  
Basic Features ◽  
Computer Interaction

The use of a physical controller such as a mouse, a keyboard for human computer interaction hinders the natural interface since the user and computer have a high barrier. Our aim is to create an application that controls some basic features of computers using hand gestures through an integrated webcam to resolve this issue. A Hand Gesture Recognition system detects gestures and translates them into specific actions to make our work easier. This can be pursued using OpenCV to capture the gestures which will be interfaced using Django, React.Js and Electron. An algorithm named YOLO is used to train the system accordingly. The gestures will get saved inside the DBMS. The main result expected is that the user will be able to control the basic functions of the system using his/her hand gestures and hence providing them utmost comfort.

A Real-Time Computer Vision-Based Static and Dynamic Hand Gesture Recognition System

2014 ◽  
Vol 14 (01n02) ◽  
pp. 1450006 ◽  
Author(s):  
Mahmood Jasim ◽  
Tao Zhang ◽  
Md. Hasanuzzaman
Keyword(s):  
Computer Vision ◽  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
The Novel ◽  
Hand Gestures ◽  
Linear Discriminant ◽  
Dynamic Gestures ◽  
Dynamic Hand Gesture Recognition ◽  
The Mean

This paper presents a novel method for computer vision-based static and dynamic hand gesture recognition. Haar-like feature-based cascaded classifier is used for hand area segmentation. Static hand gestures are recognized using linear discriminant analysis (LDA) and local binary pattern (LBP)-based feature extraction methods. Static hand gestures are classified using nearest neighbor (NN) algorithm. Dynamic hand gestures are recognized using the novel text-based principal directional features (PDFs), which are generated from the segmented image sequences. Longest common subsequence (LCS) algorithm is used to classify the dynamic gestures. For testing, the Chinese numeral gesture dataset containing static hand poses and directional gesture dataset containing complex dynamic gestures are prepared. The mean accuracy of LDA-based static hand gesture recognition on the Chinese numeral gesture dataset is 92.42%. The mean accuracy of LBP-based static hand gesture recognition on the Chinese numeral gesture dataset is 87.23%. The mean accuracy of the novel dynamic hand gesture recognition method using PDF on directional gesture dataset is 94%.

scholarly journals Dynamic Gesture Controlled User Interface Expert HCI System using Adaptative Background Masking: An Aid to Prevent Cross Infections

2020 ◽  
Author(s):  
Seema Rawat ◽  
Praveen Kumar ◽  
Ishita Singh ◽  
Shourya Banerjee ◽  
Shabana Urooj ◽  
...  
Keyword(s):  
Computer Vision ◽  
Gesture Recognition ◽  
Video Processing ◽  
Hand Gesture Recognition ◽  
Physical Contact ◽  
Hand Gesture ◽  
Input Device ◽  
Recognition Method ◽  
Hand Gestures ◽  
Dynamic Hand Gesture Recognition

Human-Computer Interaction (HCI) interfaces need unambiguous instructions in the form of mouse clicks or keyboard taps from the user and thus gets complex. To simplify this monotonous task, a real-time hand gesture recognition method using computer vision, image, and video processing techniques has been proposed. Controlling infections has turned out to be the major concern of the healthcare environment. Several input devices such as keyboards, mouse, touch screens can be considered as a breeding ground for various micro pathogens and bacteria. Direct use of hands as an input device is an innovative method for providing natural HCI ensuring minimal physical contact with the devices i.e., less transmission of bacteria and thus can prevent cross infections. Convolutional Neural Network (CNN) has been used for object detection and classification. CNN architecture for 3d object recognition has been proposed which consists of two models: 1) A detector, a CNN architecture for detection of gestures; and 2) A classifier, a CNN for classification of the detected gestures. By using dynamic hand gesture recognition to interact with the system, the interactions can be increased with the help of multidimensional use of hand gestures as compared to other input methods. The dynamic hand gesture recognition method focuses to replace the mouse for interaction with the virtual objects. This work centralises the efforts of implementing a method that employs computer vision algorithms and gesture recognition techniques for developing a low-cost interface device for interacting with objects in the virtual environment such as screens using hand gestures.

Connectionist Temporal Classification Model for Dynamic Hand Gesture Recognition using RGB and Optical flow Data

2020 ◽  
Vol 17 (4) ◽  
pp. 497-506
Author(s):  
Sunil Patel ◽  
Ramji Makwana
Keyword(s):  
Neural Network ◽  
Optical Flow ◽  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Classification Model ◽  
Hand Gesture ◽  
Flow Data ◽  
Dynamic Hand Gesture Recognition ◽  
Connectionist Temporal Classification

Automatic classification of dynamic hand gesture is challenging due to the large diversity in a different class of gesture, Low resolution, and it is performed by finger. Due to a number of challenges many researchers focus on this area. Recently deep neural network can be used for implicit feature extraction and Soft Max layer is used for classification. In this paper, we propose a method based on a two-dimensional convolutional neural network that performs detection and classification of hand gesture simultaneously from multimodal Red, Green, Blue, Depth (RGBD) and Optical flow Data and passes this feature to Long-Short Term Memory (LSTM) recurrent network for frame-to-frame probability generation with Connectionist Temporal Classification (CTC) network for loss calculation. We have calculated an optical flow from Red, Green, Blue (RGB) data for getting proper motion information present in the video. CTC model is used to efficiently evaluate all possible alignment of hand gesture via dynamic programming and check consistency via frame-to-frame for the visual similarity of hand gesture in the unsegmented input stream. CTC network finds the most probable sequence of a frame for a class of gesture. The frame with the highest probability value is selected from the CTC network by max decoding. This entire CTC network is trained end-to-end with calculating CTC loss for recognition of the gesture. We have used challenging Vision for Intelligent Vehicles and Applications (VIVA) dataset for dynamic hand gesture recognition captured with RGB and Depth data. On this VIVA dataset, our proposed hand gesture recognition technique outperforms competing state-of-the-art algorithms and gets an accuracy of 86%

Sign in / Sign up

Export Citation Format

Share Document