4DCov: A Nested Covariance Descriptor of Spatio-Temporal Features for Gesture Recognition in Depth Sequences

Dynamic hand gesture recognition based on one-shot learning requires full assimilation of the motion features from a few annotated data. However, how to effectively extract the spatio-temporal features of the hand gestures remains a challenging issue. This paper proposes a skeleton-based dynamic hand gesture recognition using an enhanced network (GREN) based on one-shot learning by improving the memory-augmented neural network, which can rapidly assimilate the motion features of dynamic hand gestures. Besides, the network effectively combines and stores the shared features between dissimilar classes, which lowers the prediction error caused by the unnecessary hyper-parameters updating, and improves the recognition accuracy with the increase of categories. In this paper, the public dynamic hand gesture database (DHGD) is used for the experimental comparison of the state-of-the-art performance of the GREN network, and although only 30% of the dataset was used for training, the accuracy of skeleton-based dynamic hand gesture recognition reached 82.29% based on one-shot learning. Experiments with the Microsoft Research Asia (MSRA) hand gesture dataset verified the robustness of the GREN network. The experimental results demonstrate that the GREN network is feasible for skeleton-based dynamic hand gesture recognition based on one-shot learning.

Download Full-text

6D motion gesture recognition using spatio-temporal features

2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ◽

10.1109/icassp.2012.6288384 ◽

2012 ◽

Cited By ~ 2

Author(s):

Mingyu Chen ◽

Ghassan AlRegib ◽

Biing-Hwang Juang

Keyword(s):

Gesture Recognition ◽

Temporal Features ◽

Spatio Temporal

Download Full-text

Fusing shape and spatio-temporal features for depth-based dynamic hand gesture recognition

Multimedia Tools and Applications ◽

10.1007/s11042-016-3988-8 ◽

2016 ◽

Vol 76 (20) ◽

pp. 20525-20544 ◽

Cited By ~ 10

Author(s):

Jinqing Zheng ◽

Zhiyong Feng ◽

Chao Xu ◽

Jing Hu ◽

Weimin Ge

Keyword(s):

Gesture Recognition ◽

Hand Gesture Recognition ◽

Hand Gesture ◽

Temporal Features ◽

Dynamic Hand Gesture Recognition ◽

Spatio Temporal

Download Full-text

Gesture Recognition Via Learning Deep Spatio-Temporal Features In Wi-Fi Sensing

10.21203/rs.3.rs-955273/v1 ◽

2021 ◽

Author(s):

Jianxiao Xie ◽

Wei Ye ◽

Kai Xu

Keyword(s):

Gesture Recognition ◽

Efficient Manner ◽

Time Frequency ◽

3D Data ◽

Temporal Features ◽

Data Segment ◽

Spatio Temporal ◽

3D Cnn ◽

Free Gesture ◽

Device Free

Abstract Internet of Things (IoT) expects to incorporate massive machine-type (MCT) devices, such as vehicles, sensors, and wearable devices, which brings a large number of application tasks that need to be processed. Additionally, data collected from various devices needs to be executed and processed in a timely, reliable, and efficient manner. Gesture recognition has enabled IoT applications such as human-computer interaction and virtual reality. In this work, we propose a cross-domain device-free gesture recognition (DFGR) model, that exploits 3D-CNN to obtain spatiotemporal features in Wi-Fi sensing. To adapt the sensing data to the 3D model, we carry out 3D data segment and supplement in addition to signal denoising and time-frequency transformation. We demonstrate that our proposed model outperforms the state-of-the-art method in the application of DFGR even cross 3 domain factors simultaneously, and is easy to converge and convenient for training with a less complicated hierarchical structure.

Download Full-text

Fire Detection Based on Fractal Analysis and Spatio-Temporal Features

Fire Technology ◽

10.1007/s10694-021-01129-7 ◽

2021 ◽

Author(s):

Monir Torabian ◽

Hossein Pourghassem ◽

Homayoun Mahdavi-Nasab

Keyword(s):

Fractal Analysis ◽

Fire Detection ◽

Temporal Features ◽

Spatio Temporal

Download Full-text

Two Stage Continuous Gesture Recognition Based on Deep Learning

Electronics ◽

10.3390/electronics10050534 ◽

2021 ◽

Vol 10 (5) ◽

pp. 534

Author(s):

Huogen Wang

Keyword(s):

Gesture Recognition ◽

Large Scale ◽

Short Term Memory ◽

Short Term ◽

Hand Motion ◽

Spatiotemporal Features ◽

Spatiotemporal Information ◽

Video Frames ◽

Depth Sequences

The paper proposes an effective continuous gesture recognition method, which includes two modules: segmentation and recognition. In the segmentation module, the video frames are divided into gesture frames and transitional frames by using the information of hand motion and appearance, and continuous gesture sequences are segmented into isolated sequences. In the recognition module, our method exploits the spatiotemporal information embedded in RGB and depth sequences. For the RGB modality, our method adopts Convolutional Long Short-Term Memory Networks to learn long-term spatiotemporal features from short-term spatiotemporal features obtained from a 3D convolutional neural network. For the depth modality, our method converts a sequence into Dynamic Images and Motion Dynamic Images through weighted rank pooling and feed them into Convolutional Neural Networks, respectively. Our method has been evaluated on both ChaLearn LAP Large-scale Continuous Gesture Dataset and Montalbano Gesture Dataset and achieved state-of-the-art performance.

Download Full-text