Dynamic Hand Gesture Recognition in In-Vehicle Environment Based on FMCW Radar and Transformer

Hand gesture recognition technology plays an important role in human-computer interaction and in-vehicle entertainment. Under in-vehicle conditions, it is a great challenge to design gesture recognition systems due to variable driving conditions, complex backgrounds, and diversified gestures. In this paper, we propose a gesture recognition system based on frequency-modulated continuous-wave (FMCW) radar and transformer for an in-vehicle environment. Firstly, the original range-Doppler maps (RDMs), range-azimuth maps (RAMs), and range-elevation maps (REMs) of the time sequence of each gesture are obtained by radar signal processing. Then we preprocess the obtained data frames by region of interest (ROI) extraction, vibration removal algorithm, background removal algorithm, and standardization. We propose a transformer-based radar gesture recognition network named RGTNet. It fully extracts and fuses the spatial-temporal information of radar feature maps to complete the classification of various gestures. The experimental results show that our method can better complete the eight gesture classification tasks in the in-vehicle environment. The recognition accuracy is 97.56%.

Download Full-text

Radar-Based Hand Gesture Recognition Using Spiking Neural Networks

Electronics ◽

10.3390/electronics10121405 ◽

2021 ◽

Vol 10 (12) ◽

pp. 1405

Author(s):

Ing Jyh Tsang ◽

Federico Corradi ◽

Manolis Sifalakis ◽

Werner Van Leekwijck ◽

Steven Latré

Keyword(s):

Neural Network ◽

Gesture Recognition ◽

Liquid State ◽

Continuous Wave ◽

Spike Trains ◽

Hand Gesture Recognition ◽

Radar Signal ◽

Support Vector ◽

Hand Gesture ◽

Wave Radar

We propose a spiking neural network (SNN) approach for radar-based hand gesture recognition (HGR), using frequency modulated continuous wave (FMCW) millimeter-wave radar. After pre-processing the range-Doppler or micro-Doppler radar signal, we use a signal-to-spike conversion scheme that encodes radar Doppler maps into spike trains. The spike trains are fed into a spiking recurrent neural network, a liquid state machine (LSM). The readout spike signal from the SNN is then used as input for different classifiers for comparison, including logistic regression, random forest, and support vector machine (SVM). Using liquid state machines of less than 1000 neurons, we achieve better than state-of-the-art results on two publicly available reference datasets, reaching over 98% accuracy on 10-fold cross-validation for both data sets.

Download Full-text

Hand Gesture Recognition System for Parking Applications based on Machine Vision

2020 IEEE 3rd International Conference on Electronic Information and Communication Technology (ICEICT) ◽

10.1109/iceict51264.2020.9334259 ◽

2020 ◽

Author(s):

Jie Wang ◽

Laxmisha Rai

Keyword(s):

Machine Vision ◽

Gesture Recognition ◽

Recognition System ◽

Hand Gesture Recognition ◽

Hand Gesture

Download Full-text

A Meta-Learning-Based Approach for Hand Gesture Recognition Using FMCW Radar

2020 International Conference on Wireless Communications and Signal Processing (WCSP) ◽

10.1109/wcsp49889.2020.9299816 ◽

2020 ◽

Author(s):

Zhongyu Fan ◽

Haifeng Zheng ◽

Xinxin Feng

Keyword(s):

Gesture Recognition ◽

Hand Gesture Recognition ◽

Hand Gesture ◽

Fmcw Radar ◽

Meta Learning

Download Full-text

A Novel Illumination-Robust Hand Gesture Recognition System With Event-Based Neuromorphic Vision Sensor

IEEE Transactions on Automation Science and Engineering ◽

10.1109/tase.2020.3045880 ◽

2021 ◽

Vol 18 (2) ◽

pp. 508-520

Author(s):

Guang Chen ◽

Zhongcong Xu ◽

Zhijun Li ◽

Huajin Tang ◽

Sanqing Qu ◽

...

Keyword(s):

Gesture Recognition ◽

Recognition System ◽

Hand Gesture Recognition ◽

Hand Gesture ◽

Vision Sensor ◽

Event Based ◽

Neuromorphic Vision

Download Full-text

Surface EMG-Based Instantaneous Hand Gesture Recognition Using Convolutional Neural Network with the Transfer Learning Method

Sensors ◽

10.3390/s21072540 ◽

2021 ◽

Vol 21 (7) ◽

pp. 2540

Author(s):

Zhipeng Yu ◽

Jianghai Zhao ◽

Yucheng Wang ◽

Linglong He ◽

Shaonan Wang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Gesture Recognition ◽

Recognition System ◽

Surface Emg ◽

Hand Gesture Recognition ◽

Hand Gesture ◽

Training Time ◽

Generalization Ability

In recent years, surface electromyography (sEMG)-based human–computer interaction has been developed to improve the quality of life for people. Gesture recognition based on the instantaneous values of sEMG has the advantages of accurate prediction and low latency. However, the low generalization ability of the hand gesture recognition method limits its application to new subjects and new hand gestures, and brings a heavy training burden. For this reason, based on a convolutional neural network, a transfer learning (TL) strategy for instantaneous gesture recognition is proposed to improve the generalization performance of the target network. CapgMyo and NinaPro DB1 are used to evaluate the validity of our proposed strategy. Compared with the non-transfer learning (non-TL) strategy, our proposed strategy improves the average accuracy of new subject and new gesture recognition by 18.7% and 8.74%, respectively, when up to three repeated gestures are employed. The TL strategy reduces the training time by a factor of three. Experiments verify the transferability of spatial features and the validity of the proposed strategy in improving the recognition accuracy of new subjects and new gestures, and reducing the training burden. The proposed TL strategy provides an effective way of improving the generalization ability of the gesture recognition system.

Download Full-text