Hand gesture recognition scheme based on millimeter-wave radar with convolutional neural network

2021 ◽  
Author(s):  
S. Wang ◽  
Z. Li ◽  
R. Huang ◽  
R. Wang ◽  
J. Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Millimeter Wave ◽  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Millimeter Wave Radar ◽  
Wave Radar

scholarly journals On the Effect of Training Convolution Neural Network for Millimeter-Wave Radar-Based Hand Gesture Recognition

Sensors ◽  
2021 ◽  
Vol 21 (1) ◽  
pp. 259
Author(s):  
Kang Zhang ◽  
Shengchang Lan ◽  
Guiyuan Zhang
Keyword(s):  
Neural Network ◽  
Millimeter Wave ◽  
Gesture Recognition ◽  
Radar Data ◽  
Hand Gesture Recognition ◽  
Convolution Neural Network ◽  
Hand Gesture ◽  
Millimeter Wave Radar ◽  
Average Accuracy ◽  
Wave Radar

The purpose of this paper was to investigate the effect of a training state-of-the-art convolution neural network (CNN) for millimeter-wave radar-based hand gesture recognition (MR-HGR). Focusing on the small training dataset problem in MR-HGR, this paper first proposed to transfer the knowledge with the CNN models in computer vision to MR-HGR by fine-tuning the models with radar data samples. Meanwhile, for the different data modality in MR-HGR, a parameterized representation of temporal space-velocity (TSV) spectrogram was proposed as an integrated data modality of the time-evolving hand gesture features in the radar echo signals. The TSV spectrograms representing six common gestures in human–computer interaction (HCI) from nine volunteers were used as the data samples in the experiment. The evaluated models included ResNet with 50, 101, and 152 layers, DenseNet with 121, 161 and 169 layers, as well as light-weight MobileNet V2 and ShuffleNet V2, mostly proposed by many latest publications. In the experiment, not only self-testing (ST), but also more persuasive cross-testing (CT), were implemented to evaluate whether the fine-tuned models generalize to the radar data samples. The CT results show that the best fine-tuned models can reach to an average accuracy higher than 93% with a comparable ST average accuracy almost 100%. Moreover, in order to alleviate the problem caused by private gesture habits, an auxiliary test was performed by augmenting four shots of the gestures with the heaviest misclassifications into the training set. This enriching test is similar with the scenario that a tablet reacts to a new user. The results of two different volunteer in the enriching test shows that the average accuracy of the enriched gesture can be improved from 55.59% and 65.58% to 90.66% and 95.95% respectively. Compared with some baseline work in MR-HGR, the investigation by this paper can be beneficial in promoting MR-HGR in future industry applications and consumer electronic design.

scholarly journals Spectrum-Based Hand Gesture Recognition Using Millimeter-Wave Radar Parameter Measurements

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 79147-79158 ◽  
Author(s):  
Changjiang Liu ◽  
Yuanhao Li ◽  
Dongyang Ao ◽  
Haiyan Tian
Keyword(s):  
Millimeter Wave ◽  
Gesture Recognition ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Millimeter Wave Radar ◽  
Wave Radar

scholarly journals Surface EMG-Based Instantaneous Hand Gesture Recognition Using Convolutional Neural Network with the Transfer Learning Method

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2540
Author(s):  
Zhipeng Yu ◽  
Jianghai Zhao ◽  
Yucheng Wang ◽  
Linglong He ◽  
Shaonan Wang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Gesture Recognition ◽  
Recognition System ◽  
Surface Emg ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Training Time ◽  
Generalization Ability

In recent years, surface electromyography (sEMG)-based human–computer interaction has been developed to improve the quality of life for people. Gesture recognition based on the instantaneous values of sEMG has the advantages of accurate prediction and low latency. However, the low generalization ability of the hand gesture recognition method limits its application to new subjects and new hand gestures, and brings a heavy training burden. For this reason, based on a convolutional neural network, a transfer learning (TL) strategy for instantaneous gesture recognition is proposed to improve the generalization performance of the target network. CapgMyo and NinaPro DB1 are used to evaluate the validity of our proposed strategy. Compared with the non-transfer learning (non-TL) strategy, our proposed strategy improves the average accuracy of new subject and new gesture recognition by 18.7% and 8.74%, respectively, when up to three repeated gestures are employed. The TL strategy reduces the training time by a factor of three. Experiments verify the transferability of spatial features and the validity of the proposed strategy in improving the recognition accuracy of new subjects and new gestures, and reducing the training burden. The proposed TL strategy provides an effective way of improving the generalization ability of the gesture recognition system.

Sign in / Sign up

Export Citation Format

Share Document