scholarly journals Comparison between different feature extraction techniques for audio-visual speech recognition

2007 ◽  
Vol 1 (1) ◽  
pp. 7-20 ◽  
Author(s):  
Alin G. Chiţu ◽  
Leon J. M. Rothkrantz ◽  
Pascal Wiggers ◽  
Jacek C. Wojdel
Keyword(s):  
Feature Extraction ◽  
Speech Recognition ◽  
Visual Speech ◽  
Extraction Techniques ◽  
Visual Speech Recognition

scholarly journals Visual Speech Recognition with Lightweight Psychologically Motivated Gabor Features

Entropy ◽  
2020 ◽  
Vol 22 (12) ◽  
pp. 1367
Author(s):  
Xuejie Zhang ◽  
Yan Xu ◽  
Andrew K. Abel ◽  
Leslie S. Smith ◽  
Roger Watt ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Speech Recognition ◽  
Recognition Rate ◽  
Psychological Research ◽  
Visual Speech ◽  
Visual Speech Recognition ◽  
Gabor Features ◽  
Image Patches ◽  
Low Dimensionality ◽  
The Cost

Extraction of relevant lip features is of continuing interest in the visual speech domain. Using end-to-end feature extraction can produce good results, but at the cost of the results being difficult for humans to comprehend and relate to. We present a new, lightweight feature extraction approach, motivated by human-centric glimpse-based psychological research into facial barcodes, and demonstrate that these simple, easy to extract 3D geometric features (produced using Gabor-based image patches), can successfully be used for speech recognition with LSTM-based machine learning. This approach can successfully extract low dimensionality lip parameters with a minimum of processing. One key difference between using these Gabor-based features and using other features such as traditional DCT, or the current fashion for CNN features is that these are human-centric features that can be visualised and analysed by humans. This means that it is easier to explain and visualise the results. They can also be used for reliable speech recognition, as demonstrated using the Grid corpus. Results for overlapping speakers using our lightweight system gave a recognition rate of over 82%, which compares well to less explainable features in the literature.

Sign in / Sign up

Export Citation Format

Share Document