Assistance of Speech Recognition in Noisy Environment with Sentence Level Lip-Reading

2017 ◽  
pp. 593-601
Author(s):  
Jianzong Wang ◽  
Yiwen Wang ◽  
Aozhi Liu ◽  
Jing Xiao
Keyword(s):  
Speech Recognition ◽  
Noisy Environment ◽  
Sentence Level ◽  
Lip Reading

scholarly journals Extraction of Lip features for the Identification of Vowels Utterances using MFCC and Geometrical Aspects

2020 ◽  
Vol 8 (5) ◽  
pp. 3978-3983
Keyword(s):  
Speech Recognition ◽  
Software Tools ◽  
Minor Axis ◽  
Noisy Environment ◽  
Video File ◽  
Feature Vectors ◽  
Individual Person ◽  
Lip Reading ◽  
Lip Movement

Identification of a person’s speech by his lip movement is a challenging task. Even though many software tools available for recognition of speech to text and vice versa, some of the words uttered may not be accurate as spoken and may vary from person to person because of their pronunciation. In addition, in the noisy environment speech uttered may not perceive effectively hence there lip movement for a given speech varies. Lip reading has added advantages when it augmented with speech recognition, thus increasing the perceived information. In this paper, the video file of a individual person are converted to frames and extraction of only the lip contour for vowels is done by calculating its area and other geometrical aspects. Once this is done as a part of testing it is compared with three to four people’s lip contour for vowels for first 20 frames. The parameters such as mean, centroid will remain approximately same for all people irrespective of their lip movement but there is change in major and minor axis and hence area changes considerably. In audio domain vowel detection is carried out by extracting unique features of English vowel utterance using Mel Frequency Cepstrum Coefficients (MFCC) and the feature vectors that are orthonormalized to compare the normalized vectors with standard database and results are obtained with approximation.

scholarly journals Research on Speaker Recognition of DRNN in Different Noise Environment

2020 ◽  
Author(s):  
chaofeng lan ◽  
yuanyuan Zhang ◽  
hongyun Zhao
Keyword(s):  
Neural Network ◽  
Speech Recognition ◽  
Recurrent Neural Network ◽  
Speaker Recognition ◽  
Signal To Noise Ratio ◽  
Recognition Rate ◽  
Noisy Environment ◽  
Signal To Noise ◽  
Noise Ratio ◽  
Improved Model

Abstract This paper draws on the training method of Recurrent Neural Network (RNN), By increasing the number of hidden layers of RNN and changing the layer activation function from traditional Sigmoid to Leaky ReLU on the input layer, the first group and the last set of data are zero-padded to enhance the effective utilization of data such that the improved reduction model of Denoise Recurrent Neural Network (DRNN) with high calculation speed and good convergence is constructed to solve the problem of low speaker recognition rate in noisy environment. According to this model, the random semantic speech signal with a sampling rate of 16 kHz and a duration of 5 seconds in the speech library is studied. The experimental settings of the signal-to-noise ratios are − 10dB, -5dB, 0dB, 5dB, 10dB, 15dB, 20dB, 25dB. In the noisy environment, the improved model is used to denoise the Mel Frequency Cepstral Coefficients (MFCC) and the Gammatone Frequency Cepstral Coefficents (GFCC), impact of the traditional model and the improved model on the speech recognition rate is analyzed. The research shows that the improved model can effectively eliminate the noise of the feature parameters and improve the speech recognition rate. When the signal-to-noise ratio is low, the speaker recognition rate can be more obvious. Furthermore, when the signal-to-noise ratio is 0dB, the speaker recognition rate of people is increased by 40%, which can be 85% improved compared with the traditional speech model. On the other hand, with the increase in the signal-to-noise ratio, the recognition rate is gradually increased. When the signal-to-noise ratio is 15dB, the recognition rate of speakers is 93%.

Motion Features for Visual Speech Recognition

2009 ◽  
pp. 388-415 ◽  
Author(s):  
Wai Chee Yau ◽  
Dinesh Kant Kumar ◽  
Hans Weghorn
Keyword(s):  
Speech Recognition ◽  
Optical Flow ◽  
Visual Speech ◽  
Image Subtraction ◽  
Phoneme Classification ◽  
Visual Speech Recognition ◽  
Lip Reading ◽  
Motion Features ◽  
Spatio Temporal ◽  
Mouth Movement

The performance of a visual speech recognition technique is greatly influenced by the choice of visual speech features. Speech information in the visual domain can be generally categorized into static (mouth appearance) and motion (mouth movement) features. This chapter reviews a number of computer-based lip-reading approaches using motion features. The motion-based visual speech recognition techniques can be broadly categorized into two types of algorithms: optical-flow and image subtraction. Image subtraction techniques have been demonstrated to outperform optical-flow based methods in lip-reading. The problem with image subtraction-based method using difference of frames (DOF) is that these features capture the changes in the images over time, but do not indicate the direction of the mouth movement. New motion features to overcome the limitation of the conventional image subtraction-based techniques in visual speech recognition are presented in this chapter. The proposed approach extracts features by applying motion segmentation on image sequences. Video data are represented in a 2-D space using grayscale images named as motion history images (MHI). MHIs are spatio-temporal templates that implicitly encode the temporal component of mouth movement. Zernike moments are computed from MHIs as image descriptors and classified using support vector machines (SVMs). Experimental results demonstrate that the proposed technique yield a high accuracy in a phoneme classification task. The results suggest that dynamic information is important for visual speech recognition.

scholarly journals Development of Intelligent Terminal-based Signal Acquisition System for Cochlear Implant

2018 ◽  
Vol 232 ◽  
pp. 01015 ◽  
Author(s):  
Yousheng Chen ◽  
Weifang Chen
Keyword(s):  
Speech Recognition ◽  
Cochlear Implant ◽  
Acquisition System ◽  
Signal Acquisition ◽  
Noisy Environment ◽  
System Monitoring ◽  
Intelligent Monitoring ◽  
Intelligent Terminal ◽  
Front End ◽  
Mismatch Condition

Present speech recognition of cochlear implant is still low for situation of noisy environment or under mismatch condition, and more researches focus on improving front-end signal acquisition and speech recognition. To simplify signal acquisition and algorithm research, we develop an intelligent terminal-based signal acquisition system for cochlear implant, in which the electric relay and many sensors are adopted to implement system monitoring function. The proposed system platform is helpful to actualize algorithm research and intelligent monitoring, adding to its value of further research of speech recognition improvement.

Sign in / Sign up

Export Citation Format

Share Document