Auditory-visual speech recognition by hearing-impaired subjects: Consonant recognition, sentence recognition, and auditory-visual integration

1998 ◽  
Vol 103 (5) ◽  
pp. 2677-2690 ◽  
Author(s):  
Ken W. Grant ◽  
Brian E. Walden ◽  
Philip F. Seitz
Keyword(s):  
Speech Recognition ◽  
Hearing Impaired ◽  
Visual Speech ◽  
Visual Integration ◽  
Sentence Recognition ◽  
Visual Speech Recognition ◽  
Consonant Recognition

Some Effects of Training on Speech Recognition by Hearing-Impaired Adults

1981 ◽  
Vol 24 (2) ◽  
pp. 207-216 ◽  
Author(s):  
Brian E. Walden ◽  
Sue A. Erdman ◽  
Allen A. Montgomery ◽  
Daniel M. Schwartz ◽  
Robert A. Prosek
Keyword(s):  
Speech Recognition ◽  
Recognition Test ◽  
Recognition Performance ◽  
Rehabilitation Program ◽  
Hearing Impaired ◽  
Aural Rehabilitation ◽  
Sentence Recognition ◽  
Consonant Recognition ◽  
The Individual

The purpose of this research was to determine some of the effects of consonant recognition training on the speech recognition performance of hearing-impaired adults. Two groups of ten subjects each received seven hours of either auditory or visual consonant recognition training, in addition to a standard two-week, group-oriented, inpatient aural rehabilitation program. A third group of fifteen subjects received the standard two-week program, but no supplementary individual consonant recognition training. An audiovisual sentence recognition test, as well as tests of auditory and visual consonant recognition, were administered both before and ibltowing training. Subjects in all three groups significantly increased in their audiovisual sentence recognition performance, but subjects receiving the individual consonant recognition training improved significantly more than subjects receiving only the standard two-week program. A significant increase in consonant recognition performance was observed in the two groups receiving the auditory or visual consonant recognition training. The data are discussed from varying statistical and clinical perspectives.

Psychologically-Inspired Audio-Visual Speech Recognition Using Coarse Speech Recognition and Missing Feature Theory

2017 ◽  
Vol 29 (1) ◽  
pp. 105-113 ◽  
Author(s):  
Kazuhiro Nakadai ◽  
◽  
Tomoaki Koiwa ◽  
Keyword(s):  
Speech Recognition ◽  
Real World ◽  
Visual Speech ◽  
Noise Robustness ◽  
Visual Integration ◽  
The Real ◽  
Visual Speech Recognition ◽  
Missing Feature Theory ◽  
Missing Feature ◽  
Feature Theory

[abstFig src='/00290001/10.jpg' width='300' text='System architecture of AVSR based on missing feature theory and P-V grouping' ] Audio-visual speech recognition (AVSR) is a promising approach to improving the noise robustness of speech recognition in the real world. For AVSR, the auditory and visual units are the phoneme and viseme, respectively. However, these are often misclassified in the real world because of noisy input. To solve this problem, we propose two psychologically-inspired approaches. One is audio-visual integration based on missing feature theory (MFT) to cope with missing or unreliable audio and visual features for recognition. The other is phoneme and viseme grouping based on coarse-to-fine recognition. Preliminary experiments show that these two approaches are effective for audio-visual speech recognition. Integration based on MFT with an appropriate weight improves the recognition performance by −5 dB. This is the case even in a noisy environment, in which most speech recognition systems do not work properly. Phoneme and viseme grouping further improved the AVSR performance, particularly at a low signal-to-noise ratio.**This work is an extension of our publication “Tomoaki Koiwa et al.: Coarse speech recognition by audio-visual integration based on missing feature theory, IROS 2007, pp.1751-1756, 2007.”

Asynchrony modeling for audio-visual speech recognition

2002 ◽  
Author(s):  
Guillaume Gravier ◽  
Gerasimos Potamianos ◽  
Chalapathy Neti
Keyword(s):  
Speech Recognition ◽  
Visual Speech ◽  
Visual Speech Recognition

scholarly journals Comparison between different feature extraction techniques for audio-visual speech recognition

2007 ◽  
Vol 1 (1) ◽  
pp. 7-20 ◽  
Author(s):  
Alin G. Chiţu ◽  
Leon J. M. Rothkrantz ◽  
Pascal Wiggers ◽  
Jacek C. Wojdel
Keyword(s):  
Feature Extraction ◽  
Speech Recognition ◽  
Visual Speech ◽  
Extraction Techniques ◽  
Visual Speech Recognition

European and American Audio-Visual Speech Recognition, Using SVM in Portuguese Language

2008 ◽  
Author(s):  
Adriano de Andrade Bresolin ◽  
Diamantino Rui da Silva da Silva Freitas ◽  
Adriao Duarte Doria Neto ◽  
Pablo Javier Alsina
Keyword(s):  
Speech Recognition ◽  
Visual Speech ◽  
Visual Speech Recognition
Sign in / Sign up

Export Citation Format

Share Document