Consequences of audiovisual asynchrony for speech perception: Implications for signal processing in aids to lipreading

AbstractSloping hearing loss imposes limits on audibility for high-frequency sounds in many hearing aid users. Signal processing algorithms that shift high-frequency sounds to lower frequencies have been introduced in hearing aids to address this challenge by improving audibility of high-frequency sounds.This study examined speech perception performance, listening effort, and subjective sound quality ratings with conventional hearing aid processing and a new frequency-lowering signal processing strategy called frequency composition (FC) in adults and children.Participants wore the study hearing aids in two signal processing conditions (conventional processing versus FC) at an initial laboratory visit and subsequently at home during two approximately six-week long trials, with the order of conditions counterbalanced across individuals in a double-blind paradigm.Children (N = 12, 7 females, mean age in years = 12.0, SD = 3.0) and adults (N = 12, 6 females, mean age in years = 56.2, SD = 17.6) with bilateral sensorineural hearing loss who were full-time hearing aid users.Individual performance with each type of processing was assessed using speech perception tasks, a measure of listening effort, and subjective sound quality surveys at an initial visit. At the conclusion of each subsequent at-home trial, participants were retested in the laboratory. Linear mixed effects analyses were completed for each outcome measure with signal processing condition, age group, visit (prehome versus posthome trial), and measures of aided audibility as predictors.Overall, there were few significant differences in speech perception, listening effort, or subjective sound quality between FC and conventional processing, effects of listener age, or longitudinal changes in performance. Listeners preferred FC to conventional processing on one of six subjective sound quality metrics. Better speech perception performance was consistently related to higher aided audibility.These results indicate that when high-frequency speech sounds are made audible with conventional processing, speech recognition ability and listening effort are similar between conventional processing and FC. Despite the lack of benefit to speech perception, some listeners still preferred FC, suggesting that qualitative measures should be considered when evaluating candidacy for this signal processing strategy.

Download Full-text

Study of audiovisual asynchrony signal processing: Robot recognition system of different ages

2016 IEEE International Conference on Mechatronics and Automation ◽

10.1109/icma.2016.7558927 ◽

2016 ◽

Author(s):

Yanna Ren ◽

Weiping Yang ◽

Qiong Wu ◽

Fengxia Wu ◽

Satoshi Takahashi ◽

...

Keyword(s):

Signal Processing ◽

Recognition System ◽

Different Ages ◽

Audiovisual Asynchrony

Download Full-text

Perception of audio-visual speech synchrony in Spanish-speaking children with and without specific language impairment

Journal of Child Language ◽

10.1017/s0305000912000189 ◽

2012 ◽

Vol 40 (3) ◽

pp. 687-700 ◽

Cited By ~ 18

Author(s):

FERRAN PONS ◽

LLORENÇ ANDREU ◽

MONICA SANZ-TORRENT ◽

LUCÍA BUIL-LEGAZ ◽

DAVID J. LEWKOWICZ

Keyword(s):

Speech Perception ◽

Speech Processing ◽

Language Impairment ◽

Specific Language Impairment ◽

Visual Speech ◽

Specific Language ◽

Visual Component ◽

Spanish Speaking ◽

Auditory Component ◽

Audiovisual Asynchrony

ABSTRACTSpeech perception involves the integration of auditory and visual articulatory information, and thus requires the perception of temporal synchrony between this information. There is evidence that children with specific language impairment (SLI) have difficulty with auditory speech perception but it is not known if this is also true for the integration of auditory and visual speech. Twenty Spanish-speaking children with SLI, twenty typically developing age-matched Spanish-speaking children, and twenty Spanish-speaking children matched for MLU-w participated in an eye-tracking study to investigate the perception of audiovisual speech synchrony. Results revealed that children with typical language development perceived an audiovisual asynchrony of 666 ms regardless of whether the auditory or visual speech attribute led the other one. Children with SLI only detected the 666 ms asynchrony when the auditory component followed the visual component. None of the groups perceived an audiovisual asynchrony of 366 ms. These results suggest that the difficulty of speech processing by children with SLI would also involve difficulties in integrating auditory and visual aspects of speech perception.

Download Full-text

Audiovisual asynchrony detection and speech perception in hearing-impaired listeners with cochlear implants: A preliminary analysis

International Journal of Audiology ◽

10.1080/14992020802644871 ◽

2009 ◽

Vol 48 (6) ◽

pp. 321-333 ◽

Cited By ~ 26

Author(s):

Marcia J. Hay-McCutcheon ◽

David B. Pisoni ◽

Kristopher K. Hunt

Keyword(s):

Speech Perception ◽

Cochlear Implants ◽

Preliminary Analysis ◽

Hearing Impaired ◽

Asynchrony Detection ◽

Audiovisual Asynchrony

Download Full-text

Clinical Outcomes of the Cochlear™ Nucleus® 5 Cochlear Implant System and SmartSound™ 2 Signal Processing

Journal of the American Academy of Audiology ◽

10.3766/jaaa.15021 ◽

2016 ◽

Vol 27 (06) ◽

pp. 425-440 ◽

Cited By ~ 7

Author(s):

Christina L. Runge ◽

Kathryn Henion ◽

Sergey Tarima ◽

Anne Beiter ◽

Teresa A. Zwolan

Keyword(s):

Quality Of Life ◽

United States ◽

Clinical Trials ◽

Signal Processing ◽

Speech Perception ◽

The United States ◽

Hearing In Noise ◽

Noise Test ◽

Listening In Noise

Background: While published data exist regarding cochlear implant (CI) outcomes from large academic programs, evidence of benefit based on national, multicenter clinical trials is needed for information regarding typical patient outcomes of devices implanted by U.S. centers representing larger academic to smaller hospital-based programs. Purpose: This nationwide trial evaluated outcomes in a group of newly implanted adult recipients of the Cochlear™ Nucleus® 5 CI system and SmartSound™ 2 signal processing. Unlike previous clinical trials, the AzBio sentence test was used and represents recent transition in our field to use of more challenging test materials. It was hypothesized that (1) speech perception scores in quiet with SmartSound™ 2 signal processing would not be statistically different from previous-generation devices; (2) speech perception scores in noise with SmartSound™ 2 signal processing would be better with enhanced microphone directionality; (3) speech perception scores in noise will be better with the preferred SmartSound™ 2 program for listening in noise; and (4) cochlear implantation would improve quality of life as assessed by the updated Health Utility Index Mark 3 (HUI3). A secondary purpose was to examine the relationships among the current and previously used speech perception tests of the Minimum Speech Test Battery (MSTB). It was hypothesized that speech perception scores within the same test interval would show predictive relationships. Research Design: Prospective, single-arm, repeated-measures study across 13 CI centers in the United States between February 2010 and June 2012. The participating centers ranged from larger academic to smaller hospital-based programs to accurately represent the diversity of programs in the United States. Study Sample: Participants were 38 postlingually deafened adult CI candidates. Data Collection and Analysis: Primary measures were Consonant-Nucleus-Consonant (CNC) words in quiet and the AzBio Sentence Test in Quiet (AzBioQ) and in Noise (AzBioN) tested at preoperative, and 3-, 6-, and 12-mo postactivation intervals. Quality of life was measured with the HUI3. For the secondary objective, statistical analyses were performed to investigate the predictive properties between current and previously used MSTB tests. Results: Mean CNC scores were significantly higher compared to the Nucleus® 24 Contour™ at 3 mo (p < 0.05) postactivation and showed no difference compared to the Nucleus® Freedom™ at 6 mo postactivation. Both SmartSound™ 2 FOCUS and NOISE programs provided significant improvements in performance in noise over the EVERYDAY program (p < 0.001), and performance with the FOCUS program was significantly better compared to the NOISE program (p < 0.001). Speech perception in noise was not related to patients’ subjective program preferences. Quality-of-life outcomes showed significant improvements from the preoperative to 6-mo postactivation interval (p < 0.05–0.001). Strong and significant correlations were found between preoperative CNC and AzBioQ and preoperative Hearing-in-Noise Test sentences in Quiet (HINTQ) and AzBioQ. At 12-mo postactivation, there were strong and highly significant correlations between CNC and AzBioQ, HINTQ and AzBioQ, and Hearing-in-Noise Test sentences in Noise and AzBioN (all p < 0.001). Conclusions: Results of this national clinical trial showed significant improvements in speech perception and quality of life following cochlear implantation. SmartSound™ 2 signal processing features showed a significant benefit of FOCUS when listening in noise, although preference of signal processing feature did not correlate with performance. Significant correlations were observed between speech perception tests. The findings of this study can be applied in clinical assessment, programming, and follow-up for CI candidates and recipients.

Download Full-text