A comparison of simultaneously-obtained measures of listening effort: pupil dilation, verbal response time and self-rating

AbstractRecent research has demonstrated that pupillometry is a robust measure for quantifying listening effort. However, pupillary responses in listening situations where multiple cognitive functions are engaged and sustained over a period of time remain hard to interpret. This limits our conceptualisation and understanding of listening effort in realistic situations, because rarely in everyday life are people challenged by one task at a time. Therefore, the purpose of this experiment was to reveal the dynamics of listening effort in a sustained listening condition using a word repeat and recall task.Words were presented in quiet and speech-shaped noise at different signal-to-noise ratios (SNR). Participants were presented with lists of 10 words, and required to repeat each word after its presentation. At the end of the list, participants either recalled as many words as possible or moved on to the next list. Simultaneously, their pupil dilation was recorded throughout the whole experiment.When only word repeating was required, peak pupil dilation (PPD) was bigger in 0dB versus other conditions; whereas when recall was required, PPD showed no difference among SNR levels and PPD in 0dB was smaller than repeat-only condition. Baseline pupil diameter and PPD followed different growth patterns across the 10 serial positions in conditions requiring recall: baseline pupil diameter built up progressively and plateaued in the later positions (but shot up at the onset of recall, i.e. the end of the list); PPD decreased at a pace quicker than in repeat-only condition.The current findings concur with the recent literature in showing that additional cognitive load during a speech intelligibility task could disturb the well-established relation between pupillary response and listening effort. Both the magnitude and temporal pattern of task-evoked pupillary response differ greatly in complex listening conditions, urging for more listening effort studies in complex and realistic listening situations.

Download Full-text

Task-Induced Mental Fatigue and Motivation Influence Listening Effort as Measured by the Pupil Dilation in a Speech-in-Noise Task

10.1101/2022.01.04.22268734 ◽

2022 ◽

Author(s):

Defne Alfandari Menase ◽

Michael Richter ◽

Dorothea Wendt ◽

Lorenz Fiedler ◽

Graham Naylor

Keyword(s):

Quantitative Research ◽

Pupil Dilation ◽

Interactive Effects ◽

Monetary Incentives ◽

Listening Effort ◽

Experimental Conditions ◽

Heavy Load ◽

Speech In Noise ◽

Before And After ◽

Load Sequence

Objectives: Listening effort and fatigue are common complaints among individuals with hearing impairment (HI); however, the underlying mechanisms, and relationships between listening effort and fatigue are not well understood. Recent quantitative research suggests that the peak pupil dilation (PPD), which is commonly measured concurrent to the performance of a speech-in-noise task as an index of listening effort, may be informative of daily-life fatigue, but it remains unknown whether the same is true for task-induce fatigue. As fatigue effects are known to manifest differently depending on motivation, the main aim of the present study was to experimentally investigate the interactive effects of task-induced fatigue and motivation on the PPD. Design: In a pre-/post- fatigue within-subject design, 18 participants with normal hearing (NH) engaged in a 98-trial-long speech-in-noise task (i.e., a load sequence, approximately 40 min. long), which either excluded or included additional memory demands (light vs. heavy load sequence). Before and after the load sequence, baseline pupil diameter (BPD) and PPD were measured during shorter probe blocks of speech-in-noise tasks. In these probe blocks, if participants correctly repeated more than 60% of the keywords, they could win vouchers of either 20 or 160 Danish krones worth (low incentive vs. high incentive). After each probe block, participants reported their invested effort, tendency for quitting, and perceived performance. Results: The BPD in anticipation of listening declined from pre- to post-load sequence, suggesting an overall decrease in arousal, but the decline did not scale with the magnitude of the load sequence, nor with the amount of monetary incentive. Overall, there was larger pre- to post-load sequence decline in PPD when the load sequence was heavy and when the monetary incentives were low. Post-hoc analyses showed that the decline in PPD was only significant in the heavy-load sequence-low reward condition. The speech-in-noise task performance, self-reported effort, and self-reported tendency to quit listening did not change with the experimental conditions. Conclusions This is the first study to investigate the influence of task-induced fatigue on BPD and PPD. Whereas BPD was not sensitive to the magnitude of previous load sequence and monetary incentives, the decline in PPD from pre- to post- load sequence was significant after the heavy load sequence when the offered monetary incentives were low. This result supports the understanding that fatigue and motivation interactively influence listening effort.

Download Full-text

A comparison of two procedures for verbal response time fractionation

Frontiers in Psychology ◽

10.3389/fpsyg.2014.01213 ◽

2014 ◽

Vol 5 ◽

Cited By ~ 8

Author(s):

Lotje van der Linden ◽

StÃ©phanie K. RiÃ¨s ◽

Thierry Legou ◽

BorÃs Burle ◽

Nicole Malfait ◽

...

Keyword(s):

Response Time ◽

Verbal Response

Download Full-text

Disentangling listening effort and memory load beyond behavioural evidence: Pupillary response to listening effort during a concurrent memory task

PLoS ONE ◽

10.1371/journal.pone.0233251 ◽

2021 ◽

Vol 16 (3) ◽

pp. e0233251

Author(s):

Yue Zhang ◽

Alexandre Lehmann ◽

Mickael Deroche

Keyword(s):

Speech Intelligibility ◽

Memory Load ◽

Pupil Diameter ◽

Memory Task ◽

Pupillary Response ◽

Pupil Dilation ◽

Listening Effort ◽

Pupillary Responses ◽

Variation Patterns ◽

Behavioural Evidence

Recent research has demonstrated that pupillometry is a robust measure for quantifying listening effort. However, pupillary responses in listening situations where multiple cognitive functions are engaged and sustained over a period of time remain hard to interpret. This limits our conceptualisation and understanding of listening effort in realistic situations, because rarely in everyday life are people challenged by one task at a time. Therefore, the purpose of this experiment was to reveal the dynamics of listening effort in a sustained listening condition using a word repeat and recall task. Words were presented in quiet and speech-shaped noise at different signal-to-noise ratios (SNR): 0dB, 7dB, 14dB and quiet. Participants were presented with lists of 10 words, and required to repeat each word after its presentation. At the end of the list, participants either recalled as many words as possible or moved on to the next list. Simultaneously, their pupil dilation was recorded throughout the whole experiment. When only word repeating was required, peak pupil dilation (PPD) was bigger in 0dB versus other conditions; whereas when recall was required, PPD showed no difference among SNR levels and PPD in 0dB was smaller than repeat-only condition. Baseline pupil diameter and PPD followed different variation patterns across the 10 serial positions within a block for conditions requiring recall: baseline pupil diameter built up progressively and plateaued in the later positions (but shot up when listeners were recalling the previously heard words from memory); PPD decreased at a pace quicker than in repeat-only condition. The current findings demonstrate that additional cognitive load during a speech intelligibility task could disturb the well-established relation between pupillary response and listening effort. Both the magnitude and temporal pattern of task-evoked pupillary response differ greatly in complex listening conditions, urging for more listening effort studies in complex and realistic listening situations.

Download Full-text

The costs (and benefits) of effortful listening on context processing: A simultaneous electrophysiology, pupillometry, and behavioral study

10.31234/osf.io/f8pzg ◽

2020 ◽

Author(s):

Jack Silcox ◽

Brennan Payne

Keyword(s):

Word Processing ◽

Semantic Processing ◽

Pupil Dilation ◽

Word Recall ◽

N400 Effect ◽

Listening Effort ◽

Context Processing ◽

Sentence Recognition ◽

Degraded Speech ◽

Negative Effect

There is a disparity between the fields of cognitive audiology and cognitive electrophysiology as to how linguistic context is utilized when listening to perceptually challenging speech. To gain a clearer picture of how listening effort impacts context use, we conducted a pre-registered study to simultaneously examine electrophysiological, pupillometric, and behavioral responses when listening to sentences varying in contextual constraint and acoustic challenge in the same sample. Participants (N = 44) listened to sentences that were highly constraining and completed with expected or unexpected sentence-final words (“The prisoners were planning their escape/party”) or were low-constraint sentences with unexpected sentence-final words (“All day she thought about the party”). Sentences were presented either in quiet or with +3 dB SNR background noise. Pupillometry and EEG were simultaneously recorded and subsequent sentence recognition and word recall were measured. While the N400 expectancy effect was diminished by noise, suggesting impaired real-time context use, we simultaneously observed a beneficial effect of constraint on subsequent recognition memory for degraded speech. Importantly, analyses of trial-to-trial coupling between pupil dilation and N400 amplitude showed that when participants’ showed increased listening effort (i.e., greater pupil dilation), there was a subsequent recovery of the N400 effect, but at the same time, higher effort was related to poorer subsequent sentence recognition and word recall. Collectively, these findings suggest divergent effects of acoustic challenge and listening effort on context use: while noise impairs the rapid use of context to facilitate lexical semantic processing in general, this negative effect is attenuated when listeners increase effort in response to noise. However, this effort-induced reliance on context for online word processing comes at the cost of poorer subsequent memory.

Download Full-text

Impact of Background Noise Fluctuation and Reverberation on Response Time in a Speech Reception Task

Journal of Speech Language and Hearing Research ◽

10.1044/2019_jslhr-h-19-0180 ◽

2019 ◽

Vol 62 (11) ◽

pp. 4179-4195 ◽

Cited By ~ 1

Author(s):

Nicola Prodi ◽

Chiara Visentin

Keyword(s):

Response Time ◽

Speech Intelligibility ◽

Response Selection ◽

Normal Hearing ◽

Signal To Noise ◽

Listening Effort ◽

Closed Set ◽

Slowing Down ◽

Speech Reception ◽

Wide Range

Purpose This study examines the effects of reverberation and noise fluctuation on the response time (RT) to the auditory stimuli in a speech reception task. Method The speech reception task was presented to 76 young adults with normal hearing in 3 simulated listening conditions (1 anechoic, 2 reverberant). Speechlike stationary and fluctuating noise were used as maskers, in a wide range of signal-to-noise ratios. The speech-in-noise tests were presented in a closed-set format; data on speech intelligibility and RT (time elapsed from the offset of the auditory stimulus to the response selection) were collected. A slowing down in RTs was interpreted as an increase in listening effort. Results RTs slowed down in the more challenging signal-to-noise ratios, with increasing reverberation and for stationary compared to fluctuating noise, consistently with a fluctuating masking release scheme. When speech intelligibility was fixed, it was found that the estimated RTs were similar or faster for stationary compared to fluctuating noise, depending on the amount of reverberation. Conclusions The current findings add to the literature on listening effort for listeners with normal hearing by indicating that the addition of reverberation to fluctuating noise increases RT in a speech reception task. The results support the importance of integrating noise and reverberation to provide accurate predictors of real-world performance in clinical settings.

Download Full-text

The Impact of Auditory Spectral Resolution on Listening Effort Revealed by Pupil Dilation

Ear and Hearing ◽

10.1097/aud.0000000000000145 ◽

2015 ◽

Vol 36 (4) ◽

pp. e153-e165 ◽

Cited By ~ 84

Author(s):

Matthew B. Winn ◽

Jan R. Edwards ◽

Ruth Y. Litovsky

Keyword(s):

Spectral Resolution ◽

Pupil Dilation ◽

Listening Effort ◽

The Impact

Download Full-text

Listening Effort in School-Aged Children With Limited Useable Hearing Unilaterally: Examining the Effects of a Personal, Digital Remote Microphone System and a Contralateral Routing of Signal System

Trends in Hearing ◽

10.1177/2331216520984700 ◽

2021 ◽

Vol 25 ◽

pp. 233121652098470

Author(s):

Ilze Oosthuizen ◽

Erin M. Picou ◽

Lidia Pottas ◽

Hermanus C. Myburgh ◽

De Wet Swanepoel

Keyword(s):

Speech Recognition ◽

Response Times ◽

Signal To Noise Ratio ◽

Recognition Task ◽

Verbal Response ◽

Behavioral Measure ◽

Subjective Ratings ◽

Listening Effort ◽

Behavioral Measures ◽

School Aged Children

Technology options for children with limited hearing unilaterally that improve the signal-to-noise ratio are expected to improve speech recognition and also reduce listening effort in challenging listening situations, although previous studies have not confirmed this. Employing behavioral and subjective indices of listening effort, this study aimed to evaluate the effects of two intervention options, remote microphone system (RMS) and contralateral routing of signal (CROS) system, in school-aged children with limited hearing unilaterally. Nineteen children (aged 7–12 years) with limited hearing unilaterally completed a digit triplet recognition task in three loudspeaker conditions: midline, monaural direct, and monaural indirect with three intervention options: unaided, RMS, and CROS system. Verbal response times were interpreted as a behavioral measure of listening effort. Participants provided subjective ratings immediately following behavioral measures. The RMS significantly improved digit triplet recognition across loudspeaker conditions and reduced verbal response times in the midline and indirect conditions. The CROS system improved speech recognition and listening effort only in the indirect condition. Subjective ratings analyses revealed that significantly more participants indicated that the remote microphone made it easier for them to listen and to stay motivated. Behavioral and subjective indices of listening effort indicated that an RMS provided the most consistent benefit for speech recognition and listening effort for children with limited unilateral hearing. RMSs could therefore be a beneficial technology option in classrooms for children with limited hearing unilaterally.

Download Full-text