PurposeSpeech recognition percent-correct scores fail to capture the effort of mentally repairing the perception of speech that was initially misheard. This study includes two experiments to simulate and measure the effort of this experience of mental repair, from the perspective of both listener and talker.MethodsCochlear implant (CI) listeners heard and repeated sentences where single words were distorted or masked by noise, but recoverable based on later context – a signature of mental repair. Changes in pupil dilation were tracked as an index of effort and time-locked with specific landmarks during perception. Later, audio recordings of the participants’ responses were played for outside observers who judged whether they thought the CI user mentally repaired a word.ResultsEffort significantly increases when a listener needs to repair a misperceived word, even if the verbal response is ultimately correct. That increased effort lasts longer for CI users than for listeners with normal hearing. There was no success in detecting that effort by nearly 200 external observers including audiologists and others who profess to have experience and skill in conversing with people who have hearing loss. Mental repair of words in a sentence were accompanied by greater prevalence of errors elsewhere in the same sentence, suggesting that effort spreads to consume resources across time. ConclusionsListening effort is better explained by the mental repair and reconstruction of words rather than the appearance of correct or incorrect perception. Linguistic coherence drives effort more heavily than the mere presence of mistakes, highlighting the importance of effort that cannot be tracked with individual words or digits. However, an external observer is not able to reliably detect that extra effort in the listener’s voice.
The effortful and invisible cost of repairing speech misperceptions by individuals who use cochlear implants
