Listeners' Perceptions of Speaker Personality Traits Based on Speech

Purpose Voice and speech are rich with information about a speaker's personality and other features of identity. This study seeks to determine the extent to which listeners agree about speakers' social, physical, and personality attributes. Method Two experiments were conducted. In the first experiment, listeners rated a group of speakers who were unbalanced for sex and personality traits. The second experiment elaborated on the first by ensuring the speaker set was balanced for sex and personality traits. Both experiments played standard speech samples from speakers who provided personality information via the Multidimensional Personality Questionnaire–Brief Form. Groups of listeners rated each speaker on the same personality traits and other features of identity. Responses were analyzed for listener agreement. Results For both experiments, listeners showed consistently high levels of agreement on the personality attributes of a speaker. For certain speakers, listener agreement on some personality traits was as high as 92% and 97% in Experiments 1 and 2, respectively. Furthermore, a range of agreement across personality subscales was observed across speakers such that some were agreed-upon across all personality ratings and others were agreed-upon only for a few personality traits. Conclusions When it comes to judging personality traits and other features of identity, most listeners might not be “correct” about speakers' traits and attributes, but they broadly agree about how the listener sounds. Some speakers send more salient voice and speech cues that drive agreement about their personality, whereas others speak in a manner that precludes consensus. Supplemental Material https://doi.org/10.23641/asha.16906990

COVID-19, Face Masks, and Social Interaction: How a Global Pandemic Is Shining a Light on the Importance of Face-to-Face Communication

Purpose The purpose of this article is to discuss the impact of COVID-19 mitigation strategies on face-to-face communication. The article covers three main areas: the effect of face masks and social distancing on aspects of communication, including speech production, speech recognition, and emotion recognition; the effect of face masks on access to visual speech cues; and downstream effects of poor speech recognition on language and cognitive abilities. Conclusions The use of face masks and social distancing are proven mitigation strategies to minimize the spread of COVID-19 and other airborne diseases. However, these strategies may place individuals with hearing, speech, and language disorders at greater risk for communication problems in their daily lives. Practitioners who work directly with these patients should consider these issues when discussing communication strategies with their patients.

Eye Gaze and Perceptual Adaptation to Audiovisual Degraded Speech

Purpose Visual cues from a speaker's face may benefit perceptual adaptation to degraded speech, but current evidence is limited. We aimed to replicate results from previous studies to establish the extent to which visual speech cues can lead to greater adaptation over time, extending existing results to a real-time adaptation paradigm (i.e., without a separate training period). A second aim was to investigate whether eye gaze patterns toward the speaker's mouth were related to better perception, hypothesizing that listeners who looked more at the speaker's mouth would show greater adaptation. Method A group of listeners ( n = 30) was presented with 90 noise-vocoded sentences in audiovisual format, whereas a control group ( n = 29) was presented with the audio signal only. Recognition accuracy was measured throughout and eye tracking was used to measure fixations toward the speaker's eyes and mouth in the audiovisual group. Results Previous studies were partially replicated: The audiovisual group had better recognition throughout and adapted slightly more rapidly, but both groups showed an equal amount of improvement overall. Longer fixations on the speaker's mouth in the audiovisual group were related to better overall accuracy. An exploratory analysis further demonstrated that the duration of fixations to the speaker's mouth decreased over time. Conclusions The results suggest that visual cues may not benefit adaptation to degraded speech as much as previously thought. Longer fixations on a speaker's mouth may play a role in successfully decoding visual speech cues; however, this will need to be confirmed in future research to fully understand how patterns of eye gaze are related to audiovisual speech recognition. All materials, data, and code are available at https://osf.io/2wqkf/ .

The Relevance of the Availability of Visual Speech Cues During Adaptation to Noise-Vocoded Speech

Purpose This study first aimed to establish whether viewing specific parts of the speaker's face (eyes or mouth), compared to viewing the whole face, affected adaptation to distorted noise-vocoded sentences. Second, this study also aimed to replicate results on processing of distorted speech from lab-based experiments in an online setup. Method We monitored recognition accuracy online while participants were listening to noise-vocoded sentences. We first established if participants were able to perceive and adapt to audiovisual four-band noise-vocoded sentences when the entire moving face was visible (AV Full). Four further groups were then tested: a group in which participants viewed the moving lower part of the speaker's face (AV Mouth), a group in which participants only see the moving upper part of the face (AV Eyes), a group in which participants could not see the moving lower or upper face (AV Blocked), and a group in which participants saw an image of a still face (AV Still). Results Participants repeated around 40% of the key words correctly and adapted during the experiment, but only when the moving mouth was visible. In contrast, performance was at floor level, and no adaptation took place, in conditions when the moving mouth was occluded. Conclusions The results show the importance of being able to observe relevant visual speech information from the speaker's mouth region, but not the eyes/upper face region, when listening and adapting to distorted sentences online. Second, the results also demonstrated that it is feasible to run speech perception and adaptation studies online, but that not all findings reported for lab studies replicate. Supplemental Material https://doi.org/10.23641/asha.14810523

Facial speech processing in children with and without dyslexia

What role does the presence of facial speech play for children with dyslexia? Current literature proposes two distinctive claims. One claim states that children with dyslexia make less use of visual information from the mouth during speech processing due to a deficit in recruitment of audiovisual areas. An opposing claim suggests that children with dyslexia are in fact reliant on such information in order to compensate for auditory/phonological impairments. The current paper aims at directly testing these contrasting hypotheses (here referred to as “mouth insensitivity” versus “mouth reliance”) in school-age children with and without dyslexia, matched on age and listening comprehension. Using eye tracking, in Study 1, we examined how children look at the mouth across conditions varying in speech processing demands. The results did not indicate significant group differences in looking at the mouth. However, correlation analyses suggest potentially important distinctions within the dyslexia group: those children with dyslexia who are better readers attended more to the mouth while presented with a person’s face in a phonologically demanding condition. In Study 2, we examined whether the presence of facial speech cues is functionally beneficial when a child is encoding written words. The results indicated lack of overall group differences on the task, although those with less severe reading problems in the dyslexia group were more accurate when reading words that were presented with articulatory facial speech cues. Collectively, our results suggest that children with dyslexia differ in their “mouth reliance” versus “mouth insensitivity,” a profile that seems to be related to the severity of their reading problems.

Individuals With Mild Cognitive Impairment and Alzheimer's Disease Benefit From Audiovisual Speech Cues and Supportive Sentence Context

Purpose Speech perception in noise becomes difficult with age but can be facilitated by audiovisual (AV) speech cues and sentence context in healthy older adults. However, individuals with Alzheimer's disease (AD) may present with deficits in AV integration, potentially limiting the extent to which they can benefit from AV cues. This study investigated the benefit of these cues in individuals with mild cognitive impairment (MCI), individuals with AD, and healthy older adult controls. Method This study compared auditory-only and AV speech perception of sentences presented in noise. These sentences had one of two levels of context: high (e.g., “Stir your coffee with a spoon”) and low (e.g., “Bob didn't think about the spoon”). Fourteen older controls ( M age = 72.71 years, SD = 9.39), 13 individuals with MCI ( M age = 79.92 years, SD = 5.52), and nine individuals with probable Alzheimer's-type dementia ( M age = 79.38 years, SD = 3.40) completed the speech perception task and were asked to repeat the terminal word of each sentence. Results All three groups benefited (i.e., identified more terminal words) from AV and sentence context. Individuals with MCI showed a smaller AV benefit compared to controls in low-context conditions, suggesting difficulties with AV integration. Individuals with AD showed a smaller benefit in high-context conditions compared to controls, indicating difficulties with AV integration and context use in AD. Conclusions Individuals with MCI and individuals with AD do benefit from AV speech and semantic context during speech perception in noise (albeit to a lower extent than healthy older adults). This suggests that engaging in face-to-face communication and providing ample context will likely foster more effective communication between patients and caregivers, professionals, and loved ones.

Visually-biased perception in cochlear implant users: a study of the McGurk and sound-induced flash illusions

The reduction in spectral resolution by cochlear implants oftentimes requires complementary visual speech cues to aid in understanding. Despite substantial clinical characterization of auditory-only speech outcome measures, relatively little is known about the audiovisual integrative abilities that most cochlear implant (CI) users rely on for daily speech comprehension. In this study, we tested audiovisual integration in 63 CI users and 69 normal-hearing (NH) controls using the McGurk and sound-induced flash illusions. This study is the largest to-date measuring the McGurk effect in this population and the first to test the sound-induced flash illusion. When presented with conflicting audiovisual speech stimuli (i.e., the phoneme "ba" dubbed onto the viseme "ga"), we found that 55 CI users (87%) reported a fused percept of "da" or "tha" on at least one trial. However, overall, we found that CI users experienced the McGurk effect less often than controls--a result that was concordant with results with the sound-induced flash illusion where the pairing of a single circle flashing on the screen with multiple beeps resulted in fewer illusory flashes for CI users. While illusion perception in these two tasks appears to be uncorrelated among CI users, we identified a negative correlation in the NH group. Because neither illusion appears to provide further explanation of variability in CI outcome measures, further research is needed to determine how these findings relate to CI users' speech understanding, particularly in ecological listening conditions that are naturally multisensory.