Vocal emotion adaptation aftereffects within and across speaker genders: Roles of timbre and fundamental frequency

Although previous research demonstrated perceptual aftereffects in emotional voice adaptation, the contribution of different vocal cues to these effects is unclear. In two experiments, we used parameter-specific morphing of adaptor voices to investigate the relative roles of fundamental frequency (F0) and timbre in vocal emotion adaptation, using angry and fearful utterances. Participants adapted to voices containing emotion-specific information in either F0 or timbre, with all other parameters kept constant at an intermediate 50% morph level. Full emotional adaptors and ambiguous adaptors were used as reference conditions. Adaptors were either of the same (Experiment 1) or opposite speaker gender (Experiment 2) of target voices. In Experiment 1, we found consistent aftereffects in all adaptation conditions. Crucially, aftereffects following timbre adaptors were much larger than following F0 adaptors and were only marginally smaller than those following full adaptors. In Experiment 2, adaptation aftereffects appeared massively and proportionally reduced, with differences between morph types being no longer significant. These results suggest that timbre plays a larger role than F0 in vocal emotion adaptation, and that vocal emotion adaptation is compromised by eliminating gender-congruency between adaptors and targets. Our findings also add to mounting evidence suggesting a major role of timbre in auditory adaptation.

Listeners’ perceptions of the certainty and honesty of a speaker are associated with a common prosodic signature

The success of human cooperation crucially depends on mechanisms enabling individuals to detect unreliability in their conspecifics. Yet, how such epistemic vigilance is achieved from naturalistic sensory inputs remains unclear. Here we show that listeners’ perceptions of the certainty and honesty of other speakers from their speech are based on a common prosodic signature. Using a data-driven method, we separately decode the prosodic features driving listeners’ perceptions of a speaker’s certainty and honesty across pitch, duration and loudness. We find that these two kinds of judgments rely on a common prosodic signature that is perceived independently from individuals’ conceptual knowledge and native language. Finally, we show that listeners extract this prosodic signature automatically, and that this impacts the way they memorize spoken words. These findings shed light on a unique auditory adaptation that enables human listeners to quickly detect and react to unreliability during linguistic interactions.

Speech-specific perceptual adaptation deficits in children and adults with dyslexia

According to several influential theoretical frameworks, phonological deficits in dyslexia result from reduced sensitivity to acoustic cues that are essential for the development of robust phonemic representations. Some accounts suggest that these deficits arise from impairments in rapid auditory adaptation processes that are either speech-specific or domain-general. Here, we examined the specificity of auditory adaptation deficits in dyslexia using a non-linguistic tone anchoring (adaptation) task and a linguistic selective adaptation task in children and adults with and without dyslexia. Children and adults with dyslexia had elevated tone-frequency discrimination thresholds, but both groups benefitted from anchoring to repeated stimuli to the same extent as typical readers. Additionally, although both dyslexia groups had overall reduced accuracy for speech sound identification, only the child group had reduced categorical perception for speech. Across both age groups, individuals with dyslexia had reduced perceptual adaptation to speech. These results highlight broad auditory perceptual deficits across development in individuals with dyslexia for both linguistic and non-linguistic domains, but speech-specific adaptation deficits. Finally, mediation models in children and adults revealed that the causal pathways from basic perception and adaptation to phonological awareness through speech categorization were not significant. Thus, rather than having causal effects, perceptual deficits may co-occur with the phonological deficits in dyslexia across development.

Investigating Effects of Visual and Auditory Adaptation on Metallic Material Appearance

In this paper, we investigated the effects of visual and auditory adaptation on material appearance. The target in this study was metallic perception. First, participants evaluated CG images using sounds and other images. In the experiment, we prepared metallic stimulus under various adaptation conditions with different combinations of metal image, non-metal image, metal sound, and non-metal sound stimuli. After these adaptations, the participants answered "metal" or "non-metal" after viewing a displayed reference image. The reference images were generated by interpolating metal and non-metal images. Next, we analyzed the results and clarified the effects of visual, auditory, and audiovisual adaptations on the metallic perception. For analyzing results, we used a logistic regression analysis based on Bayesian statistics. From the analysis results, we found visual and auditory adaptation effects. On the other hand, we did not find the cross-modal effects of audiovisual adaptation. Finally, we created a model of the linear sum of the visual and audio adaptation effects on metallic material appearance.

Auditory adaptation improves tactile frequency perception

Our ability to process temporal frequency information by touch underlies our capacity to perceive and discriminate surface textures. Auditory signals, which also provide extensive temporal frequency information, can systematically alter the perception of vibrations on the hand. How auditory signals shape tactile processing is unclear; perceptual interactions between contemporaneous sounds and vibrations are consistent with multiple neural mechanisms. Here we used a crossmodal adaptation paradigm, which separated auditory and tactile stimulation in time, to test the hypothesis that tactile frequency perception depends on neural circuits that also process auditory frequency. We reasoned that auditory adaptation effects would transfer to touch only if signals from both senses converge on common representations. We found that auditory adaptation can improve tactile frequency discrimination thresholds. This occurred only when adaptor and test frequencies overlapped. In contrast, auditory adaptation did not influence tactile intensity judgments. Thus auditory adaptation enhances touch in a frequency- and feature-specific manner. A simple network model in which tactile frequency information is decoded from sensory neurons that are susceptible to auditory adaptation recapitulates these behavioral results. Our results imply that the neural circuits supporting tactile frequency perception also process auditory signals. This finding is consistent with the notion of supramodal operators performing canonical operations, like temporal frequency processing, regardless of input modality. NEW & NOTEWORTHY Auditory signals can influence the tactile perception of temporal frequency. Multiple neural mechanisms could account for the perceptual interactions between contemporaneous auditory and tactile signals. Using a crossmodal adaptation paradigm, we found that auditory adaptation causes frequency- and feature-specific improvements in tactile perception. This crossmodal transfer of aftereffects between audition and touch implies that tactile frequency perception relies on neural circuits that also process auditory frequency.