Explaining face-voice matching decisions: The contribution of mouth movements, stimulus effects and response biases

Previous studies have shown that face-voice matching accuracy is more consistently above chance for dynamic (i.e. speaking) faces than for static faces. This suggests that dynamic information can play an important role in informing matching decisions. We initially asked whether this advantage for dynamic stimuli is due to shared information across modalities that is encoded in articulatory mouth movements. Participants completed a sequential face-voice matching task with (1) static images of faces, (2) dynamic videos of faces, (3) dynamic videos where only the mouth was visible, and (4) dynamic videos where the mouth was occluded, in a well-controlled stimulus set. Surprisingly, after accounting for random variation in the data due to design choices, accuracy for all four conditions was at chance. Crucially, however, exploratory analyses revealed that participants were not responding randomly, with different patterns of response biases being apparent for different conditions. Our findings suggest that face-voice identity matching may not be possible with above-chance accuracy but that analyses of response biases can shed light upon how people attempt face-voice matching. We discuss these findings with reference to the differential functional roles for faces and voices recently proposed for multimodal person perception.

Explaining face-voice matching decisions: The contribution of mouth movements, stimulus effects and response biases

Previous studies have shown that face-voice matching is more consistently above chance for dynamic (i.e. speaking) faces than for static faces. This suggests that dynamic information can play an important role in informing matching decisions. We initially asked whether this advantage for dynamic stimuli is due to shared information across modalities that is encoded in articulatory mouth movements. Participants completed a sequential face-voice matching task with (1) static images of faces, (2) dynamic videos of faces, (3) dynamic videos where only the mouth was visible, and (4) dynamic videos where the mouth was occluded, in a well-controlled stimulus set. Surprisingly, after accounting for random variation in the data due to design choices, accuracy for all 4 conditions was at chance. Crucially, however, exploratory analyses revealed that participants were not responding randomly, with different patterns of response biases being apparent for different conditions. Our findings suggest that face-voice identity matching may not be possible with above-chance accuracy but that analyses of response biases can shed light upon how people attempt face-voice matching. We discuss these findings with reference to the differential functional roles for faces and voices recently proposed for multimodal person perception.

Comparing unfamiliar voice and face identity perception using identity sorting tasks

Identity sorting tasks, in which participants sort multiple naturally varying stimuli of usually two identities into perceived identities, have recently gained popularity in voice and face processing research. In both modalities, participants who are unfamiliar with the identities tend to perceive multiple stimuli of the same identity as different people and thus fail to “tell people together.” These similarities across modalities suggest that modality-general mechanisms may underpin sorting behaviour. In this study, participants completed a voice sorting and a face sorting task. Taking an individual differences approach, we asked whether participants’ performance on voice and face sorting of unfamiliar identities is correlated. Participants additionally completed a voice discrimination (Bangor Voice Matching Test) and a face discrimination task (Glasgow Face Matching Test). Using these tasks, we tested whether performance on sorting related to explicit identity discrimination. Performance on voice sorting and face sorting tasks was correlated, suggesting that common modality-general processes underpin these tasks. However, no significant correlations were found between sorting and discrimination performance, with the exception of significant relationships for performance on “same identity” trials with “telling people together” for voices and faces. Overall, any reported relationships were however relatively weak, suggesting the presence of additional modality-specific and task-specific processes.

Are super-face-recognisers also super-voice-recognisers? Evidence from cross-modal identification tasks

Individual differences in face identification ability range from prosopagnosia to super-recognition. The current study examined whether face identification ability predicts voice identification ability (participants: N = 529). Superior-face-identifiers (exceptional at face memory and matching), superior-face-recognisers (exceptional at face memory only), superior-face-matchers (exceptional face matchers only), and controls completed the Bangor Voice Matching Test, Glasgow Voice Memory Test, and a Famous Voice Recognition Test. Meeting predictions, those possessing exceptional face memory and matching skills outperformed typical-range face groups at voice memory and voice matching respectively. Proportionally more super-face-identifiers also achieved our super-voice-recogniser criteria on two or more tests. Underlying cross-modality (voices vs. faces) and cross-task (memory vs. perception) mechanisms may therefore drive superior performances. Dissociations between Glasgow Voice Memory Test voice and bell recognition also suggest voice-specific effects to match those found with faces. These findings have applied implications for policing, particularly in cases when only suspect voice clips are available.

Comparing unfamiliar voice and face identity perception using identity sorting tasks

Identity sorting tasks, where participants sort a number of naturally varying stimuli of usually two identities into perceived identities, have recently gained popularity in voice and face processing research. For both modalities, striking similarities in the results of these sorting tasks are apparent: Participants who are unfamiliar with the identities usually struggle to accurately perceive identities from these variable stimuli. They tend to perceive multiple stimuli of the same identity as different people and thus fail to “tell people together”. These similarities in the reported results may suggest that modality-general mechanisms underpin the completion of sorting tasks. In the current study, participants completed a voice sorting and a face sorting task. Taking an individual differences approach, we therefore asked whether there is a relationship between participants’ performance on voice and face sorting of unfamiliar identities. Participants additionally completed a voice discrimination (Bangor Voice Matching Test) and face discrimination task (Glasgow Face Matching Test). Using these data, we furthermore tested whether performance on sorting tasks can be related to explicit identity discrimination tasks. Performance on voice sorting and face sorting tasks was correlated, suggesting that common modality-general processes underpin these tasks. However, these do not straightforwardly appear to be the same processes supporting identity discrimination: No significant correlations were found between sorting and discrimination performance, with the exception of significant relationships when correlating performance on same trials with “telling people together” for voices and faces. Overall, the reported relationships were relatively weak, suggesting the presence of additional modality-specific and task-specific processes.

Self-Assessment of Singing Accuracy

This chapter presents a series of three research studies conducted to determine how teachers can best employ self-assessment to help elementary school children improve their singing accuracy. The first study examines the accuracy of children’s self-assessment, the second details the effects of teacher feedback on self-assessment accuracy, and the third describes the effects of peer feedback on self-assessment accuracy. Implications are that informal self-assessment of singing skills, conferencing with peers regarding singing skills, and practicing singing and self-assessment are activities that may contribute to accurate self-assessment in using correct words, singing in time, and using singing voices, and to somewhat accurate self-assessment in ability to match pitch. Including teacher feedback and peer feedback individually has yielded mixed results in increasing singing self-assessment accuracy in the combined areas of using singing voice, matching pitch, and melodic contour—with peer feedback over a slightly long period of time being more effective.

Restructuring and control in Arabic

This paper examines verbal complex constructions in two varieties of Arabic, namely: Moroccan Arabic (MA) and Najdi Arabic (NA). We provide a novel analysis that departs from the two previous studies that discussed this phenomenon in Lebanese Arabic (LA): Hallman (2011), and Ouwaydah & Shlonsky (2016). We argue that verbal complex constructions are derived using a monoclausal restructuring strategy (Wurmbrand 1998, 2001) in which the matrix control verb of nisa/forget-type instantiates a monoclausal structure. This analysis is novel in various aspects. First, it departs from and argues against control-based theories and, consequently, a biclausal structure. Second, it provides an account for various properties, including: agreement facts, Cinque’s restrictions on adverb distribution, tense and aspect absence on subordinate verbs, and the unexpected occurrence of the VVSO word order. Third, it analyzes new data related to the requirement of voice matching.