AbstractEmotion perception from facial and vocal expressions is a multisensory process critical for human social interaction. When asked to judge emotions by attending to either face or voice, the accuracy was higher when facial expressions are congruent with vocal expressions than when they are incongruent. This congruency effect was shown to be affected by cultural background. Here we conducted functional MRI alongside a multisensory emotion perception task involving Japanese and Dutch participants. They were presented with movies in which congruent or incongruent emotions were expressed through faces and voices. The participants were asked to judge the emotion of either the face or the voice. Consistent with previous studies, behavioral results showed an interaction between group and task. fMRI results revealed that during voice-based judgement, congruency effects of the primary visual cortex, by means of the task related activity of congruent stimuli subtracted by that of incongruent stimuli, were more prominent in the Dutch group than in Japanese group. Finally, behavioral and neural congruency effects of the primary visual cortex were positively correlated only in the Dutch group. Taken together, our results show that culture affects the activities of early sensory areas in multisensory perception from facial and vocal expressions.
Transformation of perceptual activity into saccade-related activity in the monkey primary visual cortex.
