Facial expression processing in developmental prosopagnosia

<p>Individuals with developmental prosopagnosia experience lifelong deficits recognising facial identity, but whether their ability to process facial expression is also impaired is unclear. Addressing this issue is key for understanding the core deficit in developmental prosopagnosia, and for advancing knowledge about the mechanisms and development of normal face processing. In this thesis, I report two online studies on facial expression processing with large samples of prosopagnosics. In Study 1, I compared facial expression and facial identity perception in 124 prosopagnosics and 133 controls. I used three perceptual tasks including simultaneous matching, sequential matching, and sorting. I also measured inversion effects to examine whether prosopagnosics rely on typical face mechanisms. Prosopagnosics showed subtle deficits with facial expression, but they performed worse with facial identity. Prosopagnosics also showed reduced inversion effects for facial identity but normal inversion effects for facial expression, suggesting they use atypical mechanisms for facial identity but normal mechanisms for facial expression. In Study 2, I extended the findings of Study 1 by assessing facial expression recognition in 78 prosopagnosics and 138 controls. I used four labelling tasks that varied on whether the facial expressions were basic (e.g., happy) or complex (e.g., elated), and whether they were displayed via static (i.e., images) or dynamic (i.e., video clips) stimuli. Prosopagnosics showed subtle deficits with basic expressions but performed normally with complex expressions. Further, prosopagnosics did not show reduced inversion effects for both types of expressions, suggesting they use similar recognition mechanisms as controls. Critically, the subtle expression deficits that prosopagnosics showed in both studies can be accounted for by autism traits, suggesting that expression deficits are not a feature of prosopagnosia per se. I also provide estimates of the prevalence of deficits in facial expression perception (7.70%) and recognition (2.56% - 5.13%) in prosopagnosia, both of which suggest that facial expression processing is normal in the majority of prosopagnosics. Overall, my thesis demonstrates that facial expression processing is not impaired in developmental prosopagnosia, and suggests that facial expression and facial identity processing rely on separate mechanisms that dissociate in development.</p>

Facial expression processing in developmental prosopagnosia

<p>Individuals with developmental prosopagnosia experience lifelong deficits recognising facial identity, but whether their ability to process facial expression is also impaired is unclear. Addressing this issue is key for understanding the core deficit in developmental prosopagnosia, and for advancing knowledge about the mechanisms and development of normal face processing. In this thesis, I report two online studies on facial expression processing with large samples of prosopagnosics. In Study 1, I compared facial expression and facial identity perception in 124 prosopagnosics and 133 controls. I used three perceptual tasks including simultaneous matching, sequential matching, and sorting. I also measured inversion effects to examine whether prosopagnosics rely on typical face mechanisms. Prosopagnosics showed subtle deficits with facial expression, but they performed worse with facial identity. Prosopagnosics also showed reduced inversion effects for facial identity but normal inversion effects for facial expression, suggesting they use atypical mechanisms for facial identity but normal mechanisms for facial expression. In Study 2, I extended the findings of Study 1 by assessing facial expression recognition in 78 prosopagnosics and 138 controls. I used four labelling tasks that varied on whether the facial expressions were basic (e.g., happy) or complex (e.g., elated), and whether they were displayed via static (i.e., images) or dynamic (i.e., video clips) stimuli. Prosopagnosics showed subtle deficits with basic expressions but performed normally with complex expressions. Further, prosopagnosics did not show reduced inversion effects for both types of expressions, suggesting they use similar recognition mechanisms as controls. Critically, the subtle expression deficits that prosopagnosics showed in both studies can be accounted for by autism traits, suggesting that expression deficits are not a feature of prosopagnosia per se. I also provide estimates of the prevalence of deficits in facial expression perception (7.70%) and recognition (2.56% - 5.13%) in prosopagnosia, both of which suggest that facial expression processing is normal in the majority of prosopagnosics. Overall, my thesis demonstrates that facial expression processing is not impaired in developmental prosopagnosia, and suggests that facial expression and facial identity processing rely on separate mechanisms that dissociate in development.</p>

A Python toolbox for Automatic Face Alignment (AFA)

We describe AFA, an open-source Python package for automating the most common step in the preparation of facial stimuli for behavioral and neuro-imaging experiments – spatial alignment of faces (https://github.com/SourCherries/auto-face-align ). Face alignment is also important in the analysis of image statistics, and as a preprocessing step for machine learning. Automation of face alignment via AFA provides a reliable and efficient alternative to the very common practice of manual image-editing in graphics editors like Photoshop. As an open-source Python package, AFA encourages a clear and transparent specification of experimental method. AFA is based on facial landmark detection that is powered by the reliable and open-source DLIB library; and critical alignment code based on Generalized Procrustes Analysis (GPA) has been extensively unit-tested. AFA documentation and modularity provides opportunity for the modification and extensibility of AFA by the scientific community. As examples, we include functions for automatically generating image apertures that conceal areas outside the inner face; for image morphing between facial identities; and for shape-based averaging of facial identity. All of these are examples of stimulus preparation that have previously required manual landmark selection.

EXPRESS: Serial Dependence of Facial Identity for Own- and Other-Race Faces

It is well-established that individuals are better at recognising faces of their own-race compared to other-races, however there is ongoing debate regarding the perceptual mechanisms that may be involved and therefore sensitive to face-race. Here we ask whether serial dependence of facial identity, a bias where the perception of a face’s identity is biased towards a previously presented face, shows an other-race effect. Serial dependence is associated with face recognition ability and appears to operate on high-level, face-selective representations, like other candidate mechanisms (e.g., holistic coding). We therefore expected to find an other-race effect for serial dependence for our Caucasian and Asian participants. While participants showed robust effects of serial dependence for all faces, only Caucasian participants showed stronger serial dependence for own-race faces. Intriguingly, we found that individual variation in own-race, but not other-race, serial dependence was significantly associated with face recognition abilities. Preliminary evidence also suggested that other-race contact is associated with other-race serial dependence. In conclusion, though we did not find an overall difference in serial dependence for own versus other-race faces in both participant groups, our results highlight that this bias may be functionally different for own versus other-race faces and sensitive to racial experience.

Neural Responses to Facial Attractiveness in the Judgments of Moral Goodness and Moral Beauty

The judgments of moral goodness and moral beauty objectively refer to the perception and evaluation of moral traits, which are generally influenced by facial attractiveness. For centuries, people have equated beauty with the possession of positive qualities, but it is not clear whether the association between beauty and positive qualities exerts a similarly implicit influence on people`s responses to moral goodness and moral beauty, how it affects those responses, and what is the neural basis for such an effect. The present study is the first to examine the neural responses to facial attractiveness in the judgments of moral goodness and moral beauty. We found that beautiful faces in both moral judgments activated the left ventral occipitotemporal cortices sensitive to the geometric configuration of the faces, demonstrating that both moral goodness and moral beauty required the automatic visual analysis of geometrical configuration of attractive faces. In addition, compared to beautiful faces during moral goodness judgment, beautiful faces during moral beauty judgment induced unique activity in the ventral medial prefrontal cortex and midline cortical structures involved in the emotional-valenced information about attractive faces. The opposite comparison elicited specific activity in the left superior temporal cortex and premotor area, which play a critical role in the recognition of facial identity. Our results demonstrated that the neural responses to facial attractiveness in the process of higher order moral decision-makings exhibits both task-general and task-specific characteristics. Our findings contribute to the understanding of the essence of the relationship between morality and aesthetics.

Own-race faces promote integrated audiovisual speech information

The other-race effect indicates a perceptual advantage when processing own-race faces. This effect has been demonstrated in individuals’ recognition of facial identity and emotional expressions. However, it remains unclear whether the other-race effect also exists in multisensory domains. We conducted two experiments to provide evidence for the other-race effect in facial speech recognition, using the McGurk effect. Experiment 1 tested this issue among East Asian adults, examining the magnitude of the McGurk effect during stimuli using speakers from two different races (own-race vs. other-race). We found that own-race faces induced a stronger McGurk effect than other-race faces. Experiment 2 indicated that the other-race effect was not simply due to different levels of attention being paid to the mouths of own- and other-race speakers. Our findings demonstrated that own-race faces enhance the weight of visual input during audiovisual speech perception, and they provide evidence of the own-race effect in the audiovisual interaction for speech perception in adults.

Joint encoding of facial identity, orientation, gaze, and expression in the middle dorsal face area

The last two decades have established that a network of face-selective areas in the temporal lobe of macaque monkeys supports the visual processing of faces. Each area within the network contains a large fraction of face-selective cells. And each area encodes facial identity and head orientation differently. A recent brain-imaging study discovered an area outside of this network selective for naturalistic facial motion, the middle dorsal (MD) face area. This finding offers the opportunity to determine whether coding principles revealed inside the core network would generalize to face areas outside the core network. We investigated the encoding of static faces and objects, facial identity, and head orientation, dimensions which had been studied in multiple areas of the core face-processing network before, as well as facial expressions and gaze. We found that MD populations form a face-selective cluster with a degree of selectivity comparable to that of areas in the core face-processing network. MD encodes facial identity robustly across changes in head orientation and expression, it encodes head orientation robustly against changes in identity and expression, and it encodes expression robustly across changes in identity and head orientation. These three dimensions are encoded in a separable manner. Furthermore, MD also encodes the direction of gaze in addition to head orientation. Thus, MD encodes both structural properties (identity) and changeable ones (expression and gaze) and thus provides information about another animal’s direction of attention (head orientation and gaze). MD contains a heterogeneous population of cells that establish a multidimensional code for faces.

Dynamic face processing impairments are associated with cognitive and positive psychotic symptoms across psychiatric disorders

Impairments in social cognition—including recognition of facial expressions—are increasingly recognised as a core deficit in schizophrenia. It remains unclear whether other aspects of face processing (such as identity recognition) are also impaired, and whether such deficits can be attributed to more general cognitive difficulties. Moreover, while the majority of past studies have used picture-based tasks to assess face recognition, literature suggests that video-based tasks elicit different neural activations and have greater ecological validity. This study aimed to characterise face processing using video-based stimuli in psychiatric inpatients with and without psychosis. Symptom correlates of face processing impairments were also examined. Eighty-six psychiatric inpatients and twenty healthy controls completed a series of tasks using video-based stimuli. These included two emotion recognition tasks, two non-emotional facial identity recognition tasks, and a non-face control task. Symptoms were assessed using the Positive and Negative Syndrome Scale. Schizophrenia and bipolar disorder groups were significantly impaired on the emotion-processing tasks and the non-face task compared to healthy controls and patients without psychosis. Patients with other forms of psychosis performed intermediately. Groups did not differ in non-emotional face processing. Positive symptoms of psychosis correlated directly with both emotion-processing performance and non-face discrimination across patients. We found that identity processing performance was inversely associated with cognition-related symptoms only. Findings suggest that deficits in emotion-processing reflect symptom pathology independent of diagnosis. Emotion-processing deficits in schizophrenia may be better accounted for by task-relevant factors—such as attention—that are not specific to emotion processing.