CNN explains tuning properties of anterior, but not middle, face-processing areas in macaque IT

Mapping Intimacies ◽

10.1101/686121 ◽

2019 ◽

Author(s):

Rajani Raman ◽

Haruo Hosoya

Keyword(s):

Neural Networks ◽

Face Processing ◽

Processing System ◽

Computational Studies ◽

Comprehensive Model ◽

Entire System ◽

Face Area ◽

The Face ◽

Goal Representation ◽

Model Layer

AbstractRecent computational studies have emphasized layer-wise quantitative similarity between convolutional neural networks (CNNs) and the primate visual ventral stream. However, whether such similarity holds for the face-selective areas, a subsystem of the higher visual cortex, is not clear. Here, we extensively investigate whether CNNs exhibit tuning properties as previously observed in different macaque face areas. While simulating four past experiments on a variety of CNN models, we sought for the model layer that quantitatively matches the multiple tuning properties of each face area. Our results show that higher model layers explain reasonably well the properties of anterior areas, while no layer simultaneously explains the properties of middle areas, consistently across the model variation. Thus, some similarity may exist between CNNs and the primate face-processing system in the near-goal representation, but much less clearly in the intermediate stages, thus giving motivation for a more comprehensive model for understanding the entire system.

Download Full-text

Does Prosopagnosia Take the Eyes Out of Face Representations? Evidence for a Defect in Representing Diagnostic Facial Information following Brain Damage

Journal of Cognitive Neuroscience ◽

10.1162/089892905774597254 ◽

2005 ◽

Vol 17 (10) ◽

pp. 1652-1666 ◽

Cited By ~ 117

Author(s):

Roberto Caldara ◽

Philippe Schyns ◽

Eugéne Mayer ◽

Marie L. Smith ◽

Frédéric Gosselin ◽

...

Keyword(s):

Brain Damage ◽

Face Processing ◽

Processing System ◽

Facial Information ◽

Spatial Frequencies ◽

The Face ◽

Occipitotemporal Cortex ◽

Response Classification ◽

Unfamiliar Faces ◽

Normal Controls

One of the most impressive disorders following brain damage to the ventral occipitotemporal cortex is prosopagnosia, or the inability to recognize faces. Although acquired prosopagnosia with preserved general visual and memory functions is rare, several cases have been described in the neuropsychological literature and studied at the functional and neural level over the last decades. Here we tested a brain-damaged patient (PS) presenting a deficit restricted to the category of faces to clarify the nature of the missing and preserved components of the face processing system when it is selectively damaged. Following learning to identify 10 neutral and happy faces through extensive training, we investigated patient PS's recognition of faces using Bubbles, a response classification technique that sampled facial information across the faces in different bandwidths of spatial frequencies [Gosselin, F., & Schyns, P. E., Bubbles: A technique to reveal the use of information in recognition tasks. Vision Research, 41, 2261-2271, 2001]. Although PS gradually used less information (i.e., the number of bubbles) to identify faces over testing, the total information required was much larger than for normal controls and decreased less steeply with practice. Most importantly, the facial information used to identify individual faces differed between PS and controls. Specifically, in marked contrast to controls, PS did not use the optimal eye information to identify familiar faces, but instead the lower part of the face, including the mouth and the external contours, as normal observers typically do when processing unfamiliar faces. Together, the findings reported here suggest that damage to the face processing system is characterized by an inability to use the information that is optimal to judge identity, focusing instead on suboptimal information.

Download Full-text

Face Detection in Natural Scenes

Perception ◽

10.1068/v96l1003 ◽

1996 ◽

Vol 25 (1_suppl) ◽

pp. 67-67 ◽

Cited By ~ 1

Author(s):

H Hill ◽

R Watt

Keyword(s):

Face Detection ◽

Face Processing ◽

Feature Detection ◽

Processing System ◽

Decision Task ◽

Natural Scenes ◽

Natural Scene ◽

Energy Models ◽

The Face ◽

And Performance

The first task of any face processing system is detection of the face. We studied how the human visual system achieves face detection using a 2AFC task in which subjects were required to detect a face in the image of a natural scene. Luminance noise was added to the stimuli and performance was measured as a function of orientation and orientation bandwidth of the noise. Sensitivity levels and the effects of orientation bandwidth were similar for horizontally and vertically oriented noise. Performance was reduced for the smallest orientation bandwidth (5.6°) noise but sensitivity did not decrease further with increasing bandwidth until a point between 45° and 90°. The results suggest that important information may be oriented close to the vertical and horizontal. To test whether the results were specific to the task of face detection the same noise was added to the images in a man-made natural decision task. Equivalent levels of noise were found to be more disruptive and the effect of orientation bandwidth was different. The results are discussed in terms of models of face processing making use of oriented filters (eg Watt and Dakin, 1993 Perception22 Supplement, 12) and local energy models of feature detection (Morrone and Burr, 1988 Proceedings of the Royal Society of London B235 221 – 245).

Download Full-text

Attentional Weighting in the Face Processing Network: A Magnetic Response Image-guided Magnetoencephalography Study Using Multiple Cyclic Entrainments

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_01428 ◽

2019 ◽

Vol 31 (10) ◽

pp. 1573-1588 ◽

Cited By ~ 2

Author(s):

Eelke de Vries ◽

Daniel Baldauf

Keyword(s):

Face Processing ◽

Visual Processing ◽

Magnetic Response ◽

Fusiform Face Area ◽

Facial Identity ◽

Image Guided ◽

Face Area ◽

The Face ◽

Occipital Face Area ◽

Processing Network

We recorded magnetoencephalography using a neural entrainment paradigm with compound face stimuli that allowed for entraining the processing of various parts of a face (eyes, mouth) as well as changes in facial identity. Our magnetic response image-guided magnetoencephalography analyses revealed that different subnodes of the human face processing network were entrained differentially according to their functional specialization. Whereas the occipital face area was most responsive to the rate at which face parts (e.g., the mouth) changed, and face patches in the STS were mostly entrained by rhythmic changes in the eye region, the fusiform face area was the only subregion that was strongly entrained by the rhythmic changes in facial identity. Furthermore, top–down attention to the mouth, eyes, or identity of the face selectively modulated the neural processing in the respective area (i.e., occipital face area, STS, or fusiform face area), resembling behavioral cue validity effects observed in the participants' RT and detection rate data. Our results show the attentional weighting of the visual processing of different aspects and dimensions of a single face object, at various stages of the involved visual processing hierarchy.

Download Full-text

Multi-Item Discriminability Pattern to Faces in Developmental Prosopagnosia Reveals Distinct Mechanisms of Face Processing

Cerebral Cortex ◽

10.1093/cercor/bhz289 ◽

2019 ◽

Vol 30 (5) ◽

pp. 2986-2996

Author(s):

Xue Tian ◽

Ruosi Wang ◽

Yuanfang Zhao ◽

Zonglei Zhen ◽

Yiying Song ◽

...

Keyword(s):

Face Processing ◽

Alternative Hypothesis ◽

Normal Population ◽

Imaging Study ◽

Multiple Sources ◽

Neural Basis ◽

Face Area ◽

Normal Individuals ◽

The Face ◽

The Right

Abstract Previous studies have shown that individuals with developmental prosopagnosia (DP) show specific deficits in face processing. However, the mechanism underlying the deficits remains largely unknown. One hypothesis suggests that DP shares the same mechanism as normal population, though their faces processing is disproportionally impaired. An alternative hypothesis emphasizes a qualitatively different mechanism of DP processing faces. To test these hypotheses, we instructed DP and normal individuals to perceive faces and objects. Instead of calculating accuracy averaging across stimulus items, we used the discrimination accuracy for each item to construct a multi-item discriminability pattern. We found DP’s discriminability pattern was less similar to that of normal individuals when perceiving faces than perceiving objects, suggesting that DP has qualitatively different mechanism in representing faces. A functional magnetic resonance imaging study was conducted to reveal the neural basis and found that multi-voxel activation patterns for faces in the right fusiform face area and occipital face area of DP were deviated away from the mean activation pattern of normal individuals. Further, the face representation was more heterogeneous in DP, suggesting that deficits of DP may come from multiple sources. In short, our study provides the first direct evidence that DP processes faces qualitatively different from normal population.

Download Full-text

Representations in learning new faces: Evidence from prosopagnosia

Journal of the International Neuropsychological Society ◽

10.1017/s1355617700001181 ◽

1996 ◽

Vol 2 (3) ◽

pp. 240-248 ◽

Cited By ~ 17

Author(s):

Michael R. Polster ◽

Steven Z. Rapcsak

Keyword(s):

Left Hemisphere ◽

Face Processing ◽

Processing System ◽

Poor Performance ◽

Facial Features ◽

Study Phase ◽

Learning Tasks ◽

The Face ◽

Feature Based ◽

Pictorial Representations

AbstractWe report the performance of a prosopagnosic patient on face learning tasks under different encoding instructions (i.e., levels of processing manipulations). R.J. performs at chance when given no encoding instructions or when given “shallow” encoding instructions to focus on facial features. By contrast, he performs relatively well with “deep” encoding instructions to rate faces in terms of personality traits or when provided with semantic and name information during the study phase. We propose that the improvement associated with deep encoding instructions may be related to the establishment of distinct visually derived and identity-specific semantic codes. The benefit associated with deep encoding in R.J., however, was found to be restricted to the specific view of the face presented at study and did not generalize to other views of the same face. These observations suggest that deep encoding instructions may enhance memory for concrete or pictorial representations of faces in patients with prosopagnosia, but that these patients cannot compensate for the inability to construct abstract structural codes that normally allow faces to be recognized from different orientations. We postulate further that R.J.'s poor performance on face learning tasks may be attributable to excessive reliance on a feature-based left hemisphere face processing system that operates primarily on view-specific representations. (JINS, 1996, 2, 240–248.)

Download Full-text

Response to `What can bees really tell us about the face processing system in humans?'

Journal of Experimental Biology ◽

10.1242/jeb.02412 ◽

2006 ◽

Vol 209 (16) ◽

pp. 3267-3267 ◽

Cited By ~ 2

Author(s):

A. G. Dyer

Keyword(s):

Face Processing ◽

Processing System ◽

The Face

Download Full-text

Fuzzy-Face

Journal of Biological System ◽

10.1142/s0218339098000194 ◽

1998 ◽

Vol 06 (03) ◽

pp. 281-298 ◽

Cited By ~ 5

Author(s):

Terry Huntsberger ◽

John Rose ◽

Shashidhar Ramaka

Keyword(s):

Face Processing ◽

Gender Discrimination ◽

Computational Models ◽

Experimental Studies ◽

Processing System ◽

Facial Hair ◽

Lighting Conditions ◽

Face Images ◽

The Face ◽

Strong Motivation

The human face is one of the most important patterns our visual system receives. It establishes a person's identity and also plays a significant role in everyday communication. Humans can recognize familiar faces under varying lighting conditions, different scales, and even after the face has changed due to aging, hair style, glasses, or facial hair. Our ease at recognizing faces is a strong motivation for the investigation of computational models of face processing. This paper presents a newly developed face processing system called Fuzzy-Face that combines wavelet pre-processing of input with a fuzzy self-organizing feature map algorithm. The wavelet-derived face space is partitioned into fuzzy sets which are characterized by face exemplars and membership values to those exemplars. This system learns faces using relatively few training epochs, has total recall for faces it has been shown, generalizes to face images that are acquired under different lighting conditions, and has rudimentary gender discrimination capabilities. We also include the results of some experimental studies.

Download Full-text

Not so fast: Limited validity of deep convolutional neural networks as in silico models for human naturalistic face processing

10.1101/2021.11.17.469009 ◽

2021 ◽

Author(s):

Guo Jiahui ◽

Ma Feilong ◽

Matteo Visconti di Oleggio Castello ◽

Samuel A Nastase ◽

James V Haxby ◽

...

Keyword(s):

Neural Networks ◽

Human Brain ◽

Convolutional Neural Networks ◽

Face Processing ◽

In Silico ◽

Processing System ◽

Human Face ◽

Deep Convolutional Neural Networks ◽

Neural Representations ◽

Fully Connected

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The relationships between internal representations learned by DCNNs and those of the primate face processing system are not well understood, especially in naturalistic settings. We developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces) and used representational similarity analysis to investigate how well the representations learned by high-performing DCNNs match human brain representations across the entire distributed face processing system. DCNN representational geometries were strikingly consistent across diverse architectures and captured meaningful variance among faces. Similarly, representational geometries throughout the human face network were highly consistent across subjects. Nonetheless, correlations between DCNN and neural representations were very weak overall—DCNNs captured 3% of variance in the neural representational geometries at best. Intermediate DCNN layers better matched visual and face-selective cortices than the final fully-connected layers. Behavioral ratings of face similarity were highly correlated with intermediate layers of DCNNs, but also failed to capture representational geometry in the human brain. Our results suggest that the correspondence between intermediate DCNN layers and neural representations of naturalistic human face processing is weak at best, and diverges even further in the later fully-connected layers. This poor correspondence can be attributed, at least in part, to the dynamic and cognitive information that plays an essential role in human face processing but is not modeled by DCNNs. These mismatches indicate that current DCNNs have limited validity as in silico models of dynamic, naturalistic face processing in humans.

Download Full-text

Sensitivity to Faces with Typical and Atypical Part Configurations within Regions of the Face-processing Network: An fMRI Study

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_01255 ◽

2018 ◽

Vol 30 (7) ◽

pp. 963-972 ◽

Cited By ~ 3

Author(s):

Andrew D. Engell ◽

Na Yeon Kim ◽

Gregory McCarthy

Keyword(s):

Face Processing ◽

Brain Regions ◽

Domain Specific ◽

Face Area ◽

Fmri Study ◽

The Face ◽

Occipital Face Area ◽

Typical Configuration ◽

The Brain ◽

Processing Network

Perception of faces has been shown to engage a domain-specific set of brain regions, including the occipital face area (OFA) and the fusiform face area (FFA). It is commonly held that the OFA is responsible for the detection of faces in the environment, whereas the FFA is responsible for processing the identity of the face. However, an alternative model posits that the FFA is responsible for face detection and subsequently recruits the OFA to analyze the face parts in the service of identification. An essential prediction of the former model is that the OFA is not sensitive to the arrangement of internal face parts. In the current fMRI study, we test the sensitivity of the OFA and FFA to the configuration of face parts. Participants were shown faces in which the internal parts were presented in a typical configuration (two eyes above a nose above a mouth) or in an atypical configuration (the locations of individual parts were shuffled within the face outline). Perception of the atypical faces evoked a significantly larger response than typical faces in the OFA and in a wide swath of the surrounding posterior occipitotemporal cortices. Surprisingly, typical faces did not evoke a significantly larger response than atypical faces anywhere in the brain, including the FFA (although some subthreshold differences were observed). We propose that face processing in the FFA results in inhibitory sculpting of activation in the OFA, which accounts for this region's weaker response to typical than to atypical configurations.

Download Full-text