Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex

AbstractDeep Convolutional Neural Networks (CNNs) are gaining traction as the benchmark model of visual object recognition, with performance now surpassing humans. While CNNs can accurately assign one image to potentially thousands of categories, network performance could be the result of layers that are tuned to represent the visual shape of objects, rather than object category, since both are often confounded in natural images. Using two stimulus sets that explicitly dissociate shape from category, we correlate these two types of information with each layer of multiple CNNs. We also compare CNN output with fMRI activation along the human visual ventral stream by correlating artificial with biological representations. We find that CNNs encode category information independently from shape, peaking at the final fully connected layer in all tested CNN architectures. Comparing CNNs with fMRI brain data, early visual cortex (V1) and early layers of CNNs encode shape information. Anterior ventral temporal cortex encodes category information, which correlates best with the final layer of CNNs. The interaction between shape and category that is found along the human visual ventral pathway is echoed in multiple deep networks. Our results suggest CNNs represent category information independently from shape, much like the human visual system.

Download Full-text

Integrating Flexible Normalization into Midlevel Representations of Deep Convolutional Neural Networks

Neural Computation ◽

10.1162/neco_a_01226 ◽

2019 ◽

Vol 31 (11) ◽

pp. 2138-2176 ◽

Cited By ~ 2

Author(s):

Luis Gonzalo Sánchez Giraldo ◽

Odelia Schwartz

Keyword(s):

Neural Networks ◽

Visual Cortex ◽

Convolutional Neural Networks ◽

Neural Responses ◽

Spatial Normalization ◽

Deep Convolutional Neural Networks ◽

Cortical Areas ◽

Classical Receptive Field ◽

Divisive Normalization ◽

Spatial Dependencies

Deep convolutional neural networks (CNNs) are becoming increasingly popular models to predict neural responses in visual cortex. However, contextual effects, which are prevalent in neural processing and in perception, are not explicitly handled by current CNNs, including those used for neural prediction. In primary visual cortex, neural responses are modulated by stimuli spatially surrounding the classical receptive field in rich ways. These effects have been modeled with divisive normalization approaches, including flexible models, where spatial normalization is recruited only to the degree that responses from center and surround locations are deemed statistically dependent. We propose a flexible normalization model applied to midlevel representations of deep CNNs as a tractable way to study contextual normalization mechanisms in midlevel cortical areas. This approach captures nontrivial spatial dependencies among midlevel features in CNNs, such as those present in textures and other visual stimuli, that arise from tiling high-order features geometrically. We expect that the proposed approach can make predictions about when spatial normalization might be recruited in midlevel cortical areas. We also expect this approach to be useful as part of the CNN tool kit, therefore going beyond more restrictive fixed forms of normalization.

Download Full-text

Image discontinuity changes LFP gamma-band activity in primary visual cortex V1

Journal of Vision ◽

10.1167/9.8.753 ◽

2010 ◽

Vol 9 (8) ◽

pp. 753-753

Author(s):

D. Xing ◽

C.-I. Yeh ◽

P. Williams ◽

A. Henrie ◽

R. Shapley

Keyword(s):

Visual Cortex ◽

Primary Visual Cortex ◽

Gamma Band ◽

Gamma Band Activity ◽

Band Activity

Download Full-text

Laminar differences in responses to naturalistic texture in macaque V1 and V2

10.1101/710426 ◽

2019 ◽

Author(s):

Corey M Ziemba ◽

Richard K Perez ◽

Julia Pai ◽

Luke E Hallum ◽

Christopher Shooner ◽

...

Keyword(s):

Visual Cortex ◽

Order Statistics ◽

Higher Order ◽

Gamma Band ◽

Single Units ◽

Higher Order Statistics ◽

Image Structure ◽

Gamma Band Activity ◽

Granular Layers ◽

Band Activity

AbstractMost single units recorded from macaque V2 respond with higher firing rates to synthetic texture images containing “naturalistic” higher-order statistics than to spectrally matched “noise” images lacking these statistics. In contrast, few single units in V1 show this property. We explored how the strength and dynamics of response vary across the different layers of visual cortex by recording multiunit and gamma band activity evoked by brief presentations of naturalistic and noise images in V1 and V2 of anesthetized macaque monkeys. As previously reported, recordings in V2 showed consistently stronger responses to naturalistic texture than to spectrally matched noise. In contrast to single unit recordings, V1 multiunit activity showed some preference for images with naturalistic statistics, and in gamma band activity this preference was comparable across V1 and V2. Sensitivity to naturalistic image structure was strongest in the supragranular and infragranular layers of V1, but weak in granular layers, suggesting that it might reflect feedback from V2. Response timing was consistent with this idea. Visual responses appeared first in V1, followed by V2. Sensitivity to naturalistic texture emerged first in V2, followed by the supragranular and infragranular layers of V1, and finally in the granular layers of V1. Our results demonstrate laminar differences in the encoding of higher-order statistics of natural texture, and suggest that this sensitivity first arises in V2 and is fed back to modulate activity in V1.Significance StatementThe circuit mechanisms responsible for visual representations of intermediate complexity are largely unknown. We used a well-validated set of synthetic texture stimuli to probe the temporal and laminar profile of sensitivity to the higher-order statistical structure of natural images. We found that this sensitivity emerges first and most strongly in V2 but soon after in V1. However, sensitivity in V1 is higher in the laminae (extragranular) and recording modalities (local field potential) most likely affected by V2 connections, suggesting a feedback origin. Our results show how sensitivity to naturalistic image structure emerges across time and circuitry in the early visual cortex.

Download Full-text

Stimulus-specific plasticity in human visual gamma-band activity and functional connectivity

eLife ◽

10.7554/elife.68240 ◽

2021 ◽

Vol 10 ◽

Author(s):

Benjamin J Stauch ◽

Alina Peter ◽

Heike Schuler ◽

Pascal Fries

Keyword(s):

Visual Cortex ◽

Human Subjects ◽

Gamma Band ◽

Stimulus Repetition ◽

Oscillatory Dynamics ◽

Early Visual Cortex ◽

Neuronal Processing ◽

Plastic Changes ◽

Gamma Band Activity ◽

Band Activity

Under natural conditions, the visual system often sees a given input repeatedly. This provides an opportunity to optimize processing of the repeated stimuli. Stimulus repetition has been shown to strongly modulate neuronal-gamma band synchronization, yet crucial questions remained open. Here we used magnetoencephalography in 30 human subjects and find that gamma decreases across ≈10 repetitions and then increases across further repetitions, revealing plastic changes of the activated neuronal circuits. Crucially, increases induced by one stimulus did not affect responses to other stimuli, demonstrating stimulus specificity. Changes partially persisted when the inducing stimulus was repeated after 25 minutes of intervening stimuli. They were strongest in early visual cortex and increased interareal feedforward influences. Our results suggest that early visual cortex gamma synchronization enables adaptive neuronal processing of recurring stimuli. These and previously reported changes might be due to an interaction of oscillatory dynamics with established synaptic plasticity mechanisms.

Download Full-text