Decision letter: Perceptual processing in the ventral visual stream requires area TE but not rhinal cortex

There is an on-going debate over whether area TE, or the anatomically adjacent rhinal cortex, is the final stage of visual object processing. Both regions have been implicated in visual perception, but their involvement in non-perceptual functions, such as short-term memory, hinders clear-cut interpretation. Here, using a two-interval forced choice task without a short-term memory demand, we find that after bilateral removal of area TE, monkeys trained to categorize images based on perceptual similarity (morphs between dogs and cats), are, on the initial viewing, badly impaired when given a new set of images. They improve markedly with a small amount of practice but nonetheless remain moderately impaired indefinitely. The monkeys with bilateral removal of rhinal cortex are, under all conditions, indistinguishable from unoperated controls. We conclude that the final stage of the integration of visual perceptual information into object percepts in the ventral visual stream occurs in area TE.

Download Full-text

When the ventral visual stream is not enough: A deep learning account of medial temporal lobe involvement in perception

10.1101/2020.10.07.327171 ◽

2020 ◽

Author(s):

Tyler Bonnen ◽

Daniel L.K. Yamins ◽

Anthony D. Wagner

Keyword(s):

Deep Learning ◽

Temporal Lobe ◽

Medial Temporal Lobe ◽

Perceptual Processing ◽

Visual Stream ◽

Computational Framework ◽

Electrophysiological Recordings ◽

Visual Behaviors ◽

Ventral Visual Stream ◽

Experimental Findings

The medial temporal lobe (MTL) supports a constellation of memory-related behaviors. Its involvement in perceptual processing, however, has been subject to an enduring debate. This debate centers on perirhinal cortex (PRC), an MTL structure at the apex of the ventral visual stream (VVS). Here we leverage a deep learning approach that approximates visual behaviors supported by the VVS. We first apply this approach retroactively, modeling 29 published concurrent visual discrimination experiments: Excluding misclassified stimuli, there is a striking correspondence between VVS-modeled and PRC-lesioned behavior, while each are outperformed by PRC-intact participants. We corroborate these results using high-throughput psychophysics experiments: PRC-intact participants outperform a linear readout of electrophysiological recordings from the macaque VVS. Finally, in silico experiments suggest PRC enables out-of-distribution visual behaviors at rapid timescales. By situating these lesion, electrophysiological, and behavioral results within a shared computational framework, this work resolves decades of seemingly inconsistent experimental findings surrounding PRC involvement in perception.

Download Full-text

Using Convolutional Neural Networks to measure the contribution of visual features to the representation of object animacy in the brain

10.31237/osf.io/fxz4q ◽

2019 ◽

Cited By ~ 1

Author(s):

Sushrut Thorat

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Temporal Cortex ◽

Large Degree ◽

Visual Features ◽

Visual Feature ◽

Visual Stream ◽

Feature Information ◽

Ventral Visual Stream ◽

Animal Images

A mediolateral gradation in neural responses for images spanning animals to artificial objects is observed in the ventral temporal cortex (VTC). Which information streams drive this organisation is an ongoing debate. Recently, in Proklova et al. (2016), the visual shape and category (“animacy”) dimensions in a set of stimuli were dissociated using a behavioural measure of visual feature information. fMRI responses revealed a neural cluster (extra-visual animacy cluster - xVAC) which encoded category information unexplained by visual feature information, suggesting extra-visual contributions to the organisation in the ventral visual stream. We reassess these findings using Convolutional Neural Networks (CNNs) as models for the ventral visual stream. The visual features developed in the CNN layers can categorise the shape-matched stimuli from Proklova et al. (2016) in contrast to the behavioural measures used in the study. The category organisations in xVAC and VTC are explained to a large degree by the CNN visual feature differences, casting doubt over the suggestion that visual feature differences cannot account for the animacy organisation. To inform the debate further, we designed a set of stimuli with animal images to dissociate the animacy organisation driven by the CNN visual features from the degree of familiarity and agency (thoughtfulness and feelings). Preliminary results from a new fMRI experiment designed to understand the contribution of these non-visual features are presented.

Download Full-text

Expectation and Surprise Determine Neural Population Responses in the Ventral Visual Stream

Journal of Neuroscience ◽

10.1523/jneurosci.2770-10.2010 ◽

2010 ◽

Vol 30 (49) ◽

pp. 16601-16608 ◽

Cited By ~ 185

Author(s):

T. Egner ◽

J. M. Monti ◽

C. Summerfield

Keyword(s):

Neural Population ◽

Visual Stream ◽

Population Responses ◽

Ventral Visual Stream

Download Full-text

Orthographic processing deficits in developmental dyslexia: Beyond the ventral visual stream

NeuroImage ◽

10.1016/j.neuroimage.2016.01.014 ◽

2016 ◽

Vol 128 ◽

pp. 316-327 ◽

Cited By ~ 32

Author(s):

Marianna Boros ◽

Jean-Luc Anton ◽

Catherine Pech-Georgel ◽

Jonathan Grainger ◽

Marcin Szwed ◽

...

Keyword(s):

Developmental Dyslexia ◽

Orthographic Processing ◽

Visual Stream ◽

Ventral Visual Stream ◽

Processing Deficits

Download Full-text

Predictive coding of action intentions in dorsal and ventral visual stream is based on visual anticipations, memory-based information and motor preparation

10.1101/480590 ◽

2018 ◽

Author(s):

Simona Monaco ◽

Giulia Malfatti ◽

Alessandro Zendron ◽

Elisa Pellencin ◽

Luca Turella

Keyword(s):

Visual Information ◽

Visual Memory ◽

Visual Recognition ◽

Predictive Coding ◽

Action Planning ◽

Neural Signals ◽

Motor Representation ◽

Visual Stream ◽

Ventral Visual Stream ◽

Action Intention

AbstractPredictions of upcoming movements are based on several types of neural signals that span the visual, somatosensory, motor and cognitive system. Thus far, pre-movement signals have been investigated while participants viewed the object to be acted upon. Here, we studied the contribution of information other than vision to the classification of preparatory signals for action, even in absence of online visual information. We used functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis (MVPA) to test whether the neural signals evoked by visual, memory-based and somato-motor information can be reliably used to predict upcoming actions in areas of the dorsal and ventral visual stream during the preparatory phase preceding the action, while participants were lying still. Nineteen human participants (nine women) performed one of two actions towards an object with their eyes open or closed. Despite the well-known role of ventral stream areas in visual recognition tasks and the specialization of dorsal stream areas in somato-motor processes, we decoded action intention in areas of both streams based on visual, memory-based and somato-motor signals. Interestingly, we could reliably decode action intention in absence of visual information based on neural activity evoked when visual information was available, and vice-versa. Our results show a similar visual, memory and somato-motor representation of action planning in dorsal and ventral visual stream areas that allows predicting action intention across domains, regardless of the availability of visual information.

Download Full-text

Parallel processing of colors and faces in human ventral visual stream: functional evidence and technical challenges

Journal of Vision ◽

10.1167/14.10.985 ◽

2014 ◽

Vol 14 (10) ◽

pp. 985-985

Author(s):

R. Lafer-Sousa ◽

A. Kell ◽

A. Takahashi ◽

J. Feather ◽

B. Conway ◽

...

Keyword(s):

Parallel Processing ◽

Visual Stream ◽

Ventral Visual Stream ◽

Technical Challenges

Download Full-text

Image content is more important than Bouma’s Law for scene metamers

eLife ◽

10.7554/elife.42512 ◽

2019 ◽

Vol 8 ◽

Cited By ~ 8

Author(s):

Thomas SA Wallis ◽

Christina M Funke ◽

Alexander S Ecker ◽

Leon A Gatys ◽

Felix A Wichmann ◽

...

Keyword(s):

Perceptual Experience ◽

Receptive Fields ◽

Natural Images ◽

Retinal Eccentricity ◽

High Fidelity ◽

Prior Work ◽

Visual Stream ◽

Area V2 ◽

Ventral Visual Stream ◽

Stream Model

We subjectively perceive our visual field with high fidelity, yet peripheral distortions can go unnoticed and peripheral objects can be difficult to identify (crowding). Prior work showed that humans could not discriminate images synthesised to match the responses of a mid-level ventral visual stream model when information was averaged in receptive fields with a scaling of about half their retinal eccentricity. This result implicated ventral visual area V2, approximated ‘Bouma’s Law’ of crowding, and has subsequently been interpreted as a link between crowding zones, receptive field scaling, and our perceptual experience. However, this experiment never assessed natural images. We find that humans can easily discriminate real and model-generated images at V2 scaling, requiring scales at least as small as V1 receptive fields to generate metamers. We speculate that explaining why scenes look as they do may require incorporating segmentation and global organisational constraints in addition to local pooling.

Download Full-text