scholarly journals When the ventral visual stream is not enough: A deep learning account of medial temporal lobe involvement in perception

2020 ◽  
Author(s):  
Tyler Bonnen ◽  
Daniel L.K. Yamins ◽  
Anthony D. Wagner
Keyword(s):  
Deep Learning ◽  
Temporal Lobe ◽  
Medial Temporal Lobe ◽  
Perceptual Processing ◽  
Visual Stream ◽  
Computational Framework ◽  
Electrophysiological Recordings ◽  
Visual Behaviors ◽  
Ventral Visual Stream ◽  
Experimental Findings

The medial temporal lobe (MTL) supports a constellation of memory-related behaviors. Its involvement in perceptual processing, however, has been subject to an enduring debate. This debate centers on perirhinal cortex (PRC), an MTL structure at the apex of the ventral visual stream (VVS). Here we leverage a deep learning approach that approximates visual behaviors supported by the VVS. We first apply this approach retroactively, modeling 29 published concurrent visual discrimination experiments: Excluding misclassified stimuli, there is a striking correspondence between VVS-modeled and PRC-lesioned behavior, while each are outperformed by PRC-intact participants. We corroborate these results using high-throughput psychophysics experiments: PRC-intact participants outperform a linear readout of electrophysiological recordings from the macaque VVS. Finally, in silico experiments suggest PRC enables out-of-distribution visual behaviors at rapid timescales. By situating these lesion, electrophysiological, and behavioral results within a shared computational framework, this work resolves decades of seemingly inconsistent experimental findings surrounding PRC involvement in perception.

scholarly journals Animacy and real world size shape object representations in the human medial temporal lobes

2018 ◽  
Author(s):  
Anna Blumenthal ◽  
Bobby Stojanoski ◽  
Chris Martin ◽  
Rhodri Cusack ◽  
Stefan Köhler
Keyword(s):  
Recognition Memory ◽  
Real World ◽  
Medial Temporal Lobe ◽  
Memory Task ◽  
Functional Differentiation ◽  
Object Representations ◽  
Neural Basis ◽  
Visual Stream ◽  
Ventral Visual Stream ◽  
Domain Level

ABSTRACTIdentifying what an object is, and whether an object has been encountered before, is a crucial aspect of human behavior. Despite this importance, we do not have a complete understanding of the neural basis of these abilities. Investigations into the neural organization of human object representations have revealed category specific organization in the ventral visual stream in perceptual tasks. Interestingly, these categories fall within broader domains of organization, with distinctions between animate, inanimate large, and inanimate small objects. While there is some evidence for category specific effects in the medial temporal lobe (MTL), it is currently unclear whether domain level organization is also present across these structures. To this end, we used fMRI with a continuous recognition memory task. Stimuli were images of objects from several different categories, which were either animate or inanimate, or large or small within the inanimate domain. We employed representational similarity analysis (RSA) to test the hypothesis that object-evoked responses in MTL structures during recognition-memory judgments also show evidence for domain-level organization along both dimensions. Our data support this hypothesis. Specifically, object representations were shaped by either animacy, real-world size, or both, in perirhinal and parahippocampal cortex, as well as the hippocampus. While sensitivity to these dimensions differed when structures when probed individually, hinting at interesting links to functional differentiation, similarities in organization across MTL structures were more prominent overall. These results argue for continuity in the organization of object representations in the ventral visual stream and the MTL.

scholarly journals The end point of the ventral visual stream: face and non-face perceptual deficits following unilateral anterior temporal lobe damage

Neurocase ◽  
2014 ◽  
Vol 21 (5) ◽  
pp. 554-562 ◽  
Author(s):  
Ingrid R. Olson ◽  
Youssef Ezzyat ◽  
Alan Plotzker ◽  
Anjan Chatterjee
Keyword(s):  
Temporal Lobe ◽  
Anterior Temporal Lobe ◽  
Visual Stream ◽  
End Point ◽  
Ventral Visual Stream

scholarly journals A Deep Learning Account of How Language Affects Thought

2021 ◽  
Author(s):  
Xiaoliang Luo ◽  
Nicholas J. Sexton ◽  
Bradley C. Love
Keyword(s):  
Deep Learning ◽  
Language Learning ◽  
Coarse Grained ◽  
Semantic Representations ◽  
Visual Stream ◽  
Fine Grained ◽  
Linguistic Label ◽  
Underlying Theory ◽  
Ventral Visual Stream ◽  
Label Use

How can words shape meaning? Shared labels highlight commonalities between concepts whereas contrasting labels make differences apparent. To address such findings, we propose a deep learning account that spans perception to decision (i.e., labelling). The model takes photographs as input, transforms them to semantic representations through computations that parallel the ventral visual stream, and finally determines the appropriate linguistic label. The underlying theory is that minimising error on two prediction tasks (predicting the meaning and label of a stimulus) requires a compromise in the network's semantic representations. Thus, differences in label use, whether across languages or levels of expertise, manifest in differences in the semantic representations that support label discrimination. We confirm these predictions in simulations involving fine-grained and coarse-grained labels. We hope these and allied efforts which model perception, semantics, and labelling at scale will advance developmental and neurocomputational accounts of concept and language learning.

scholarly journals Author response: Perceptual processing in the ventral visual stream requires area TE but not rhinal cortex

2018 ◽  
Author(s):  
Mark AG Eldridge ◽  
Narihisa Matsumoto ◽  
John H Wittig ◽  
Evan C Masseau ◽  
Richard C Saunders ◽  
...  
Keyword(s):  
Perceptual Processing ◽  
Author Response ◽  
Visual Stream ◽  
Rhinal Cortex ◽  
Ventral Visual Stream ◽  
Area Te

Functional Dissociations within the Ventral Object Processing Pathway: Cognitive Modules or a Hierarchical Continuum?

2010 ◽  
Vol 22 (11) ◽  
pp. 2460-2479 ◽  
Author(s):  
Rosemary A. Cowell ◽  
Timothy J. Bussey ◽  
Lisa M. Saksida
Keyword(s):  
Visual Perception ◽  
Temporal Lobe ◽  
Visual Discrimination ◽  
Visual Memory ◽  
Modular Organization ◽  
Visual Object ◽  
Visual Stream ◽  
Object Processing ◽  
Double Dissociation ◽  
Ventral Visual Stream

We examined the organization and function of the ventral object processing pathway. The prevailing theoretical approach in this field holds that the ventral object processing stream has a modular organization, in which visual perception is carried out in posterior regions and visual memory is carried out, independently, in the anterior temporal lobe. In contrast, recent work has argued against this modular framework, favoring instead a continuous, hierarchical account of cognitive processing in these regions. We join the latter group and illustrate our view with simulations from a computational model that extends the perceptual-mnemonic feature-conjunction model of visual discrimination proposed by Bussey and Saksida [Bussey, T. J., & Saksida, L. M. The organization of visual object representations: A connectionist model of effects of lesions in perirhinal cortex. European Journal of Neuroscience, 15, 355–364, 2002]. We use the extended model to revisit early data from Iwai and Mishkin [Iwai, E., & Mishkin, M. Two visual foci in the temporal lobe of monkeys. In N. Yoshii & N. Buchwald (Eds.), Neurophysiological basis of learning and behavior (pp. 1–11). Japan: Osaka University Press, 1968]; this seminal study was interpreted as evidence for the modularity of visual perception and visual memory. The model accounts for a double dissociation in monkeys' visual discrimination performance following lesions to different regions of the ventral visual stream. This double dissociation is frequently cited as evidence for separate systems for perception and memory. However, the model provides a parsimonious, mechanistic, single-system account of the double dissociation data. We propose that the effects of lesions in ventral visual stream on visual discrimination are due to compromised representations within a hierarchical representational continuum rather than impairment in a specific type of learning, memory, or perception. We argue that consideration of the nature of stimulus representations and their processing in cortex is a more fruitful approach than attempting to map cognition onto functional modules.

Sign in / Sign up

Export Citation Format

Share Document