scholarly journals When the ventral visual stream is not enough: A deep learning account of medial temporal lobe involvement in perception

Neuron ◽  
2021 ◽  
Author(s):  
Tyler Bonnen ◽  
Daniel L.K. Yamins ◽  
Anthony D. Wagner
Keyword(s):  
Deep Learning ◽  
Temporal Lobe ◽  
Medial Temporal Lobe ◽  
Visual Stream ◽  
Ventral Visual Stream

scholarly journals When the ventral visual stream is not enough: A deep learning account of medial temporal lobe involvement in perception

2020 ◽  
Author(s):  
Tyler Bonnen ◽  
Daniel L.K. Yamins ◽  
Anthony D. Wagner
Keyword(s):  
Deep Learning ◽  
Temporal Lobe ◽  
Medial Temporal Lobe ◽  
Perceptual Processing ◽  
Visual Stream ◽  
Computational Framework ◽  
Electrophysiological Recordings ◽  
Visual Behaviors ◽  
Ventral Visual Stream ◽  
Experimental Findings

The medial temporal lobe (MTL) supports a constellation of memory-related behaviors. Its involvement in perceptual processing, however, has been subject to an enduring debate. This debate centers on perirhinal cortex (PRC), an MTL structure at the apex of the ventral visual stream (VVS). Here we leverage a deep learning approach that approximates visual behaviors supported by the VVS. We first apply this approach retroactively, modeling 29 published concurrent visual discrimination experiments: Excluding misclassified stimuli, there is a striking correspondence between VVS-modeled and PRC-lesioned behavior, while each are outperformed by PRC-intact participants. We corroborate these results using high-throughput psychophysics experiments: PRC-intact participants outperform a linear readout of electrophysiological recordings from the macaque VVS. Finally, in silico experiments suggest PRC enables out-of-distribution visual behaviors at rapid timescales. By situating these lesion, electrophysiological, and behavioral results within a shared computational framework, this work resolves decades of seemingly inconsistent experimental findings surrounding PRC involvement in perception.

scholarly journals Animacy and real world size shape object representations in the human medial temporal lobes

2018 ◽  
Author(s):  
Anna Blumenthal ◽  
Bobby Stojanoski ◽  
Chris Martin ◽  
Rhodri Cusack ◽  
Stefan Köhler
Keyword(s):  
Recognition Memory ◽  
Real World ◽  
Medial Temporal Lobe ◽  
Memory Task ◽  
Functional Differentiation ◽  
Object Representations ◽  
Neural Basis ◽  
Visual Stream ◽  
Ventral Visual Stream ◽  
Domain Level

ABSTRACTIdentifying what an object is, and whether an object has been encountered before, is a crucial aspect of human behavior. Despite this importance, we do not have a complete understanding of the neural basis of these abilities. Investigations into the neural organization of human object representations have revealed category specific organization in the ventral visual stream in perceptual tasks. Interestingly, these categories fall within broader domains of organization, with distinctions between animate, inanimate large, and inanimate small objects. While there is some evidence for category specific effects in the medial temporal lobe (MTL), it is currently unclear whether domain level organization is also present across these structures. To this end, we used fMRI with a continuous recognition memory task. Stimuli were images of objects from several different categories, which were either animate or inanimate, or large or small within the inanimate domain. We employed representational similarity analysis (RSA) to test the hypothesis that object-evoked responses in MTL structures during recognition-memory judgments also show evidence for domain-level organization along both dimensions. Our data support this hypothesis. Specifically, object representations were shaped by either animacy, real-world size, or both, in perirhinal and parahippocampal cortex, as well as the hippocampus. While sensitivity to these dimensions differed when structures when probed individually, hinting at interesting links to functional differentiation, similarities in organization across MTL structures were more prominent overall. These results argue for continuity in the organization of object representations in the ventral visual stream and the MTL.

scholarly journals The end point of the ventral visual stream: face and non-face perceptual deficits following unilateral anterior temporal lobe damage

Neurocase ◽  
2014 ◽  
Vol 21 (5) ◽  
pp. 554-562 ◽  
Author(s):  
Ingrid R. Olson ◽  
Youssef Ezzyat ◽  
Alan Plotzker ◽  
Anjan Chatterjee
Keyword(s):  
Temporal Lobe ◽  
Anterior Temporal Lobe ◽  
Visual Stream ◽  
End Point ◽  
Ventral Visual Stream

scholarly journals A Deep Learning Account of How Language Affects Thought

2021 ◽  
Author(s):  
Xiaoliang Luo ◽  
Nicholas J. Sexton ◽  
Bradley C. Love
Keyword(s):  
Deep Learning ◽  
Language Learning ◽  
Coarse Grained ◽  
Semantic Representations ◽  
Visual Stream ◽  
Fine Grained ◽  
Linguistic Label ◽  
Underlying Theory ◽  
Ventral Visual Stream ◽  
Label Use

How can words shape meaning? Shared labels highlight commonalities between concepts whereas contrasting labels make differences apparent. To address such findings, we propose a deep learning account that spans perception to decision (i.e., labelling). The model takes photographs as input, transforms them to semantic representations through computations that parallel the ventral visual stream, and finally determines the appropriate linguistic label. The underlying theory is that minimising error on two prediction tasks (predicting the meaning and label of a stimulus) requires a compromise in the network's semantic representations. Thus, differences in label use, whether across languages or levels of expertise, manifest in differences in the semantic representations that support label discrimination. We confirm these predictions in simulations involving fine-grained and coarse-grained labels. We hope these and allied efforts which model perception, semantics, and labelling at scale will advance developmental and neurocomputational accounts of concept and language learning.

Functional Dissociations within the Ventral Object Processing Pathway: Cognitive Modules or a Hierarchical Continuum?

2010 ◽  
Vol 22 (11) ◽  
pp. 2460-2479 ◽  
Author(s):  
Rosemary A. Cowell ◽  
Timothy J. Bussey ◽  
Lisa M. Saksida
Keyword(s):  
Visual Perception ◽  
Temporal Lobe ◽  
Visual Discrimination ◽  
Visual Memory ◽  
Modular Organization ◽  
Visual Object ◽  
Visual Stream ◽  
Object Processing ◽  
Double Dissociation ◽  
Ventral Visual Stream

We examined the organization and function of the ventral object processing pathway. The prevailing theoretical approach in this field holds that the ventral object processing stream has a modular organization, in which visual perception is carried out in posterior regions and visual memory is carried out, independently, in the anterior temporal lobe. In contrast, recent work has argued against this modular framework, favoring instead a continuous, hierarchical account of cognitive processing in these regions. We join the latter group and illustrate our view with simulations from a computational model that extends the perceptual-mnemonic feature-conjunction model of visual discrimination proposed by Bussey and Saksida [Bussey, T. J., & Saksida, L. M. The organization of visual object representations: A connectionist model of effects of lesions in perirhinal cortex. European Journal of Neuroscience, 15, 355–364, 2002]. We use the extended model to revisit early data from Iwai and Mishkin [Iwai, E., & Mishkin, M. Two visual foci in the temporal lobe of monkeys. In N. Yoshii & N. Buchwald (Eds.), Neurophysiological basis of learning and behavior (pp. 1–11). Japan: Osaka University Press, 1968]; this seminal study was interpreted as evidence for the modularity of visual perception and visual memory. The model accounts for a double dissociation in monkeys' visual discrimination performance following lesions to different regions of the ventral visual stream. This double dissociation is frequently cited as evidence for separate systems for perception and memory. However, the model provides a parsimonious, mechanistic, single-system account of the double dissociation data. We propose that the effects of lesions in ventral visual stream on visual discrimination are due to compromised representations within a hierarchical representational continuum rather than impairment in a specific type of learning, memory, or perception. We argue that consideration of the nature of stimulus representations and their processing in cortex is a more fruitful approach than attempting to map cognition onto functional modules.

Children show adult-like hippocampal pattern separation for familiar but not novel events

2020 ◽  
Author(s):  
Susan L. Benear ◽  
Elizabeth A. Horwath ◽  
Emily Cowan ◽  
M. Catalina Camacho ◽  
Chi Ngo ◽  
...  
Keyword(s):  
Episodic Memory ◽  
Temporal Lobe ◽  
Medial Temporal Lobe ◽  
Developmental Change ◽  
Pattern Separation ◽  
Developmental Changes ◽  
Similarity Analysis ◽  
Separation Processes ◽  
Parahippocampal Cortex ◽  
Pattern Similarity

The medial temporal lobe (MTL) undergoes critical developmental change throughout childhood, which aligns with developmental changes in episodic memory. We used representational similarity analysis to compare neural pattern similarity for children and adults in hippocampus and parahippocampal cortex during naturalistic viewing of clips from the same movie or different movies. Some movies were more familiar to participants than others. Neural pattern similarity was generally lower for clips from the same movie, indicating that related content taxes pattern separation-like processes. However, children showed this effect only for movies with which they were familiar, whereas adults showed the effect consistently. These data suggest that children need more exposures to stimuli in order to show mature pattern separation processes.

Using Convolutional Neural Networks to measure the contribution of visual features to the representation of object animacy in the brain

2019 ◽  
Author(s):  
Sushrut Thorat
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Temporal Cortex ◽  
Large Degree ◽  
Visual Features ◽  
Visual Feature ◽  
Visual Stream ◽  
Feature Information ◽  
Ventral Visual Stream ◽  
Animal Images

A mediolateral gradation in neural responses for images spanning animals to artificial objects is observed in the ventral temporal cortex (VTC). Which information streams drive this organisation is an ongoing debate. Recently, in Proklova et al. (2016), the visual shape and category (“animacy”) dimensions in a set of stimuli were dissociated using a behavioural measure of visual feature information. fMRI responses revealed a neural cluster (extra-visual animacy cluster - xVAC) which encoded category information unexplained by visual feature information, suggesting extra-visual contributions to the organisation in the ventral visual stream. We reassess these findings using Convolutional Neural Networks (CNNs) as models for the ventral visual stream. The visual features developed in the CNN layers can categorise the shape-matched stimuli from Proklova et al. (2016) in contrast to the behavioural measures used in the study. The category organisations in xVAC and VTC are explained to a large degree by the CNN visual feature differences, casting doubt over the suggestion that visual feature differences cannot account for the animacy organisation. To inform the debate further, we designed a set of stimuli with animal images to dissociate the animacy organisation driven by the CNN visual features from the degree of familiarity and agency (thoughtfulness and feelings). Preliminary results from a new fMRI experiment designed to understand the contribution of these non-visual features are presented.

Sign in / Sign up

Export Citation Format

Share Document