scholarly journals Cortical response to naturalistic stimuli is largely predictable with deep neural networks

2021 ◽  
Vol 7 (22) ◽  
pp. eabe7547
Author(s):  
Meenakshi Khosla ◽  
Gia H. Ngo ◽  
Keith Jamison ◽  
Amy Kuceyeski ◽  
Mert R. Sabuncu

Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict neural responses to arbitrary stimuli can be very useful for studying brain function. However, existing models focus on limited aspects of naturalistic stimuli, ignoring the dynamic interactions of modalities in this inherently context-rich paradigm. Using movie-watching data from the Human Connectome Project, we build group-level models of neural activity that incorporate several inductive biases about neural information processing, including hierarchical processing, temporal assimilation, and auditory-visual interactions. We demonstrate how incorporating these biases leads to remarkable prediction performance across large areas of the cortex, beyond the sensory-specific cortices into multisensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize to task-bound paradigms. Together, our findings underscore the potential of encoding models as powerful tools for studying brain function in ecologically valid conditions.

2020 ◽  
Author(s):  
Meenakshi Khosla ◽  
Gia H. Ngo ◽  
Keith Jamison ◽  
Amy Kuceyeski ◽  
Mert R. Sabuncu

Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict the neural response to a given stimulus can be very useful for studying brain function. However, existing neural encoding models focus on limited aspects of naturalistic stimuli, ignoring the complex and dynamic interactions of modalities in this inherently context-rich paradigm. Using movie watching data from the Human Connectome Project (HCP, N = 158) database, we build group-level models of neural activity that incorporate several inductive biases about information processing in the brain, including hierarchical processing, assimilation over longer timescales and multi-sensory auditory-visual interactions. We demonstrate how incorporating this joint information leads to remarkable prediction performance across large areas of the cortex, well beyond the visual and auditory cortices into multi-sensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize remarkably well to alternate task-bound paradigms. Taken together, our findings underscore the potential of neural encoding models as a powerful tool for studying brain function in ecologically valid conditions.


2017 ◽  
Author(s):  
Kendrick N. Kay ◽  
Kevin S. Weiner

AbstractThe goal of cognitive neuroscience is to understand how mental operations are performed by the brain. Given the complexity of the brain, this is a challenging endeavor that requires the development of formal models. Here, we provide a perspective on models of neural information processing in cognitive neuroscience. We define what these models are, explain why they are useful, and specify criteria for evaluating models. We also highlight the difference between functional and mechanistic models, and call attention to the value that neuroanatomy has for understanding brain function. Based on the principles we propose, we proceed to evaluate the merit of recently touted deep neural network models. We contend that these models are promising, but substantial work is necessary to (i) clarify what type of explanation these models provide, (ii) determine what specific effects they accurately explain, and (iii) improve our understanding of how they work.


2021 ◽  
pp. 1-16
Author(s):  
Stefanie Duyck ◽  
Farah Martens ◽  
Chiu-Yueh Chen ◽  
Hans Op de Beeck

Abstract Many people develop expertise in specific domains of interest, such as chess, microbiology, radiology, and, the case in point in our study: ornithology. It is poorly understood to what extent such expertise alters brain function. Previous neuroimaging studies of expertise have typically focused upon the category level, for example, selectivity for birds versus nonbird stimuli. We present a multivariate fMRI study focusing upon the representational similarity among objects of expertise at the subordinate level. We compare the neural representational spaces of experts and novices to behavioral judgments. At the behavioral level, ornithologists (n = 20) have more fine-grained and task-dependent representations of item similarity that are more consistent among experts compared to control participants. At the neural level, the neural patterns of item similarity are more distinct and consistent in experts than in novices, which is in line with the behavioral results. In addition, these neural patterns in experts show stronger correlations with behavior compared to novices. These findings were prominent in frontal regions, and some effects were also found in occipitotemporal regions. This study illustrates the potential of an analysis of representational geometry to understand to what extent expertise changes neural information processing.


2012 ◽  
Vol 24 (5) ◽  
pp. 1147-1185 ◽  
Author(s):  
C. C. Alan Fung ◽  
K. Y. Michael Wong ◽  
He Wang ◽  
Si Wu

Experimental data have revealed that neuronal connection efficacy exhibits two forms of short-term plasticity: short-term depression (STD) and short-term facilitation (STF). They have time constants residing between fast neural signaling and rapid learning and may serve as substrates for neural systems manipulating temporal information on relevant timescales. This study investigates the impact of STD and STF on the dynamics of continuous attractor neural networks and their potential roles in neural information processing. We find that STD endows the network with slow-decaying plateau behaviors: the network that is initially being stimulated to an active state decays to a silent state very slowly on the timescale of STD rather than on that of neuralsignaling. This provides a mechanism for neural systems to hold sensory memory easily and shut off persistent activities gracefully. With STF, we find that the network can hold a memory trace of external inputs in the facilitated neuronal interactions, which provides a way to stabilize the network response to noisy inputs, leading to improved accuracy in population decoding. Furthermore, we find that STD increases the mobility of the network states. The increased mobility enhances the tracking performance of the network in response to time-varying stimuli, leading to anticipative neural responses. In general, we find that STD and STP tend to have opposite effects on network dynamics and complementary computational advantages, suggesting that the brain may employ a strategy of weighting them differentially depending on the computational purpose.


2014 ◽  
Vol 111 (12) ◽  
pp. 2433-2444 ◽  
Author(s):  
Y. Lerner ◽  
C. J. Honey ◽  
M. Katkov ◽  
U. Hasson

Different brain areas integrate information over different timescales, and this capacity to accumulate information increases from early sensory areas to higher order perceptual and cognitive areas. It is currently unknown whether the timescale capacity of each brain area is fixed or whether it adaptively rescales depending on the rate at which information arrives from the world. Here, using functional MRI, we measured brain responses to an auditory narrative presented at different rates. We asked whether neural responses to slowed (speeded) versions of the narrative could be compressed (stretched) to match neural responses to the original narrative. Temporal rescaling was observed in early auditory regions (which accumulate information over short timescales) as well as linguistic and extra-linguistic brain areas (which can accumulate information over long timescales). The temporal rescaling phenomenon started to break down for stimuli presented at double speed, and intelligibility was also impaired for these stimuli. These data suggest that 1) the rate of neural information processing can be rescaled according to the rate of incoming information, both in early sensory regions as well as in higher order cortexes, and 2) the rescaling of neural dynamics is confined to a range of rates that match the range of behavioral performance.


1996 ◽  
Vol 06 (04) ◽  
pp. 575-582 ◽  
Author(s):  
HAVA T. SIEGELMANN

Analog recurrent neural networks have attracted much attention lately as powerful tools of automatic learning. However, they are not as popular in industry as should be justified by their usefulness. The lack of any programming tool for networks. and their vague internal representation, leave the networks for the use of experts only. We propose a way to make the neural networks friendly to users by formally defining a high level language, called Neural Information Processing Programming Langage, which is rich enough to express any computer algorithm or rule-based system. We show how to compile a NIL program into a network which computes exactly as the original program and requires the same computation/convergence time and physical size. Allowing for a natural neural evolution after the construction, the neural networks are both capable of dynamical continuous learning and represent any given symbolic knowledge. Thus, the language along with its compiler may be thought of as the ultimate bridge from symbolic to analog computation.


2020 ◽  
Author(s):  
Yang Tian ◽  
Justin L. Gardner ◽  
Guoqi Li ◽  
Pei Sun

AbstractInformation experiences complex transformation processes in the brain, involving various errors. A daunting and critical challenge in neuroscience is to understand the origin of these errors and their effects on neural information processing. While previous efforts have made substantial progresses in studying the information errors in bounded, unreliable and noisy transformation cases, it still remains elusive whether the neural system is inherently error-free under an ideal and noise-free condition. This work brings the controversy to an end with a negative answer. We propose a novel neural information confusion theory, indicating the widespread presence of information confusion phenomenon after the end of transmission process, which originates from innate neuron characteristics rather than external noises. Then, we reformulate the definition of zero-error capacity under the context of neuroscience, presenting an optimal upper bound of the zero-error transformation rates determined by the tuning properties of neurons. By applying this theory to neural coding analysis, we unveil the multi-dimensional impacts of information confusion on neural coding. Although it reduces the variability of neural responses and limits mutual information, it controls the stimulus-irrelevant neural activities and improves the interpretability of neural responses based on stimuli. Together, the present study discovers an inherent and ubiquitous precision limitation of neural information transformation, which shapes the coding process by neural ensembles. These discoveries reveal that the neural system is intrinsically error-prone in information processing even in the most ideal cases.Author summaryOne of the most central challenges in neuroscience is to understand the information processing capacity of the neural system. Decades of efforts have identified various errors in nonideal neural information processing cases, indicating that the neural system is not optimal in information processing because of the widespread presences of external noises and limitations. These incredible progresses, however, can not address the problem about whether the neural system is essentially error-free and optimal under ideal information processing conditions, leading to extensive controversies in neuroscience. Our work brings this well-known controversy to an end with a negative answer. We demonstrate that the neural system is intrinsically error-prone in information processing even in the most ideal cases, challenging the conventional ideas about the superior neural information processing capacity. We further indicate that the neural coding process is shaped by this innate limit, revealing how the characteristics of neural information functions and further cognitive functions are determined by the inherent limitation of the neural system.


2018 ◽  
Vol 115 (14) ◽  
pp. E3276-E3285 ◽  
Author(s):  
N. Apurva Ratan Murty ◽  
S. P. Arun

Object recognition is challenging because the same object can produce vastly different images, mixing signals related to its identity with signals due to its image attributes, such as size, position, rotation, etc. Previous studies have shown that both signals are present in high-level visual areas, but precisely how they are combined has remained unclear. One possibility is that neurons might encode identity and attribute signals multiplicatively so that each can be efficiently decoded without interference from the other. Here, we show that, in high-level visual cortex, responses of single neurons can be explained better as a product rather than a sum of tuning for object identity and tuning for image attributes. This subtle effect in single neurons produced substantially better population decoding of object identity and image attributes in the neural population as a whole. This property was absent both in low-level vision models and in deep neural networks. It was also unique to invariances: when tested with two-part objects, neural responses were explained better as a sum than as a product of part tuning. Taken together, our results indicate that signals requiring separate decoding, such as object identity and image attributes, are combined multiplicatively in IT neurons, whereas signals that require integration (such as parts in an object) are combined additively.


1967 ◽  
Vol 12 (11) ◽  
pp. 558-559
Author(s):  
STEPHAN L. CHOROVER

Sign in / Sign up

Export Citation Format

Share Document