scholarly journals Chunking sequence information by mutually predicting recurrent neural networks

2017 ◽  
Author(s):  
Toshitake Asabuki ◽  
Naoki Hiratani ◽  
Tomoki Fukai
Keyword(s):  
Neural Networks ◽  
Basal Ganglia ◽  
Language Processing ◽  
Cognitive Tasks ◽  
Sequence Information ◽  
Reservoir Computing ◽  
Neural Responses ◽  
Complex Sequences ◽  
Unsupervised Neural Networks ◽  
The Brain

AbstractInterpretation and execution of complex sequences is crucial for various cognitive tasks such as language processing and motor control. The brain solves this problem arguably by dividing a sequence into discrete chunks of contiguous items. While chunking has been accounted for by predictive uncertainty, alternative mechanisms have also been suggested, and the mechanism underlying chunking is poorly understood. Here, we propose a class of unsupervised neural networks for learning and identifying repeated patterns in sequence input with various degrees of complexity. In this model, a pair of reservoir computing modules, each of which comprises a recurrent neural network and readout units, supervise each other to consistently predict others’ responses to frequently recurring segments. Interestingly, this system generates neural responses similar to those formed in the basal ganglia during habit formation. Our model extends reservoir computing to higher cognitive function and demonstrates its resemblance to sequence processing by cortico-basal ganglia loops.

scholarly journals Attentional Reinforcement Learning in the Brain

2020 ◽  
Vol 38 (1) ◽  
pp. 49-64 ◽  
Author(s):  
Hiroshi Yamakawa
Keyword(s):  
Deep Learning ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Language Processing ◽  
Information Source ◽  
Attention Mechanism ◽  
Transmission Route ◽  
Thalamic Relay ◽  
Signal Changes ◽  
The Brain

AbstractRecently, attention mechanisms have significantly boosted the performance of natural language processing using deep learning. An attention mechanism can select the information to be used, such as by conducting a dictionary lookup; this information is then used, for example, to select the next utterance word in a sentence. In neuroscience, the basis of the function of sequentially selecting words is considered to be the cortico-basal ganglia-thalamocortical loop. Here, we first show that the attention mechanism used in deep learning corresponds to the mechanism in which the cerebral basal ganglia suppress thalamic relay cells in the brain. Next, we demonstrate that, in neuroscience, the output of the basal ganglia is associated with the action output in the actor of reinforcement learning. Based on these, we show that the aforementioned loop can be generalized as reinforcement learning that controls the transmission of the prediction signal so as to maximize the prediction reward. We call this attentional reinforcement learning (ARL). In ARL, the actor selects the information transmission route according to the attention, and the prediction signal changes according to the context detected by the information source of the route. Hence, ARL enables flexible action selection that depends on the situation, unlike traditional reinforcement learning, wherein the actor must directly select an action.

scholarly journals The temporal binding deficit hypothesis of autism

2002 ◽  
Vol 14 (2) ◽  
pp. 209-224 ◽  
Author(s):  
JON BROCK ◽  
CAROLINE C. BROWN ◽  
JILL BOUCHER ◽  
GINA RIPPON
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
High Frequency ◽  
Executive Dysfunction ◽  
Weak Central Coherence ◽  
Gamma Activity ◽  
Central Coherence ◽  
Temporal Binding ◽  
Local Networks ◽  
The Brain

Frith has argued that people with autism show “weak central coherence,” an unusual bias toward piecemeal rather than configurational processing and a reduction in the normal tendency to process information in context. However, the precise cognitive and neurological mechanisms underlying weak central coherence are still unknown. We propose the hypothesis that the features of autism associated with weak central coherence result from a reduction in the integration of specialized local neural networks in the brain caused by a deficit in temporal binding. The visuoperceptual anomalies associated with weak central coherence may be attributed to a reduction in synchronization of high-frequency gamma activity between local networks processing local features. The failure to utilize context in language processing in autism can be explained in similar terms. Temporal binding deficits could also contribute to executive dysfunction in autism and to some of the deficits in socialization and communication.

scholarly journals The neural architecture of language: Integrative reverse-engineering converges on a model for predictive processing

2020 ◽  
Author(s):  
Martin Schrimpf ◽  
Idan Blank ◽  
Greta Tuckute ◽  
Carina Kauf ◽  
Eghbal A. Hosseini ◽  
...  
Keyword(s):  
Reverse Engineering ◽  
Language Processing ◽  
Brain Function ◽  
Computational Models ◽  
Predictive Processing ◽  
Neural Responses ◽  
Data Types ◽  
Neural Architecture ◽  
And Behavior ◽  
The Brain

AbstractThe neuroscience of perception has recently been revolutionized with an integrative reverse-engineering approach in which computation, brain function, and behavior are linked across many different datasets and many computational models. We here present a first systematic study taking this approach into higher-level cognition: human language processing, our species’ signature cognitive skill. We find that the most powerful ‘transformer’ networks predict neural responses at nearly 100% and generalize across different datasets and data types (fMRI, ECoG). Across models, significant correlations are observed among all three metrics of performance: neural fit, fit to behavioral responses, and accuracy on the next-word prediction task (but not other language tasks), consistent with the long-standing hypothesis that the brain’s language system is optimized for predictive processing. Model architectures with initial weights further perform surprisingly similar to final trained models, suggesting that inherent structure – and not just experience with language – crucially contributes to a model’s match to the brain.

scholarly journals A recurrent neural network framework for flexible and adaptive decision making based on sequence learning

2020 ◽  
Vol 16 (11) ◽  
pp. e1008342
Author(s):  
Zhewei Zhang ◽  
Huzi Cheng ◽  
Tianming Yang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Sequence Learning ◽  
Empirical Studies ◽  
Activity Patterns ◽  
Network Models ◽  
Neural Network Approach ◽  
Encode Task ◽  
The Brain

The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism’s survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurrent unit (GRU) networks in particular, reflect how the brain solves the contingency problem. Therefore, we build a GRU network framework inspired by the statistical learning approach of NLP and test it with four exemplar behavior tasks previously used in empirical studies. The network models are trained to predict future events based on past events, both comprising sensory, action, and reward events. We show the networks can successfully reproduce animal and human behavior. The networks generalize the training, perform Bayesian inference in novel conditions, and adapt their choices when event contingencies vary. Importantly, units in the network encode task variables and exhibit activity patterns that match previous neurophysiology findings. Our results suggest that the neural network approach based on statistical sequence learning may reflect the brain’s computational principle underlying flexible and adaptive behaviors and serve as a useful approach to understand the brain.

scholarly journals Modeling Neurodegeneration in silico With Deep Learning

2021 ◽  
Vol 15 ◽  
Author(s):  
Anup Tuladhar ◽  
Jasmine A. Moore ◽  
Zahinoor Ismail ◽  
Nils D. Forkert
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Object Recognition ◽  
Language Processing ◽  
Neural Plasticity ◽  
In Silico ◽  
Cortical Atrophy ◽  
Visual Object ◽  
Deep Convolutional Neural Networks ◽  
The Brain

Deep neural networks, inspired by information processing in the brain, can achieve human-like performance for various tasks. However, research efforts to use these networks as models of the brain have primarily focused on modeling healthy brain function so far. In this work, we propose a paradigm for modeling neural diseases in silico with deep learning and demonstrate its use in modeling posterior cortical atrophy (PCA), an atypical form of Alzheimer’s disease affecting the visual cortex. We simulated PCA in deep convolutional neural networks (DCNNs) trained for visual object recognition by randomly injuring connections between artificial neurons. Results showed that injured networks progressively lost their object recognition capability. Simulated PCA impacted learned representations hierarchically, as networks lost object-level representations before category-level representations. Incorporating this paradigm in computational neuroscience will be essential for developing in silico models of the brain and neurological diseases. The paradigm can be expanded to incorporate elements of neural plasticity and to other cognitive domains such as motor control, auditory cognition, language processing, and decision making.

Interpolation and Extrapolation in Human Behavior and Neural Networks

2004 ◽  
Vol 16 (3) ◽  
pp. 382-389 ◽  
Author(s):  
Emmanuel Guigon
Keyword(s):  
Neural Networks ◽  
Human Behavior ◽  
Neural Computation ◽  
Least Square ◽  
Cognitive Tasks ◽  
Visual Behavior ◽  
Function Learning ◽  
Visuomotor Transformations ◽  
Error Learning ◽  
The Brain

Unlike most artificial systems, the brain is able to face situations that it has not learned or even encountered before. This ability is not in general echoed by the properties of most neural networks. Here, we show that neural computation based on least-square error learning between populations of intensitycoded neurons can explain interpolation and extrapolation capacities of the nervous system in sensorimotor and cognitive tasks. We present simulations for function learning experiments, auditory-visual behavior, and visuomotor transformations. The results suggest that induction in human behavior, be it sensorimotor or cognitive, could arise from a common neural associative mechanism.

scholarly journals Neural Contributions of the Hypothalamus to Parental Behaviour

2021 ◽  
Vol 22 (13) ◽  
pp. 6998
Author(s):  
Chitose Orikasa
Keyword(s):  
Neural Networks ◽  
Neural Circuits ◽  
Social Cues ◽  
Male Mice ◽  
Neural Responses ◽  
Parental Behaviour ◽  
Melanin Concentrating Hormone ◽  
Virgin Male ◽  
And Function ◽  
The Brain

Parental behaviour is a comprehensive set of neural responses to social cues. The neural circuits that govern parental behaviour reside in several putative nuclei in the brain. Melanin concentrating hormone (MCH), a neuromodulator that integrates physiological functions, has been confirmed to be involved in parental behaviour, particularly in crouching behaviour during nursing. Abolishing MCH neurons in innate MCH knockout males promotes infanticide in virgin male mice. To understand the mechanism and function of neural networks underlying parental care and aggression against pups, it is essential to understand the basic organisation and function of the involved nuclei. This review presents newly discovered aspects of neural circuits within the hypothalamus that regulate parental behaviours.

Magnetoencephalography and the Cortical Dynamics of Language Processing

2019 ◽  
pp. 114-153
Author(s):  
Riitta Salmelin ◽  
Jan Kujala ◽  
Mia Liljeström
Keyword(s):  
Language Processing ◽  
Language Disorders ◽  
Scientific Data ◽  
Neural Processing ◽  
Powerful Method ◽  
Cortical Dynamics ◽  
Brain Correlates ◽  
Adults And Children ◽  
The Brain ◽  
Selection Of

When seeking to uncover the brain correlates of language processing, timing and location are of the essence. Magnetoencephalography (MEG) offers them both, with the highest sensitivity to cortical activity. MEG has shown its worth in revealing cortical dynamics of reading, speech perception, and speech production in adults and children, in unimpaired language processing as well as developmental and acquired language disorders. The MEG signals, once recorded, provide an extensive selection of measures for examination of neural processing. Like all other neuroimaging tools, MEG has its own strengths and limitations of which the user should be aware in order to make the best possible use of this powerful method and to generate meaningful and reliable scientific data. This chapter reviews MEG methodology and how MEG has been used to study the cortical dynamics of language.

scholarly journals Basal Ganglia Herniation into the Fourth Ventricle

1997 ◽  
Vol 24 (1) ◽  
pp. 62-63 ◽  
Author(s):  
Mensura Altumbabic ◽  
Marc R. Del Bigio ◽  
Scott Sutherland
Keyword(s):  
Basal Ganglia ◽  
Intracerebral Hemorrhage ◽  
Fourth Ventricle ◽  
Transtentorial Herniation ◽  
Intracerebral Bleeding ◽  
Rare Phenomenon ◽  
The Brain

ABSTRACT:Background:Transtentorial herniation of large cerebral fragments is a rare phenomenon.Method:Case StudyResults:Examination of the brain of a 35-year-old male showed massive intracerebral hemorrhage resulting in displacement of basal ganglia components into the fourth ventricle.Conclusions:Sufficiently rapid intracerebral bleeding can dissect fragments of cerebrum and displace them long distances across the tentorial opening.

Sign in / Sign up

Export Citation Format

Share Document