scholarly journals An Exploration of Error-Driven Learning in Simple Two-Layer Networks From a Discriminative Learning Perspective

2020 ◽  
Author(s):  
Dorothée B. Hoppe ◽  
Petra Hendriks ◽  
Michael Ramscar ◽  
Jacolien van Rij
Keyword(s):  
Prediction Error ◽  
Historical Development ◽  
Discriminative Learning ◽  
Learning Models ◽  
Cognitive Sciences ◽  
Vast Array ◽  
Simple Error ◽  
Analysis Of Error ◽  
The Brain ◽  
Theoretical Analyses

Error-driven learning algorithms, which iteratively adjust expectations based on prediction error, are the basis for a vast array of models in the brain and cognitive sciences that often differ widely in their precise form and application: historically, they range from simple models in psychology and cybernetics to current complex deep learning models dominating discussions in machine learning and artificial intelligence. However, despite the ubiquity of this mechanism, detailed theoretical analyses of its basic workings uninfluenced by existing theories or specific research goals are rare in the literature. To address this, we present an exposition and theoretical analysis of error-driven learning - focusing on its simplest form for clarity - and relate this to the historical development of error-driven learning models in the cognitive sciences. Although historically error-driven models have been thought of as associative, such that learning is thought to combine preexisting elemental representations, this analysis will highlight the discriminative nature of learning in these models and the implications of this for the way how learning is conceptualized. We complement our theoretical exposition of error-driven learning with a practical guide to the application of simple error-driven learning models in which we discuss a number of example simulations, that are presented in detail in an accompanying tutorial.

scholarly journals L-DOPA Reduces Model-Free Control of Behavior by Attenuating the Transfer of Value to Action

2016 ◽  
Author(s):  
Nils B. Kroemer ◽  
Ying Lee ◽  
Shakoor Pooseh ◽  
Ben Eppinger ◽  
Thomas Goschke ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Behavioral Control ◽  
Decision Task ◽  
Learning Models ◽  
Model Free ◽  
Markov Decision ◽  
Model Free Control ◽  
Reinforcement Learning Models ◽  
The Brain

AbstractDopamine is a key neurotransmitter in reinforcement learning and action control. Recent findings suggest that these components are inherently entangled. Here, we tested if increases in dopamine tone by administration of L-DOPA upregulate deliberative “model-based” control of behavior or reflexive “model-free” control as predicted by dual-control reinforcement-learning models. Alternatively, L-DOPA may impair learning as suggested by “value” or “thrift” theories of dopamine. To this end, we employed a two-stage Markov decision-task to investigate the effect of L-DOPA (randomized cross-over) on behavioral control while brain activation was measured using fMRI. L-DOPA led to attenuated model-free control of behavior as indicated by the reduced impact of reward on choice and increased stochasticity of model-free choices. Correspondingly, in the brain, L-DOPA decreased the effect of reward while prediction-error signals were unaffected. Taken together, our results suggest that L-DOPA reduces model-free control of behavior by attenuating the transfer of value to action.

scholarly journals The neurobiology of uncertainty: implications for statistical learning

2017 ◽  
Vol 372 (1711) ◽  
pp. 20160048 ◽  
Author(s):  
Uri Hasson
Keyword(s):  
Statistical Learning ◽  
Historical Development ◽  
Visual Stimuli ◽  
Learning Mechanisms ◽  
Cognitive Sciences ◽  
Input Uncertainty ◽  
Environmental Features ◽  
Online Prediction ◽  
Neurobiological Research ◽  
The Brain

The capacity for assessing the degree of uncertainty in the environment relies on estimating statistics of temporally unfolding inputs. This, in turn, allows calibration of predictive and bottom-up processing, and signalling changes in temporally unfolding environmental features. In the last decade, several studies have examined how the brain codes for and responds to input uncertainty. Initial neurobiological experiments implicated frontoparietal and hippocampal systems, based largely on paradigms that manipulated distributional features of visual stimuli. However, later work in the auditory domain pointed to different systems, whose activation profiles have interesting implications for computational and neurobiological models of statistical learning (SL). This review begins by briefly recapping the historical development of ideas pertaining to the sensitivity to uncertainty in temporally unfolding inputs. It then discusses several issues at the interface of studies of uncertainty and SL. Following, it presents several current treatments of the neurobiology of uncertainty and reviews recent findings that point to principles that serve as important constraints on future neurobiological theories of uncertainty, and relatedly, SL. This review suggests it may be useful to establish closer links between neurobiological research on uncertainty and SL, considering particularly mechanisms sensitive to local and global structure in inputs, the degree of input uncertainty, the complexity of the system generating the input, learning mechanisms that operate on different temporal scales and the use of learnt information for online prediction. This article is part of the themed issue ‘New frontiers for statistical learning in the cognitive sciences’.

scholarly journals Prefrontal solution to the bias-variance tradeoff during reinforcement learning

2020 ◽  
Author(s):  
Dongjae Kim ◽  
Jaeseung Jeong ◽  
Sang Wan Lee
Keyword(s):  
Adaptive Control ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Brain Regions ◽  
Decision Task ◽  
Prediction Errors ◽  
Model Based ◽  
Model Free ◽  
Bias Variance ◽  
The Brain

AbstractThe goal of learning is to maximize future rewards by minimizing prediction errors. Evidence have shown that the brain achieves this by combining model-based and model-free learning. However, the prediction error minimization is challenged by a bias-variance tradeoff, which imposes constraints on each strategy’s performance. We provide new theoretical insight into how this tradeoff can be resolved through the adaptive control of model-based and model-free learning. The theory predicts the baseline correction for prediction error reduces the lower bound of the bias–variance error by factoring out irreducible noise. Using a Markov decision task with context changes, we showed behavioral evidence of adaptive control. Model-based behavioral analyses show that the prediction error baseline signals context changes to improve adaptability. Critically, the neural results support this view, demonstrating multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free learning.One sentence summaryA theoretical, behavioral, computational, and neural account of how the brain resolves the bias-variance tradeoff during reinforcement learning is described.

scholarly journals Glutamatergic modulation of auditory cortex connectivity with attentional brain networks in unpredictable perceptual environment

2019 ◽  
Author(s):  
K. Kompus ◽  
V. Volehaugen ◽  
A. Craven ◽  
K. Specht
Keyword(s):  
Predictive Model ◽  
Prediction Error ◽  
Model Building ◽  
Signal Propagation ◽  
Auditory Stimulation ◽  
Perceptual Inference ◽  
Auditory Oddball ◽  
Frontoparietal Network ◽  
Young Subjects ◽  
The Brain

AbstractIn a stable environment the brain can minimize processing required for sensory input by forming a predictive model of the surrounding world and suppressing neural response to predicted stimuli. Unpredicted stimuli lead to a prediction error signal propagation through the perceptual network, and resulting adjustment to the predictive model. The inter-regional plasticity which enables the model-building and model-adjustment is hypothesized to be mediated via glutamatergic receptors. While pharmacological challenge studies with glutamate receptor ligands have demonstrated impact on prediction-error indices, it is not clear how inter-individual differences in the glutamate system affect the prediction-error processing in non-medicated state. In the present study we examined 20 healthy young subjects with resting-state proton MRS spectroscopy to characterize glutamate+glutamine (rs-Glx) levels in their Heschl’s gyrus (HG), and related this to HG functional connectivity during a roving auditory oddball protocol. No rs-Glx effects were found within the frontotemporal prediction-error network. Larger rs-Glx signal was related to stronger connectivity between HG and bilateral inferior parietal lobule during unpredictable auditory stimulation. We also found effects of rs-Glx on the coherence of default mode network (DMN) and frontoparietal network (FPN) during unpredictable auditory stimulation. Our results demonstrate the importance of Glx in modulating long-range connections and wider networks in the brain during perceptual inference.

The Engineer

Rebel Genius ◽  
2016 ◽  
Author(s):  
Tara H. Abraham
Keyword(s):  
Artificial Intelligence ◽  
Reticular Formation ◽  
Research Laboratory ◽  
Biological Systems ◽  
Theoretical Modelling ◽  
Research Culture ◽  
Cognitive Sciences ◽  
New Research ◽  
Scientific Identity ◽  
The Brain

This chapter examines the ways that McCulloch’s new research culture at MIT’s Research Laboratory of Electronics shaped the evolution of his scientific identity into that of an engineer. This was an open, fluid, multidisciplinary culture that allowed McCulloch to shift his focus more squarely onto understanding the brain from the perspective of theoretical modelling, and to promote the cybernetic vision to diverse audiences. McCulloch’s practices, performed with a new set of student-collaborators, involved modeling the neurophysiology of perception, understanding reliability in biological systems, and pursuing knowledge of the reticular formation of the brain. The chapter provides a nuanced account of the relations between McCulloch’s work and the emerging fields of artificial intelligence and the cognitive sciences. It also highlights McCulloch’s identities as sage-collaborator and polymath, two roles that in part were the result of his students’ observations and in part products of his own self-fashioning.

Epilogue: Missing Information?

2017 ◽  
Author(s):  
Daniel D. Hutto ◽  
Erik Myin
Keyword(s):  
Explanatory Power ◽  
Informational Content ◽  
Hard Problem ◽  
Missing Information ◽  
Cognitive Sciences ◽  
Positioning Systems ◽  
The Hard Problem ◽  
The Brain ◽  
Hard Problem Of Content

The epilogue takes a last look at the possibility that REC may be leaving out something explanatorily important because it says nothing about how the brain processes informational content. Focusing on a prominent case, it is demonstrated that REC has the resources to understand the groundbreaking research on positioning systems in rat brains. It is argued that rat brains can be informationally sensitive without processing informational content. No explanatory power is lost in adopting REC’s deflated explanation; but much is gained by doing so since it avoids the Hard Problem of Content. The chapter concludes by showing how REC’s proposed vision of neurodynamics is wholly compatible with its dynamical and extensive account of cognition; a vision of cognition that opens the door to broader lines of research in the cognitive sciences that taking into account the ways in which culture can permeate cognition.

The Media-Sphere as Dream

2016 ◽  
pp. 232-260
Author(s):  
Stephen Brock Schafer
Keyword(s):  
Psychological Research ◽  
Electronic Media ◽  
Cognitive Sciences ◽  
Media Environment ◽  
Psychological States ◽  
Cognitive Organization ◽  
Dramatic Form ◽  
The Media ◽  
The Brain ◽  
Cognitive Dimensions

The psychological nature of the electronic media environment is a virtual reality that—according to Jungian principles—is dreamlike. Perhaps it can be analyzed with Jung's Analytical Psychology. Science is experiencing a paradigm shift into a reality of mediated illusion, and psychological research on this illusion has become the human imperative. It may be stipulated that physics has abolished matter, conceding that “reality is organized mind stuff.” If cosmos is structured holographically and the brain is structured holonomically, it is probable that “mind stuff” is structured holographically. The Jungian concept of Psyche is a good place to begin researching the Media-sphere as mind stuff. Cognitive sciences are probing the brain and nervous system in search of the template for cognitive organization, and the salient features have already emerged. It appears that both conscious and unconsciousness cognitive dimensions have dramatic form. This dreamlike structure can be employed to analyze the media dream, and to foster coherent psychological states in contextual collectives.

The Media-Sphere as Dream

2020 ◽  
pp. 343-371
Author(s):  
Stephen Brock Schafer
Keyword(s):  
Psychological Research ◽  
Electronic Media ◽  
Cognitive Sciences ◽  
Media Environment ◽  
Psychological States ◽  
Cognitive Organization ◽  
Dramatic Form ◽  
The Media ◽  
The Brain ◽  
Cognitive Dimensions

The psychological nature of the electronic media environment is a virtual reality that—according to Jungian principles—is dreamlike. Perhaps it can be analyzed with Jung's Analytical Psychology. Science is experiencing a paradigm shift into a reality of mediated illusion, and psychological research on this illusion has become the human imperative. It may be stipulated that physics has abolished matter, conceding that “reality is organized mind stuff.” If cosmos is structured holographically and the brain is structured holonomically, it is probable that “mind stuff” is structured holographically. The Jungian concept of Psyche is a good place to begin researching the Media-sphere as mind stuff. Cognitive sciences are probing the brain and nervous system in search of the template for cognitive organization, and the salient features have already emerged. It appears that both conscious and unconsciousness cognitive dimensions have dramatic form. This dreamlike structure can be employed to analyze the media dream, and to foster coherent psychological states in contextual collectives.

The history and scope of neuropsychiatry

2020 ◽  
pp. 3-12
Author(s):  
Michael Trimble
Keyword(s):  
Twentieth Century ◽  
Nervous System ◽  
Human Brain ◽  
Limbic System ◽  
Historical Development ◽  
The Body ◽  
The Enlightenment ◽  
Romantic Era ◽  
The Brain

This chapter discusses the clinical necessity from which the intersection of neurology and psychiatry arose, exploring different eras and their associated intellectual milestones in order to understand the historical framework of contemporary neuropsychiatry. Identifying Hippocrates’ original acknowledgement of the relation of the human brain to epilepsy as a start point, the historical development of the field is traced. This encompasses Thomas Willis and his nascent descriptions of the limbic system, the philosophical and alchemical strides of the Enlightenment, and the motivations behind the Romantic era attempts to understand the brain. It then follows the growth of the field through the turn of the twentieth century, in spite of the prominence of psychoanalysis and the idea of the brainless mind, and finally the understanding of the ‘integrated action’ of the body and nervous system, which led to the integration of psychiatry and neurology, allowing for the first neuropsychiatric examinations of epilepsy.

Sign in / Sign up

Export Citation Format

Share Document