scholarly journals Locus Coeruleus tracking of prediction errors optimises cognitive flexibility: an Active Inference model

2018 ◽  
Author(s):  
Anna C Sales ◽  
Karl J. Friston ◽  
Matthew W. Jones ◽  
Anthony E. Pickering ◽  
Rosalyn J. Moran
Keyword(s):  
Locus Coeruleus ◽  
Prediction Error ◽  
Internal Models ◽  
Prediction Errors ◽  
Active Inference ◽  
Behavioural Flexibility ◽  
Action Prediction ◽  
Belief Updating ◽  
State Action ◽  
The Brain

AbstractThe locus coeruleus (LC) in the pons is the major source of noradrenaline (NA) in the brain. Two modes of LC firing have been associated with distinct cognitive states: changes in tonic rates of firing are correlated with global levels of arousal and behavioural flexibility, whilst phasic LC responses are evoked by salient stimuli. Here, we unify these two modes of firing by modelling the response of the LC as a correlate of a prediction error when inferring states for action planning under Active Inference (AI).We simulate a classic Go/No-go reward learning task and a three-arm foraging task and show that, if LC activity is considered to reflect the magnitude of high level ‘state-action’ prediction errors, then both tonic and phasic modes of firing are emergent features of belief updating. We also demonstrate that when contingencies change, AI agents can update their internal models more quickly by feeding back this state-action prediction error – reflected in LC firing and noradrenaline release – to optimise learning rate, enabling large adjustments over short timescales. We propose that such prediction errors are mediated by cortico-LC connections, whilst ascending input from LC to cortex modulates belief updating in anterior cingulate cortex (ACC).In short, we characterise the LC/ NA system within a general theory of brain function. In doing so, we show that contrasting, behaviour-dependent firing patterns are an emergent property of the LC’s crucial role in translating prediction errors into an optimal mediation between plasticity and stability.Author SummaryThe brain uses sensory information to build internal models and make predictions about the world. When errors of prediction occur, models must be updated to ensure desired outcomes are still achieved. Neuromodulator chemicals provide a possible pathway for triggering such changes in brain state. One such neuromodulator, noradrenaline, originates predominantly from a cluster of neurons in the brainstem – the locus coeruleus (LC) – and plays a key role in behaviour, for instance, in determining the balance between exploiting or exploring the environment.Here we use Active Inference (AI), a mathematical model of perception and action, to formally describe LC function. We propose that LC activity is triggered by errors in prediction and that the subsequent release of noradrenaline alters the rate of learning about the environment. Biologically, this describes an LC-cortex feedback loop promoting behavioural flexibility in times of uncertainty. We model LC output as a simulated animal performs two tasks known to elicit archetypal responses. We find that experimentally observed ‘phasic’ and ‘tonic’ patterns of LC activity emerge naturally, and that modulation of learning rates improves task performance. This provides a simple, unified computational account of noradrenergic computational function within a general model of behaviour.

scholarly journals Prefrontal solution to the bias-variance tradeoff during reinforcement learning

2020 ◽  
Author(s):  
Dongjae Kim ◽  
Jaeseung Jeong ◽  
Sang Wan Lee
Keyword(s):  
Adaptive Control ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Brain Regions ◽  
Decision Task ◽  
Prediction Errors ◽  
Model Based ◽  
Model Free ◽  
Bias Variance ◽  
The Brain

AbstractThe goal of learning is to maximize future rewards by minimizing prediction errors. Evidence have shown that the brain achieves this by combining model-based and model-free learning. However, the prediction error minimization is challenged by a bias-variance tradeoff, which imposes constraints on each strategy’s performance. We provide new theoretical insight into how this tradeoff can be resolved through the adaptive control of model-based and model-free learning. The theory predicts the baseline correction for prediction error reduces the lower bound of the bias–variance error by factoring out irreducible noise. Using a Markov decision task with context changes, we showed behavioral evidence of adaptive control. Model-based behavioral analyses show that the prediction error baseline signals context changes to improve adaptability. Critically, the neural results support this view, demonstrating multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free learning.One sentence summaryA theoretical, behavioral, computational, and neural account of how the brain resolves the bias-variance tradeoff during reinforcement learning is described.

scholarly journals Locus Coeruleus tracking of prediction errors optimises cognitive flexibility: An Active Inference model

2019 ◽  
Vol 15 (1) ◽  
pp. e1006267 ◽  
Author(s):  
Anna C. Sales ◽  
Karl J. Friston ◽  
Matthew W. Jones ◽  
Anthony E. Pickering ◽  
Rosalyn J. Moran
Keyword(s):  
Locus Coeruleus ◽  
Cognitive Flexibility ◽  
Prediction Errors ◽  
Active Inference ◽  
Inference Model

scholarly journals Backwards is the way forward: Feedback in the cortical hierarchy predicts the expected future

2013 ◽  
Vol 36 (3) ◽  
pp. 221-221 ◽  
Author(s):  
Lars Muckli ◽  
Lucy S. Petro ◽  
Fraser W. Smith
Keyword(s):  
Predictive Coding ◽  
Internal Models ◽  
Prediction Errors ◽  
Conceptual Level ◽  
Cortical Hierarchy ◽  
The Brain ◽  
Descriptive Level ◽  
Perception Action ◽  
The Way

AbstractClark offers a powerful description of the brain as a prediction machine, which offers progress on two distinct levels. First, on an abstract conceptual level, it provides a unifying framework for perception, action, and cognition (including subdivisions such as attention, expectation, and imagination). Second, hierarchical prediction offers progress on a concrete descriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction errors, and internal models).

Active inference and cognitive-emotional interactions in the brain

2015 ◽  
Vol 38 ◽  
Author(s):  
Giovanni Pezzulo ◽  
Laura Barca ◽  
Karl J. Friston
Keyword(s):  
Prediction Errors ◽  
Active Inference ◽  
Bayesian Fusion ◽  
The Brain

AbstractAll organisms must integrate cognition, emotion, and motivation to guide action toward valuable (goal) states, as described by active inference. Within this framework, cognition, emotion, and motivation interact through the (Bayesian) fusion of exteroceptive, proprioceptive, and interoceptive signals, the precision-weighting of prediction errors, and the “affective tuning” of neuronal representations. Crucially, misregulation of these processes may have profound psychopathological consequences.

scholarly journals Precision-weighting of superior frontal cortex unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis

2019 ◽  
Author(s):  
J. Haarsma ◽  
P.C. Fletcher ◽  
J.D. Griffin ◽  
H.J. Taverne ◽  
H. Ziauddeen ◽  
...  
Keyword(s):  
Prediction Error ◽  
Computational Modelling ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Healthy Individuals ◽  
Cortical Function ◽  
Dopaminergic Modulation ◽  
Hierarchical Bayesian Inference ◽  
The Brain ◽  
Superior Frontal Cortex

AbstractRecent theories of cortical function construe the brain as performing hierarchical Bayesian inference. According to these theories, the precision of cortical unsigned prediction error (i.e., surprise) signals plays a key role in learning and decision-making, to be controlled by dopamine, and to contribute to the pathogenesis of psychosis. To test these hypotheses, we studied learning with variable outcome-precision in healthy individuals after dopaminergic modulation and in patients with early psychosis. Behavioural computational modelling indicated that precision-weighting of unsigned prediction errors benefits learning in health, and is impaired in psychosis. FMRI revealed coding of unsigned prediction errors relative to their precision in bilateral superior frontal gyri and dorsal anterior cingulate, which was perturbed by dopaminergic modulation, impaired in psychosis, and associated with task performance and schizotypy. We conclude that precision-weighting of cortical prediction error signals is a key mechanism through which dopamine modulates inference and contributes to the pathogenesis of psychosis.

scholarly journals Adaptive coding of pain prediction error in the anterior insula

2021 ◽  
Author(s):  
Robert Hoskin ◽  
Deborah Talmi
Keyword(s):  
Prediction Error ◽  
Contextual Information ◽  
Neural Mechanism ◽  
Axiomatic Approach ◽  
Anterior Insula ◽  
Prediction Errors ◽  
Adaptive Coding ◽  
Pain Prediction ◽  
The Brain ◽  
Behavioural Evidence

Background: To reduce the computational demands of the task of determining values, the brain is thought to engage in adaptive coding, where the sensitivity of some neurons to value is modulated by contextual information. There is good behavioural evidence that pain is coded adaptively, but controversy regarding the underlying neural mechanism. Additionally, there is evidence that reward prediction errors are coded adaptively, but no parallel evidence regarding pain prediction errors. Methods: We tested the hypothesis that pain prediction errors are coded adaptively by scanning 19 healthy adults with fMRI while they performed a cued pain task. Our analysis followed an axiomatic approach. Results: We found that the left anterior insula was the only region which was sensitive both to predicted pain magnitudes and the unexpectedness of pain delivery, but not to the magnitude of delivered pain. Conclusions: This pattern suggests that the left anterior insula is part of a neural mechanism that serves the adaptive prediction error of pain.

scholarly journals Prediction error induced motor contagions in human behaviors

eLife ◽  
2018 ◽  
Vol 7 ◽  
Author(s):  
Tsuyoshi Ikegami ◽  
Gowrishankar Ganesh ◽  
Tatsuya Takeuchi ◽  
Hiroki Nakamoto
Keyword(s):  
Prediction Error ◽  
Action Observation ◽  
Whole Body ◽  
Prediction Errors ◽  
Action Prediction ◽  
Human Actions ◽  
Human Behaviors ◽  
Motor Contagion ◽  
Arm Reaching

Motor contagions refer to implicit effects on one's actions induced by observed actions. Motor contagions are believed to be induced simply by action observation and cause an observer's action to become similar to the action observed. In contrast, here we report a new motor contagion that is induced only when the observation is accompanied by prediction errors - differences between actions one observes and those he/she predicts or expects. In two experiments, one on whole-body baseball pitching and another on simple arm reaching, we show that the observation of the same action induces distinct motor contagions, depending on whether prediction errors are present or not. In the absence of prediction errors, as in previous reports, participants' actions changed to become similar to the observed action, while in the presence of prediction errors, their actions changed to diverge away from it, suggesting distinct effects of action observation and action prediction on human actions.

scholarly journals Predictability Changes What We Remember in Familiar Temporal Contexts

2020 ◽  
Vol 32 (1) ◽  
pp. 124-140 ◽  
Author(s):  
Hyojeong Kim ◽  
Margaret L. Schlichting ◽  
Alison R. Preston ◽  
Jarrod A. Lewis-Peacock
Keyword(s):  
Human Brain ◽  
Prediction Error ◽  
Fmri Data ◽  
Prediction Errors ◽  
Associative Memories ◽  
The Past ◽  
Item Level ◽  
Abstract Level ◽  
The Brain

The human brain constantly anticipates the future based on memories of the past. Encountering a familiar situation reactivates memory of previous encounters, which can trigger a prediction of what comes next to facilitate responsiveness. However, a prediction error can lead to pruning of the offending memory, a process that weakens its representation in the brain and leads to forgetting. Our goal in this study was to evaluate whether memories are spared from such pruning in situations that allow for accurate predictions at the categorical level, despite prediction errors at the item level. Participants viewed a sequence of objects, some of which reappeared multiple times (“cues”), followed always by novel items. Half of the cues were followed by new items from different (unpredictable) categories, while others were followed by new items from a single (predictable) category. Pattern classification of fMRI data was used to identify category-specific predictions after each cue. Pruning was observed only in unpredictable contexts, while encoding of new items was less robust in predictable contexts. These findings demonstrate that how associative memories are updated is influenced by the reliability of abstract-level predictions in familiar contexts.

scholarly journals Learning prediction error neurons in a canonical interneuron circuit

eLife ◽  
2020 ◽  
Vol 9 ◽  
Author(s):  
Loreen Hertäg ◽  
Henning Sprekeler
Keyword(s):  
Visual Cortex ◽  
Primary Visual Cortex ◽  
Prediction Error ◽  
Sensory Information ◽  
Prediction Errors ◽  
Negative Prediction ◽  
Inhibitory Plasticity ◽  
The Brain ◽  
Shed Light ◽  
Model Circuit

Sensory systems constantly compare external sensory information with internally generated predictions. While neural hallmarks of prediction errors have been found throughout the brain, the circuit-level mechanisms that underlie their computation are still largely unknown. Here, we show that a well-orchestrated interplay of three interneuron types shapes the development and refinement of negative prediction-error neurons in a computational model of mouse primary visual cortex. By balancing excitation and inhibition in multiple pathways, experience-dependent inhibitory plasticity can generate different variants of prediction-error circuits, which can be distinguished by simulated optogenetic experiments. The experience-dependence of the model circuit is consistent with that of negative prediction-error circuits in layer 2/3 of mouse primary visual cortex. Our model makes a range of testable predictions that may shed light on the circuitry underlying the neural computation of prediction errors.

scholarly journals The predictive brain in action: Involuntary actions reduce body prediction errors

2020 ◽  
Author(s):  
Pablo Lanillos ◽  
Sae Franklin ◽  
David W. Franklin
Keyword(s):  
Prediction Error ◽  
Sensory Information ◽  
The Body ◽  
Predictive Processing ◽  
Body Ownership ◽  
Prediction Errors ◽  
Uncertain Information ◽  
Energy Principle ◽  
Virtual Hand ◽  
The Brain

AbstractThe perception of our body in space is flexible and manipulable. The predictive brain hypothesis explains this malleability as a consequence of the interplay between incoming sensory information and our body expectations. However, given the interaction between perception and action, we might also expect that actions would arise due to prediction errors, especially in conflicting situations. Here we describe a computational model, based on the free-energy principle, that forecasts involuntary movements in sensorimotor conflicts. We experimentally confirm those predictions in humans by means of a virtual reality rubber-hand illusion. Participants generated movements (forces) towards the virtual hand, regardless of its location with respect to the real arm, with little to no forces produced when the virtual hand overlaid their physical hand. The congruency of our model predictions and human observations shows that the brain-body is generating actions to reduce the prediction error between the expected arm location and the new visual arm. This observed unconscious mechanism is an empirical validation of the perception-action duality in body adaptation to uncertain situations and evidence of the active component of predictive processing.Author SummaryHumans’ capacity to perceive and control their body in space is central in awareness, adaptation and safe interaction. From low-level body perception to body-ownership, discovering how the brain represents the body and generates actions is of major importance for cognitive science and also for robotics and artificial intelligence. The present study shows that humans move their body to match the expected location according to other (visual) sensory input, which corresponds to reducing the prediction error. This means that the brain adapts to conflicting or uncertain information from the senses by unconsciously acting in the world.

Sign in / Sign up

Export Citation Format

Share Document