scholarly journals Precision-weighting of superior frontal cortex unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis

2019 ◽  
Author(s):  
J. Haarsma ◽  
P.C. Fletcher ◽  
J.D. Griffin ◽  
H.J. Taverne ◽  
H. Ziauddeen ◽  
...  
Keyword(s):  
Prediction Error ◽  
Computational Modelling ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Healthy Individuals ◽  
Cortical Function ◽  
Dopaminergic Modulation ◽  
Hierarchical Bayesian Inference ◽  
The Brain ◽  
Superior Frontal Cortex

AbstractRecent theories of cortical function construe the brain as performing hierarchical Bayesian inference. According to these theories, the precision of cortical unsigned prediction error (i.e., surprise) signals plays a key role in learning and decision-making, to be controlled by dopamine, and to contribute to the pathogenesis of psychosis. To test these hypotheses, we studied learning with variable outcome-precision in healthy individuals after dopaminergic modulation and in patients with early psychosis. Behavioural computational modelling indicated that precision-weighting of unsigned prediction errors benefits learning in health, and is impaired in psychosis. FMRI revealed coding of unsigned prediction errors relative to their precision in bilateral superior frontal gyri and dorsal anterior cingulate, which was perturbed by dopaminergic modulation, impaired in psychosis, and associated with task performance and schizotypy. We conclude that precision-weighting of cortical prediction error signals is a key mechanism through which dopamine modulates inference and contributes to the pathogenesis of psychosis.

scholarly journals Prefrontal solution to the bias-variance tradeoff during reinforcement learning

2020 ◽  
Author(s):  
Dongjae Kim ◽  
Jaeseung Jeong ◽  
Sang Wan Lee
Keyword(s):  
Adaptive Control ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Brain Regions ◽  
Decision Task ◽  
Prediction Errors ◽  
Model Based ◽  
Model Free ◽  
Bias Variance ◽  
The Brain

AbstractThe goal of learning is to maximize future rewards by minimizing prediction errors. Evidence have shown that the brain achieves this by combining model-based and model-free learning. However, the prediction error minimization is challenged by a bias-variance tradeoff, which imposes constraints on each strategy’s performance. We provide new theoretical insight into how this tradeoff can be resolved through the adaptive control of model-based and model-free learning. The theory predicts the baseline correction for prediction error reduces the lower bound of the bias–variance error by factoring out irreducible noise. Using a Markov decision task with context changes, we showed behavioral evidence of adaptive control. Model-based behavioral analyses show that the prediction error baseline signals context changes to improve adaptability. Critically, the neural results support this view, demonstrating multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free learning.One sentence summaryA theoretical, behavioral, computational, and neural account of how the brain resolves the bias-variance tradeoff during reinforcement learning is described.

scholarly journals Adaptive coding of pain prediction error in the anterior insula

2021 ◽  
Author(s):  
Robert Hoskin ◽  
Deborah Talmi
Keyword(s):  
Prediction Error ◽  
Contextual Information ◽  
Neural Mechanism ◽  
Axiomatic Approach ◽  
Anterior Insula ◽  
Prediction Errors ◽  
Adaptive Coding ◽  
Pain Prediction ◽  
The Brain ◽  
Behavioural Evidence

Background: To reduce the computational demands of the task of determining values, the brain is thought to engage in adaptive coding, where the sensitivity of some neurons to value is modulated by contextual information. There is good behavioural evidence that pain is coded adaptively, but controversy regarding the underlying neural mechanism. Additionally, there is evidence that reward prediction errors are coded adaptively, but no parallel evidence regarding pain prediction errors. Methods: We tested the hypothesis that pain prediction errors are coded adaptively by scanning 19 healthy adults with fMRI while they performed a cued pain task. Our analysis followed an axiomatic approach. Results: We found that the left anterior insula was the only region which was sensitive both to predicted pain magnitudes and the unexpectedness of pain delivery, but not to the magnitude of delivered pain. Conclusions: This pattern suggests that the left anterior insula is part of a neural mechanism that serves the adaptive prediction error of pain.

scholarly journals Locus Coeruleus tracking of prediction errors optimises cognitive flexibility: an Active Inference model

2018 ◽  
Author(s):  
Anna C Sales ◽  
Karl J. Friston ◽  
Matthew W. Jones ◽  
Anthony E. Pickering ◽  
Rosalyn J. Moran
Keyword(s):  
Locus Coeruleus ◽  
Prediction Error ◽  
Internal Models ◽  
Prediction Errors ◽  
Active Inference ◽  
Behavioural Flexibility ◽  
Action Prediction ◽  
Belief Updating ◽  
State Action ◽  
The Brain

AbstractThe locus coeruleus (LC) in the pons is the major source of noradrenaline (NA) in the brain. Two modes of LC firing have been associated with distinct cognitive states: changes in tonic rates of firing are correlated with global levels of arousal and behavioural flexibility, whilst phasic LC responses are evoked by salient stimuli. Here, we unify these two modes of firing by modelling the response of the LC as a correlate of a prediction error when inferring states for action planning under Active Inference (AI).We simulate a classic Go/No-go reward learning task and a three-arm foraging task and show that, if LC activity is considered to reflect the magnitude of high level ‘state-action’ prediction errors, then both tonic and phasic modes of firing are emergent features of belief updating. We also demonstrate that when contingencies change, AI agents can update their internal models more quickly by feeding back this state-action prediction error – reflected in LC firing and noradrenaline release – to optimise learning rate, enabling large adjustments over short timescales. We propose that such prediction errors are mediated by cortico-LC connections, whilst ascending input from LC to cortex modulates belief updating in anterior cingulate cortex (ACC).In short, we characterise the LC/ NA system within a general theory of brain function. In doing so, we show that contrasting, behaviour-dependent firing patterns are an emergent property of the LC’s crucial role in translating prediction errors into an optimal mediation between plasticity and stability.Author SummaryThe brain uses sensory information to build internal models and make predictions about the world. When errors of prediction occur, models must be updated to ensure desired outcomes are still achieved. Neuromodulator chemicals provide a possible pathway for triggering such changes in brain state. One such neuromodulator, noradrenaline, originates predominantly from a cluster of neurons in the brainstem – the locus coeruleus (LC) – and plays a key role in behaviour, for instance, in determining the balance between exploiting or exploring the environment.Here we use Active Inference (AI), a mathematical model of perception and action, to formally describe LC function. We propose that LC activity is triggered by errors in prediction and that the subsequent release of noradrenaline alters the rate of learning about the environment. Biologically, this describes an LC-cortex feedback loop promoting behavioural flexibility in times of uncertainty. We model LC output as a simulated animal performs two tasks known to elicit archetypal responses. We find that experimentally observed ‘phasic’ and ‘tonic’ patterns of LC activity emerge naturally, and that modulation of learning rates improves task performance. This provides a simple, unified computational account of noradrenergic computational function within a general model of behaviour.

scholarly journals Predictability Changes What We Remember in Familiar Temporal Contexts

2020 ◽  
Vol 32 (1) ◽  
pp. 124-140 ◽  
Author(s):  
Hyojeong Kim ◽  
Margaret L. Schlichting ◽  
Alison R. Preston ◽  
Jarrod A. Lewis-Peacock
Keyword(s):  
Human Brain ◽  
Prediction Error ◽  
Fmri Data ◽  
Prediction Errors ◽  
Associative Memories ◽  
The Past ◽  
Item Level ◽  
Abstract Level ◽  
The Brain

The human brain constantly anticipates the future based on memories of the past. Encountering a familiar situation reactivates memory of previous encounters, which can trigger a prediction of what comes next to facilitate responsiveness. However, a prediction error can lead to pruning of the offending memory, a process that weakens its representation in the brain and leads to forgetting. Our goal in this study was to evaluate whether memories are spared from such pruning in situations that allow for accurate predictions at the categorical level, despite prediction errors at the item level. Participants viewed a sequence of objects, some of which reappeared multiple times (“cues”), followed always by novel items. Half of the cues were followed by new items from different (unpredictable) categories, while others were followed by new items from a single (predictable) category. Pattern classification of fMRI data was used to identify category-specific predictions after each cue. Pruning was observed only in unpredictable contexts, while encoding of new items was less robust in predictable contexts. These findings demonstrate that how associative memories are updated is influenced by the reliability of abstract-level predictions in familiar contexts.

scholarly journals Neural Representation of Costs and Rewards in Decision Making

2021 ◽  
Vol 11 (8) ◽  
pp. 1096
Author(s):  
Yixuan Chen
Keyword(s):  
Decision Making ◽  
Anterior Cingulate Cortex ◽  
Orbitofrontal Cortex ◽  
Internal State ◽  
Cingulate Cortex ◽  
Brain Regions ◽  
Anterior Cingulate ◽  
Neural Representation ◽  
Prediction Errors ◽  
The Brain

Decision making is crucial for animal survival because the choices they make based on their current situation could influence their future rewards and could have potential costs. This review summarises recent developments in decision making, discusses how rewards and costs could be encoded in the brain, and how different options are compared such that the most optimal one is chosen. The reward and cost are mainly encoded by the forebrain structures (e.g., anterior cingulate cortex, orbitofrontal cortex), and their value is updated through learning. The recent development on dopamine and the lateral habenula’s role in reporting prediction errors and instructing learning will be emphasised. The importance of dopamine in powering the choice and accounting for the internal state will also be discussed. While the orbitofrontal cortex is the place where the state values are stored, the anterior cingulate cortex is more important when the environment is volatile. All of these structures compare different attributes of the task simultaneously, and the local competition of different neuronal networks allows for the selection of the most appropriate one. Therefore, the total value of the task is not encoded as a scalar quantity in the brain but, instead, as an emergent phenomenon, arising from the computation at different brain regions.

scholarly journals Learning prediction error neurons in a canonical interneuron circuit

eLife ◽  
2020 ◽  
Vol 9 ◽  
Author(s):  
Loreen Hertäg ◽  
Henning Sprekeler
Keyword(s):  
Visual Cortex ◽  
Primary Visual Cortex ◽  
Prediction Error ◽  
Sensory Information ◽  
Prediction Errors ◽  
Negative Prediction ◽  
Inhibitory Plasticity ◽  
The Brain ◽  
Shed Light ◽  
Model Circuit

Sensory systems constantly compare external sensory information with internally generated predictions. While neural hallmarks of prediction errors have been found throughout the brain, the circuit-level mechanisms that underlie their computation are still largely unknown. Here, we show that a well-orchestrated interplay of three interneuron types shapes the development and refinement of negative prediction-error neurons in a computational model of mouse primary visual cortex. By balancing excitation and inhibition in multiple pathways, experience-dependent inhibitory plasticity can generate different variants of prediction-error circuits, which can be distinguished by simulated optogenetic experiments. The experience-dependence of the model circuit is consistent with that of negative prediction-error circuits in layer 2/3 of mouse primary visual cortex. Our model makes a range of testable predictions that may shed light on the circuitry underlying the neural computation of prediction errors.

scholarly journals The predictive brain in action: Involuntary actions reduce body prediction errors

2020 ◽  
Author(s):  
Pablo Lanillos ◽  
Sae Franklin ◽  
David W. Franklin
Keyword(s):  
Prediction Error ◽  
Sensory Information ◽  
The Body ◽  
Predictive Processing ◽  
Body Ownership ◽  
Prediction Errors ◽  
Uncertain Information ◽  
Energy Principle ◽  
Virtual Hand ◽  
The Brain

AbstractThe perception of our body in space is flexible and manipulable. The predictive brain hypothesis explains this malleability as a consequence of the interplay between incoming sensory information and our body expectations. However, given the interaction between perception and action, we might also expect that actions would arise due to prediction errors, especially in conflicting situations. Here we describe a computational model, based on the free-energy principle, that forecasts involuntary movements in sensorimotor conflicts. We experimentally confirm those predictions in humans by means of a virtual reality rubber-hand illusion. Participants generated movements (forces) towards the virtual hand, regardless of its location with respect to the real arm, with little to no forces produced when the virtual hand overlaid their physical hand. The congruency of our model predictions and human observations shows that the brain-body is generating actions to reduce the prediction error between the expected arm location and the new visual arm. This observed unconscious mechanism is an empirical validation of the perception-action duality in body adaptation to uncertain situations and evidence of the active component of predictive processing.Author SummaryHumans’ capacity to perceive and control their body in space is central in awareness, adaptation and safe interaction. From low-level body perception to body-ownership, discovering how the brain represents the body and generates actions is of major importance for cognitive science and also for robotics and artificial intelligence. The present study shows that humans move their body to match the expected location according to other (visual) sensory input, which corresponds to reducing the prediction error. This means that the brain adapts to conflicting or uncertain information from the senses by unconsciously acting in the world.

Experienced Wholeness

2018 ◽  
Author(s):  
Wanja Wiese
Keyword(s):  
Prediction Error ◽  
Predictive Processing ◽  
Second Step ◽  
Error Minimization ◽  
Gestalt Theory ◽  
Bodily Experiences ◽  
Prediction Error Minimization ◽  
Hierarchical Bayesian Inference ◽  
The Brain ◽  
Phenomenal Unity

The unity of the experienced world and the experienced self have puzzled humanity for centuries. How can we understand this and related types of phenomenal (i.e., experienced) unity? This book develops an interdisciplinary account of phenomenal unity. It focuses on examples of experienced wholes such as perceived objects (chairs and tables, but also groups of objects), bodily experiences, successions of events, and the attentional structure of consciousness. As a first step, the book investigates how the unity of consciousness can be characterized phenomenologically: what is it like to experience wholes, what is the experiential contribution of phenomenal unity? This raises conceptual and empirical questions. In addressing these questions, connections are drawn to phenomenological accounts and research on Gestalt theory. As a second step, the book suggests how phenomenal unity can be analyzed computationally, by drawing on concepts and ideas of the framework of predictive processing. The result is a conceptual framework, as well as an interdisciplinary account of phenomenal unity: the regularity account of phenomenal unity. According to this account, experienced wholes correspond to a hierarchy of connecting regularities. The brain tracks these regularities by hierarchical prediction error minimization, which approximates hierarchical Bayesian inference.

scholarly journals Learning prediction error neurons in a canonical interneuron circuit

2020 ◽  
Author(s):  
Loreen Hertäg ◽  
Henning Sprekeler
Keyword(s):  
Visual Cortex ◽  
Primary Visual Cortex ◽  
Prediction Error ◽  
Sensory Information ◽  
Prediction Errors ◽  
Negative Prediction ◽  
Inhibitory Plasticity ◽  
The Brain ◽  
Shed Light ◽  
Model Circuit

AbstractSensory systems constantly compare external sensory information with internally generated predictions. While neural hallmarks of prediction errors have been found throughout the brain, the circuit-level mechanisms that underlie their computation are still largely unknown. Here, we show that a well-orchestrated interplay of three interneuron types shapes the development and refinement of negative prediction-error neurons in a computational model of mouse primary visual cortex. By balancing excitation and inhibition in multiple pathways, experience-dependent inhibitory plasticity can generate different variants of prediction-error circuits, which can be distinguished by simulated optogenetic experiments. The experience-dependence of the model circuit is consistent with that of negative prediction-error circuits in layer 2/3 of mouse primary visual cortex. Our model makes a range of testable predictions that may shed light on the circuitry underlying the neural computation of prediction errors.

scholarly journals Offline Replay Supports Planning: fMRI Evidence from Reward Revaluation

2017 ◽  
Author(s):  
Ida Momennejad ◽  
A. Ross Otto ◽  
Nathaniel D. Daw ◽  
Kenneth A. Norman
Keyword(s):  
Machine Learning ◽  
Computational Cost ◽  
Decision Time ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Activity Increase ◽  
Planning Methods ◽  
Goal Directed Behavior ◽  
The Brain

AbstractMaking decisions in sequentially structured tasks requires integrating distally acquired information. The extensive computational cost of such integration challenges planning methods that integrate online, at decision time. Furthermore, it remains unclear whether “offline” integration during replay supports planning, and if so which memories should be replayed. Inspired by machine learning, we propose that (a) offline replay of trajectories facilitates integrating representations that guide decisions, and (b) unsigned prediction errors (uncertainty) trigger such integrative replay. We designed a 2-step revaluation task for fMRI, whereby participants needed to integrate changes in rewards with past knowledge to optimally replan decisions. As predicted, we found that (a) multi-voxel pattern evidence for off-task replay predicts subsequent replanning; (b) neural sensitivity to uncertainty predicts subsequent replay and replanning; (c) off-task hippocampus and anterior cingulate activity increase when revaluation is required. These findings elucidate how the brain leverages offline mechanisms in planning and goal-directed behavior under uncertainty.

Sign in / Sign up

Export Citation Format

Share Document