scholarly journals Adaptive coding of pain prediction error in the anterior insula

2021 ◽  
Author(s):  
Robert Hoskin ◽  
Deborah Talmi
Keyword(s):  
Prediction Error ◽  
Contextual Information ◽  
Neural Mechanism ◽  
Axiomatic Approach ◽  
Anterior Insula ◽  
Prediction Errors ◽  
Adaptive Coding ◽  
Pain Prediction ◽  
The Brain ◽  
Behavioural Evidence

Background: To reduce the computational demands of the task of determining values, the brain is thought to engage in adaptive coding, where the sensitivity of some neurons to value is modulated by contextual information. There is good behavioural evidence that pain is coded adaptively, but controversy regarding the underlying neural mechanism. Additionally, there is evidence that reward prediction errors are coded adaptively, but no parallel evidence regarding pain prediction errors. Methods: We tested the hypothesis that pain prediction errors are coded adaptively by scanning 19 healthy adults with fMRI while they performed a cued pain task. Our analysis followed an axiomatic approach. Results: We found that the left anterior insula was the only region which was sensitive both to predicted pain magnitudes and the unexpectedness of pain delivery, but not to the magnitude of delivered pain. Conclusions: This pattern suggests that the left anterior insula is part of a neural mechanism that serves the adaptive prediction error of pain.

scholarly journals Prefrontal solution to the bias-variance tradeoff during reinforcement learning

2020 ◽  
Author(s):  
Dongjae Kim ◽  
Jaeseung Jeong ◽  
Sang Wan Lee
Keyword(s):  
Adaptive Control ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Brain Regions ◽  
Decision Task ◽  
Prediction Errors ◽  
Model Based ◽  
Model Free ◽  
Bias Variance ◽  
The Brain

AbstractThe goal of learning is to maximize future rewards by minimizing prediction errors. Evidence have shown that the brain achieves this by combining model-based and model-free learning. However, the prediction error minimization is challenged by a bias-variance tradeoff, which imposes constraints on each strategy’s performance. We provide new theoretical insight into how this tradeoff can be resolved through the adaptive control of model-based and model-free learning. The theory predicts the baseline correction for prediction error reduces the lower bound of the bias–variance error by factoring out irreducible noise. Using a Markov decision task with context changes, we showed behavioral evidence of adaptive control. Model-based behavioral analyses show that the prediction error baseline signals context changes to improve adaptability. Critically, the neural results support this view, demonstrating multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free learning.One sentence summaryA theoretical, behavioral, computational, and neural account of how the brain resolves the bias-variance tradeoff during reinforcement learning is described.

scholarly journals Striatal structure and function predict individual biases in learning to avoid pain

2016 ◽  
Vol 113 (17) ◽  
pp. 4812-4817 ◽  
Author(s):  
Eran Eldar ◽  
Tobias U. Hauser ◽  
Peter Dayan ◽  
Raymond J. Dolan
Keyword(s):  
Individual Differences ◽  
Prediction Error ◽  
Outcome Prediction ◽  
Structure And Function ◽  
Pain Modulation ◽  
Prediction Errors ◽  
Learning Biases ◽  
Individual Traits ◽  
And Function ◽  
Pain Prediction

Pain is an elemental inducer of avoidance. Here, we demonstrate that people differ in how they learn to avoid pain, with some individuals refraining from actions that resulted in painful outcomes, whereas others favor actions that helped prevent pain. These individual biases were best explained by differences in learning from outcome prediction errors and were associated with distinct forms of striatal responses to painful outcomes. Specifically, striatal responses to pain were modulated in a manner consistent with an aversive prediction error in individuals who learned predominantly from pain, whereas in individuals who learned predominantly from success in preventing pain, modulation was consistent with an appetitive prediction error. In contrast, striatal responses to success in preventing pain were consistent with an appetitive prediction error in both groups. Furthermore, variation in striatal structure, encompassing the region where pain prediction errors were expressed, predicted participants’ predominant mode of learning, suggesting the observed learning biases may reflect stable individual traits. These results reveal functional and structural neural components underlying individual differences in avoidance learning, which may be important contributors to psychiatric disorders involving pathological harm avoidance behavior.

scholarly journals Precision-weighting of superior frontal cortex unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis

2019 ◽  
Author(s):  
J. Haarsma ◽  
P.C. Fletcher ◽  
J.D. Griffin ◽  
H.J. Taverne ◽  
H. Ziauddeen ◽  
...  
Keyword(s):  
Prediction Error ◽  
Computational Modelling ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Healthy Individuals ◽  
Cortical Function ◽  
Dopaminergic Modulation ◽  
Hierarchical Bayesian Inference ◽  
The Brain ◽  
Superior Frontal Cortex

AbstractRecent theories of cortical function construe the brain as performing hierarchical Bayesian inference. According to these theories, the precision of cortical unsigned prediction error (i.e., surprise) signals plays a key role in learning and decision-making, to be controlled by dopamine, and to contribute to the pathogenesis of psychosis. To test these hypotheses, we studied learning with variable outcome-precision in healthy individuals after dopaminergic modulation and in patients with early psychosis. Behavioural computational modelling indicated that precision-weighting of unsigned prediction errors benefits learning in health, and is impaired in psychosis. FMRI revealed coding of unsigned prediction errors relative to their precision in bilateral superior frontal gyri and dorsal anterior cingulate, which was perturbed by dopaminergic modulation, impaired in psychosis, and associated with task performance and schizotypy. We conclude that precision-weighting of cortical prediction error signals is a key mechanism through which dopamine modulates inference and contributes to the pathogenesis of psychosis.

scholarly journals Locus Coeruleus tracking of prediction errors optimises cognitive flexibility: an Active Inference model

2018 ◽  
Author(s):  
Anna C Sales ◽  
Karl J. Friston ◽  
Matthew W. Jones ◽  
Anthony E. Pickering ◽  
Rosalyn J. Moran
Keyword(s):  
Locus Coeruleus ◽  
Prediction Error ◽  
Internal Models ◽  
Prediction Errors ◽  
Active Inference ◽  
Behavioural Flexibility ◽  
Action Prediction ◽  
Belief Updating ◽  
State Action ◽  
The Brain

AbstractThe locus coeruleus (LC) in the pons is the major source of noradrenaline (NA) in the brain. Two modes of LC firing have been associated with distinct cognitive states: changes in tonic rates of firing are correlated with global levels of arousal and behavioural flexibility, whilst phasic LC responses are evoked by salient stimuli. Here, we unify these two modes of firing by modelling the response of the LC as a correlate of a prediction error when inferring states for action planning under Active Inference (AI).We simulate a classic Go/No-go reward learning task and a three-arm foraging task and show that, if LC activity is considered to reflect the magnitude of high level ‘state-action’ prediction errors, then both tonic and phasic modes of firing are emergent features of belief updating. We also demonstrate that when contingencies change, AI agents can update their internal models more quickly by feeding back this state-action prediction error – reflected in LC firing and noradrenaline release – to optimise learning rate, enabling large adjustments over short timescales. We propose that such prediction errors are mediated by cortico-LC connections, whilst ascending input from LC to cortex modulates belief updating in anterior cingulate cortex (ACC).In short, we characterise the LC/ NA system within a general theory of brain function. In doing so, we show that contrasting, behaviour-dependent firing patterns are an emergent property of the LC’s crucial role in translating prediction errors into an optimal mediation between plasticity and stability.Author SummaryThe brain uses sensory information to build internal models and make predictions about the world. When errors of prediction occur, models must be updated to ensure desired outcomes are still achieved. Neuromodulator chemicals provide a possible pathway for triggering such changes in brain state. One such neuromodulator, noradrenaline, originates predominantly from a cluster of neurons in the brainstem – the locus coeruleus (LC) – and plays a key role in behaviour, for instance, in determining the balance between exploiting or exploring the environment.Here we use Active Inference (AI), a mathematical model of perception and action, to formally describe LC function. We propose that LC activity is triggered by errors in prediction and that the subsequent release of noradrenaline alters the rate of learning about the environment. Biologically, this describes an LC-cortex feedback loop promoting behavioural flexibility in times of uncertainty. We model LC output as a simulated animal performs two tasks known to elicit archetypal responses. We find that experimentally observed ‘phasic’ and ‘tonic’ patterns of LC activity emerge naturally, and that modulation of learning rates improves task performance. This provides a simple, unified computational account of noradrenergic computational function within a general model of behaviour.

scholarly journals Predictability Changes What We Remember in Familiar Temporal Contexts

2020 ◽  
Vol 32 (1) ◽  
pp. 124-140 ◽  
Author(s):  
Hyojeong Kim ◽  
Margaret L. Schlichting ◽  
Alison R. Preston ◽  
Jarrod A. Lewis-Peacock
Keyword(s):  
Human Brain ◽  
Prediction Error ◽  
Fmri Data ◽  
Prediction Errors ◽  
Associative Memories ◽  
The Past ◽  
Item Level ◽  
Abstract Level ◽  
The Brain

The human brain constantly anticipates the future based on memories of the past. Encountering a familiar situation reactivates memory of previous encounters, which can trigger a prediction of what comes next to facilitate responsiveness. However, a prediction error can lead to pruning of the offending memory, a process that weakens its representation in the brain and leads to forgetting. Our goal in this study was to evaluate whether memories are spared from such pruning in situations that allow for accurate predictions at the categorical level, despite prediction errors at the item level. Participants viewed a sequence of objects, some of which reappeared multiple times (“cues”), followed always by novel items. Half of the cues were followed by new items from different (unpredictable) categories, while others were followed by new items from a single (predictable) category. Pattern classification of fMRI data was used to identify category-specific predictions after each cue. Pruning was observed only in unpredictable contexts, while encoding of new items was less robust in predictable contexts. These findings demonstrate that how associative memories are updated is influenced by the reliability of abstract-level predictions in familiar contexts.

scholarly journals Learning prediction error neurons in a canonical interneuron circuit

eLife ◽  
2020 ◽  
Vol 9 ◽  
Author(s):  
Loreen Hertäg ◽  
Henning Sprekeler
Keyword(s):  
Visual Cortex ◽  
Primary Visual Cortex ◽  
Prediction Error ◽  
Sensory Information ◽  
Prediction Errors ◽  
Negative Prediction ◽  
Inhibitory Plasticity ◽  
The Brain ◽  
Shed Light ◽  
Model Circuit

Sensory systems constantly compare external sensory information with internally generated predictions. While neural hallmarks of prediction errors have been found throughout the brain, the circuit-level mechanisms that underlie their computation are still largely unknown. Here, we show that a well-orchestrated interplay of three interneuron types shapes the development and refinement of negative prediction-error neurons in a computational model of mouse primary visual cortex. By balancing excitation and inhibition in multiple pathways, experience-dependent inhibitory plasticity can generate different variants of prediction-error circuits, which can be distinguished by simulated optogenetic experiments. The experience-dependence of the model circuit is consistent with that of negative prediction-error circuits in layer 2/3 of mouse primary visual cortex. Our model makes a range of testable predictions that may shed light on the circuitry underlying the neural computation of prediction errors.

scholarly journals A Bayesian model of context-sensitive value attribution

eLife ◽  
2016 ◽  
Vol 5 ◽  
Author(s):  
Francesco Rigoli ◽  
Karl J Friston ◽  
Cristina Martinelli ◽  
Mirjana Selaković ◽  
Sukhwinder S Shergill ◽  
...  
Keyword(s):  
Contextual Information ◽  
Empirical Support ◽  
Prediction Errors ◽  
Substantial Evidence ◽  
Prior Beliefs ◽  
Context Sensitive ◽  
Incentive Value ◽  
Hierarchical Levels ◽  
Key Features ◽  
The Brain

Substantial evidence indicates that incentive value depends on an anticipation of rewards within a given context. However, the computations underlying this context sensitivity remain unknown. To address this question, we introduce a normative (Bayesian) account of how rewards map to incentive values. This assumes that the brain inverts a model of how rewards are generated. Key features of our account include (i) an influence of prior beliefs about the context in which rewards are delivered (weighted by their reliability in a Bayes-optimal fashion), (ii) the notion that incentive values correspond to precision-weighted prediction errors, (iii) and contextual information unfolding at different hierarchical levels. This formulation implies that incentive value is intrinsically context-dependent. We provide empirical support for this model by showing that incentive value is influenced by context variability and by hierarchically nested contexts. The perspective we introduce generates new empirical predictions that might help explaining psychopathologies, such as addiction.

scholarly journals The predictive brain in action: Involuntary actions reduce body prediction errors

2020 ◽  
Author(s):  
Pablo Lanillos ◽  
Sae Franklin ◽  
David W. Franklin
Keyword(s):  
Prediction Error ◽  
Sensory Information ◽  
The Body ◽  
Predictive Processing ◽  
Body Ownership ◽  
Prediction Errors ◽  
Uncertain Information ◽  
Energy Principle ◽  
Virtual Hand ◽  
The Brain

AbstractThe perception of our body in space is flexible and manipulable. The predictive brain hypothesis explains this malleability as a consequence of the interplay between incoming sensory information and our body expectations. However, given the interaction between perception and action, we might also expect that actions would arise due to prediction errors, especially in conflicting situations. Here we describe a computational model, based on the free-energy principle, that forecasts involuntary movements in sensorimotor conflicts. We experimentally confirm those predictions in humans by means of a virtual reality rubber-hand illusion. Participants generated movements (forces) towards the virtual hand, regardless of its location with respect to the real arm, with little to no forces produced when the virtual hand overlaid their physical hand. The congruency of our model predictions and human observations shows that the brain-body is generating actions to reduce the prediction error between the expected arm location and the new visual arm. This observed unconscious mechanism is an empirical validation of the perception-action duality in body adaptation to uncertain situations and evidence of the active component of predictive processing.Author SummaryHumans’ capacity to perceive and control their body in space is central in awareness, adaptation and safe interaction. From low-level body perception to body-ownership, discovering how the brain represents the body and generates actions is of major importance for cognitive science and also for robotics and artificial intelligence. The present study shows that humans move their body to match the expected location according to other (visual) sensory input, which corresponds to reducing the prediction error. This means that the brain adapts to conflicting or uncertain information from the senses by unconsciously acting in the world.

scholarly journals Learning prediction error neurons in a canonical interneuron circuit

2020 ◽  
Author(s):  
Loreen Hertäg ◽  
Henning Sprekeler
Keyword(s):  
Visual Cortex ◽  
Primary Visual Cortex ◽  
Prediction Error ◽  
Sensory Information ◽  
Prediction Errors ◽  
Negative Prediction ◽  
Inhibitory Plasticity ◽  
The Brain ◽  
Shed Light ◽  
Model Circuit

AbstractSensory systems constantly compare external sensory information with internally generated predictions. While neural hallmarks of prediction errors have been found throughout the brain, the circuit-level mechanisms that underlie their computation are still largely unknown. Here, we show that a well-orchestrated interplay of three interneuron types shapes the development and refinement of negative prediction-error neurons in a computational model of mouse primary visual cortex. By balancing excitation and inhibition in multiple pathways, experience-dependent inhibitory plasticity can generate different variants of prediction-error circuits, which can be distinguished by simulated optogenetic experiments. The experience-dependence of the model circuit is consistent with that of negative prediction-error circuits in layer 2/3 of mouse primary visual cortex. Our model makes a range of testable predictions that may shed light on the circuitry underlying the neural computation of prediction errors.

Pupil Drift Rate Indexes Groove Ratings

2021 ◽  
Author(s):  
Connor Spiech ◽  
George Sioros ◽  
Tor Endestad ◽  
Anne Danielsen ◽  
Bruno Laeng
Keyword(s):  
Prediction Error ◽  
Drift Rate ◽  
Temporal Structure ◽  
Predictive Coding ◽  
Prediction Errors ◽  
Musical Ability ◽  
Moderate Complexity ◽  
Musical Rhythms ◽  
Sensory Prediction ◽  
The Brain

Groove, understood as a pleasurable compulsion to move to musical rhythms, typically varies along an inverted U-curve with increasing rhythmic complexity (e.g., syncopation, pickups). Predictive coding accounts posit that moderate complexity drives us to move to reduce sensory prediction errors and model the temporal structure. While musicologists generally distinguish the effects of pickups (anacruses) and syncopations, their difference remains unexplored in groove. We used pupillometry as an index to noradrenergic arousal while subjects listened to and rated drumbeats varying in rhythmic complexity. We replicated the inverted U-shaped relationship between rhythmic complexity and groove and showed this is modulated by musical ability, based on a psychoacoustic beat perception test. The pupil drift rates suggest that groovier rhythms hold attention longer than ones rated less groovy. Moreover, we found complementary effects of syncopations and pickups on groove ratings and pupil size, respectively, discovering a distinct predictive process related to pickups. We suggest that the brain deploys attention to pickups to sharpen subsequent strong beats, augmenting the predictive scaffolding’s focus on beats that reduce syncopations’ prediction errors. This interpretation is in accordance with groove envisioned as an embodied resolution of precision-weighted prediction error.

Sign in / Sign up

Export Citation Format

Share Document