scholarly journals Beyond reward prediction errors: the role of dopamine in movement kinematics

2015 ◽  
Vol 9 ◽  
Author(s):  
Joseph W. Barter ◽  
Suellen Li ◽  
Dongye Lu ◽  
Ryan A. Bartholomew ◽  
Mark A. Rossi ◽  
...  
Keyword(s):  
Prediction Errors ◽  
Movement Kinematics ◽  
Reward Prediction

Reward Prediction Errors Drive Declarative Learning Irrespective of Agency

2020 ◽  
Author(s):  
Kate Ergo ◽  
Luna De Vilder ◽  
Esther De Loof ◽  
Tom Verguts
Keyword(s):  
Learning Theory ◽  
Learning Effect ◽  
Steady Increase ◽  
Prediction Errors ◽  
Experimental Paradigm ◽  
Reward Prediction ◽  
Declarative Learning

Recent years have witnessed a steady increase in the number of studies investigating the role of reward prediction errors (RPEs) in declarative learning. Specifically, in several experimental paradigms RPEs drive declarative learning; with larger and more positive RPEs enhancing declarative learning. However, it is unknown whether this RPE must derive from the participant’s own response, or whether instead any RPE is sufficient to obtain the learning effect. To test this, we generated RPEs in the same experimental paradigm where we combined an agency and a non-agency condition. We observed no interaction between RPE and agency, suggesting that any RPE (irrespective of its source) can drive declarative learning. This result holds implications for declarative learning theory.

scholarly journals Task errors contribute to implicit remapping in sensorimotor adaptation

2018 ◽  
Author(s):  
Li-Ann Leow ◽  
Welber Marinovic ◽  
Aymar de Rugy ◽  
Timothy J Carroll
Keyword(s):  
Sensory Feedback ◽  
Hand Position ◽  
Sensorimotor Adaptation ◽  
Prediction Errors ◽  
Reward Prediction ◽  
Strategic Processes ◽  
The Relationship ◽  
Sensory Prediction ◽  
Or Task

AbstractPerturbations of sensory feedback evoke sensory prediction errors (discrepancies between predicted and actual sensory outcomes of movements), and reward prediction errors (discrepancies between predicted rewards and actual rewards). Sensory prediction errors result in obligatory remapping of the relationship between motor commands and predicted sensory outcomes. The role of reward prediction errors in sensorimotor adaptation is less clear. When moving towards a target, we expect to obtain the reward of hitting the target, and so we experience a reward prediction error if the perturbation causes us to miss it. These discrepancies between desired task outcomes and actual task outcomes, or “task errors”, are thought to drive the use of strategic processes to restore success, although their role is not fully understood. Here, we investigated the role of task errors in sensorimotor adaptation: during target-reaching, we either removed task errors by moving the target mid-movement to align with cursor feedback of hand position, or enforced task error by moving the target away from the cursor feedback of hand position. Removing task errors not only reduced the rate and extent of adaptation during exposure to the perturbation, but also reduced the amount of post-adaptation implicit remapping. Hence, task errors contribute to implicit remapping resulting from sensory prediction errors. This suggests that the system which implicitly acquires new sensorimotor maps via exposure to sensory prediction errors is also sensitive to reward prediction errors.

scholarly journals What, If, and When to Move: Basal Ganglia Circuits and Self-Paced Action Initiation

2019 ◽  
Vol 42 (1) ◽  
pp. 459-483 ◽  
Author(s):  
Andreas Klaus ◽  
Joaquim Alves da Silva ◽  
Rui M. Costa
Keyword(s):  
Basal Ganglia ◽  
Prediction Errors ◽  
Movement Initiation ◽  
Reward Prediction ◽  
Behavioral Transitions ◽  
Main Input ◽  
Action Initiation

Deciding what to do and when to move is vital to our survival. Clinical and fundamental studies have identified basal ganglia circuits as critical for this process. The main input nucleus of the basal ganglia, the striatum, receives inputs from frontal, sensory, and motor cortices and interconnected thalamic areas that provide information about potential goals, context, and actions and directly or indirectly modulates basal ganglia outputs. The striatum also receives dopaminergic inputs that can signal reward prediction errors and also behavioral transitions and movement initiation. Here we review studies and models of how direct and indirect pathways can modulate basal ganglia outputs to facilitate movement initiation, and we discuss the role of cortical and dopaminergic inputs to the striatum in determining what to do and if and when to do it. Complex but exciting scenarios emerge that shed new light on how basal ganglia circuits modulate self-paced movement initiation.

scholarly journals Causal evidence supporting the proposal that dopamine transients function as a temporal difference prediction error

2019 ◽  
Author(s):  
Etienne JP Maes ◽  
Melissa J Sharpe ◽  
Matthew P.H. Gardner ◽  
Chun Yun Chang ◽  
Geoffrey Schoenbaum ◽  
...  
Keyword(s):  
Prediction Error ◽  
Second Order ◽  
Prediction Errors ◽  
Temporal Difference ◽  
Reward Prediction ◽  
Order Conditioning ◽  
Central Tenet

Reward-evoked dopamine is well-established as a prediction error. However the central tenet of temporal difference accounts – that similar transients evoked by reward-predictive cues also function as errors – remains untested. To address this, we used two phenomena, second-order conditioning and blocking, in order to examine the role of dopamine in prediction error versus reward prediction. We show that optogenetically-shunting dopamine activity at the start of a reward-predicting cue prevents second-order conditioning without affecting blocking. These results support temporal difference accounts by providing causal evidence that cue-evoked dopamine transients function as prediction errors.

scholarly journals Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory

2020 ◽  
Author(s):  
Cristian Buc Calderon ◽  
Esther De Loof ◽  
Kate Ergo ◽  
Anna Snoeck ◽  
Carsten Nico Boehler ◽  
...  
Keyword(s):  
Episodic Memory ◽  
Ventral Striatum ◽  
Declarative Memory ◽  
Choice Task ◽  
Prediction Errors ◽  
Reward Prediction ◽  
Key Factor ◽  
Memory Acquisition ◽  
Behavioral Evidence

AbstractA growing body of behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPE in declarative memory acquisition remain to be tested. Using a novel variable-choice task, we experimentally manipulated RPEs and found support for key predictions on the neural level with fMRI. Specifically, we demonstrate that trial-specific RPE responses in the ventral striatum (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (e.g., face-selective areas) and hippocampus, ventral tegmental area, and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key operations subtending the formation of episodic memory.

Emotion prediction errors guide socially adaptive behavior

2021 ◽  
Author(s):  
Joseph Heffner ◽  
Jae-Young Son ◽  
Oriel FeldmanHall
Keyword(s):  
Decision Making ◽  
Real Time ◽  
Adaptive Behavior ◽  
Emotional Response ◽  
New Method ◽  
Prediction Errors ◽  
Emotional Experiences ◽  
Past Work ◽  
Reward Prediction ◽  
Expected Outcomes

People make decisions based on deviations from expected outcomes, known as prediction errors. Past work has focused on reward prediction errors, largely ignoring violations of expected emotional experiences—emotion prediction errors. We leverage a new method to measure real-time fluctuations in emotion as people decide to punish or forgive others. Across four studies (N=1,016), we reveal that emotion and reward prediction errors have distinguishable contributions to choice, such that emotion prediction errors exert the strongest impact during decision-making. We additionally find that a choice to punish or forgive can be decoded in less than a second from an evolving emotional response, suggesting emotions swiftly influence choice. Finally, individuals reporting significant levels of depression exhibit selective impairments in using emotion—but not reward—prediction errors. Evidence for emotion prediction errors potently guiding social behaviors challenge standard decision-making models that have focused solely on reward.

The Embodiment of Concepts

2018 ◽  
pp. 640-660
Author(s):  
Michiel Van Elk ◽  
Harold Bekkering
Keyword(s):  
Prediction Error ◽  
Predictive Processing ◽  
Prediction Errors ◽  
Conceptual Representation ◽  
Recurrent Processing ◽  
Different Dimensions ◽  
Prior Models ◽  
Processing Framework

We characterize theories of conceptual representation as embodied, disembodied, or hybrid according to their stance on a number of different dimensions: the nature of concepts, the relation between language and concepts, the function of concepts, the acquisition of concepts, the representation of concepts, and the role of context. We propose to extend an embodied view of concepts, by taking into account the importance of multimodal associations and predictive processing. We argue that concepts are dynamically acquired and updated, based on recurrent processing of prediction error signals in a hierarchically structured network. Concepts are thus used as prior models to generate multimodal expectations, thereby reducing surprise and enabling greater precision in the perception of exemplars. This view places embodied theories of concepts in a novel predictive processing framework, by highlighting the importance of concepts for prediction, learning and shaping categories on the basis of prediction errors.

scholarly journals Epilepsy and Ecstatic Experiences: The Role of the Insula

2021 ◽  
Vol 11 (11) ◽  
pp. 1384
Author(s):  
Fabienne Picard ◽  
Peter Bossaerts ◽  
Fabrice Bartolomei
Keyword(s):  
Temporal Lobe Epilepsy ◽  
Temporal Lobe ◽  
Mystical Experience ◽  
Focal Epilepsy ◽  
Anterior Insula ◽  
External Signal ◽  
Prediction Errors ◽  
Rare Form ◽  
The World

Ecstatic epilepsy is a rare form of focal epilepsy in which the aura (beginning of the seizures) consists of a blissful state of mental clarity/feeling of certainty. Such a state has also been described as a “religious” or mystical experience. While this form of epilepsy has long been recognized as a temporal lobe epilepsy, we have accumulated evidence converging toward the location of the symptomatogenic zone in the dorsal anterior insula during the 10 last years. The neurocognitive hypothesis for the genesis of a mental clarity is the suppression of the interoceptive prediction errors and of the unexpected surprise associated with any incoming internal or external signal, usually processed by the dorsal anterior insula. This mimics a perfect prediction of the world and induces a feeling of certainty. The ecstatic epilepsy is thus an amazing model for the role of anterior insula in uncertainty and surprise.

Sign in / Sign up

Export Citation Format

Share Document