scholarly journals Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory

2020 ◽  
Author(s):  
Cristian Buc Calderon ◽  
Esther De Loof ◽  
Kate Ergo ◽  
Anna Snoeck ◽  
Carsten Nico Boehler ◽  
...  
Keyword(s):  
Episodic Memory ◽  
Ventral Striatum ◽  
Declarative Memory ◽  
Choice Task ◽  
Prediction Errors ◽  
Reward Prediction ◽  
Key Factor ◽  
Memory Acquisition ◽  
Behavioral Evidence

AbstractA growing body of behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPE in declarative memory acquisition remain to be tested. Using a novel variable-choice task, we experimentally manipulated RPEs and found support for key predictions on the neural level with fMRI. Specifically, we demonstrate that trial-specific RPE responses in the ventral striatum (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (e.g., face-selective areas) and hippocampus, ventral tegmental area, and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key operations subtending the formation of episodic memory.

scholarly journals Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory

2020 ◽  
pp. JN-RM-1785-20
Author(s):  
Cristian B. Calderon ◽  
Esther De Loof ◽  
Kate Ergo ◽  
Anna Snoeck ◽  
Carsten N. Boehler ◽  
...  
Keyword(s):  
Episodic Memory ◽  
Ventral Striatum ◽  
Prediction Errors ◽  
Reward Prediction

Reward Prediction Errors Drive Declarative Learning Irrespective of Agency

2020 ◽  
Author(s):  
Kate Ergo ◽  
Luna De Vilder ◽  
Esther De Loof ◽  
Tom Verguts
Keyword(s):  
Learning Theory ◽  
Learning Effect ◽  
Steady Increase ◽  
Prediction Errors ◽  
Experimental Paradigm ◽  
Reward Prediction ◽  
Declarative Learning

Recent years have witnessed a steady increase in the number of studies investigating the role of reward prediction errors (RPEs) in declarative learning. Specifically, in several experimental paradigms RPEs drive declarative learning; with larger and more positive RPEs enhancing declarative learning. However, it is unknown whether this RPE must derive from the participant’s own response, or whether instead any RPE is sufficient to obtain the learning effect. To test this, we generated RPEs in the same experimental paradigm where we combined an agency and a non-agency condition. We observed no interaction between RPE and agency, suggesting that any RPE (irrespective of its source) can drive declarative learning. This result holds implications for declarative learning theory.

scholarly journals Beyond reward prediction errors: the role of dopamine in movement kinematics

2015 ◽  
Vol 9 ◽  
Author(s):  
Joseph W. Barter ◽  
Suellen Li ◽  
Dongye Lu ◽  
Ryan A. Bartholomew ◽  
Mark A. Rossi ◽  
...  
Keyword(s):  
Prediction Errors ◽  
Movement Kinematics ◽  
Reward Prediction

scholarly journals Task errors contribute to implicit remapping in sensorimotor adaptation

2018 ◽  
Author(s):  
Li-Ann Leow ◽  
Welber Marinovic ◽  
Aymar de Rugy ◽  
Timothy J Carroll
Keyword(s):  
Sensory Feedback ◽  
Hand Position ◽  
Sensorimotor Adaptation ◽  
Prediction Errors ◽  
Reward Prediction ◽  
Strategic Processes ◽  
The Relationship ◽  
Sensory Prediction ◽  
Or Task

AbstractPerturbations of sensory feedback evoke sensory prediction errors (discrepancies between predicted and actual sensory outcomes of movements), and reward prediction errors (discrepancies between predicted rewards and actual rewards). Sensory prediction errors result in obligatory remapping of the relationship between motor commands and predicted sensory outcomes. The role of reward prediction errors in sensorimotor adaptation is less clear. When moving towards a target, we expect to obtain the reward of hitting the target, and so we experience a reward prediction error if the perturbation causes us to miss it. These discrepancies between desired task outcomes and actual task outcomes, or “task errors”, are thought to drive the use of strategic processes to restore success, although their role is not fully understood. Here, we investigated the role of task errors in sensorimotor adaptation: during target-reaching, we either removed task errors by moving the target mid-movement to align with cursor feedback of hand position, or enforced task error by moving the target away from the cursor feedback of hand position. Removing task errors not only reduced the rate and extent of adaptation during exposure to the perturbation, but also reduced the amount of post-adaptation implicit remapping. Hence, task errors contribute to implicit remapping resulting from sensory prediction errors. This suggests that the system which implicitly acquires new sensorimotor maps via exposure to sensory prediction errors is also sensitive to reward prediction errors.

scholarly journals Contextual Prediction Errors Reorganize Episodic Memories in Time

2020 ◽  
Author(s):  
Fahd Yazin ◽  
Moumita Das ◽  
Arpan Banerjee ◽  
Dipanjan Roy
Keyword(s):  
Declarative Memory ◽  
Temporal Organization ◽  
Prediction Errors ◽  
Decision Threshold ◽  
Time Dimension ◽  
Memory Processing ◽  
Sequential Memory ◽  
Episodic Memories ◽  
Novel Associations

AbstractEpisodic memories are contextual experiences ordered in time. This is underpinned by associative binding between events within the same contexts. The role of prediction errors in strengthening declarative memory is well established but has not been investigated in the time dimension of complex episodic memories. Here we used 3-day movie viewing paradigm to test the hypothesis that contextual prediction errors leads to temporal organization of sequential memory processing. Our main findings uncover that prediction errors lead to changes in temporal organization of events, secondly, new unexpected sequences show as high accuracy as control sequences viewed repeatedly, and these effects are specifically due to prediction errors, and not novel associations. A drift-diffusion modelling further revealed a lower decision threshold for the newer, unexpected sequences compared to older sequences reflected by their faster recall leads to reorganization of episodes in time. Moreover, we found individual decision threshold could significantly predict their relative speed of sequence memory recall. Taking together our results suggest a temporally distinct role for prediction errors in ordering sequences of events in episodic memory.

scholarly journals Reward prediction error and declarative memory

2020 ◽  
Author(s):  
Kate Ergo ◽  
Esther De Loof ◽  
Tom Verguts
Keyword(s):  
Prediction Error ◽  
Declarative Memory ◽  
Empirical Studies ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Emerging Patterns ◽  
Declarative Learning ◽  
Open Issues

Learning based on reward prediction error (RPE) was originally proposed in the context of non-declarative memory. We postulate that RPE may support declarative memory as well. Indeed, recent years have witnessed a number of independent empirical studies reporting effects of RPE on declarative memory. In this paper, we provide a brief overview of these studies, point out emerging patterns, and identify open issues such as the role of signed versus unsigned RPEs in declarative learning.

scholarly journals What, If, and When to Move: Basal Ganglia Circuits and Self-Paced Action Initiation

2019 ◽  
Vol 42 (1) ◽  
pp. 459-483 ◽  
Author(s):  
Andreas Klaus ◽  
Joaquim Alves da Silva ◽  
Rui M. Costa
Keyword(s):  
Basal Ganglia ◽  
Prediction Errors ◽  
Movement Initiation ◽  
Reward Prediction ◽  
Behavioral Transitions ◽  
Main Input ◽  
Action Initiation

Deciding what to do and when to move is vital to our survival. Clinical and fundamental studies have identified basal ganglia circuits as critical for this process. The main input nucleus of the basal ganglia, the striatum, receives inputs from frontal, sensory, and motor cortices and interconnected thalamic areas that provide information about potential goals, context, and actions and directly or indirectly modulates basal ganglia outputs. The striatum also receives dopaminergic inputs that can signal reward prediction errors and also behavioral transitions and movement initiation. Here we review studies and models of how direct and indirect pathways can modulate basal ganglia outputs to facilitate movement initiation, and we discuss the role of cortical and dopaminergic inputs to the striatum in determining what to do and if and when to do it. Complex but exciting scenarios emerge that shed new light on how basal ganglia circuits modulate self-paced movement initiation.

scholarly journals Striatal Dopamine and Reward Prediction Error Signaling in Unmedicated Schizophrenia Patients

2020 ◽  
Vol 46 (6) ◽  
pp. 1535-1546
Author(s):  
Teresa Katthagen ◽  
Jakob Kaminski ◽  
Andreas Heinz ◽  
Ralph Buchert ◽  
Florian Schlagenhauf
Keyword(s):  
Reversal Learning ◽  
Prediction Error ◽  
Negative Symptoms ◽  
Ventral Striatum ◽  
Striatal Dopamine ◽  
Dopamine Synthesis ◽  
Positive Symptoms ◽  
Prediction Errors ◽  
Reward Prediction Error ◽  
Reward Prediction

Abstract Increased striatal dopamine synthesis capacity has consistently been reported in patients with schizophrenia. However, the mechanism translating this into behavior and symptoms remains unclear. It has been proposed that heightened striatal dopamine may blunt dopaminergic reward prediction error signaling during reinforcement learning. In this study, we investigated striatal dopamine synthesis capacity, reward prediction errors, and their association in unmedicated schizophrenia patients (n = 19) and healthy controls (n = 23). They took part in FDOPA-PET and underwent functional magnetic resonance imaging (fMRI) scanning, where they performed a reversal-learning paradigm. The groups were compared regarding dopamine synthesis capacity (Kicer), fMRI neural prediction error signals, and the correlation of both. Patients did not differ from controls with respect to striatal Kicer. Taking into account, comorbid alcohol abuse revealed that patients without such abuse showed elevated Kicer in the associative striatum, while those with abuse did not differ from controls. Comparing all patients to controls, patients performed worse during reversal learning and displayed reduced prediction error signaling in the ventral striatum. In controls, Kicer in the limbic striatum correlated with higher reward prediction error signaling, while there was no significant association in patients. Kicer in the associative striatum correlated with higher positive symptoms and blunted reward prediction error signaling was associated with negative symptoms. Our results suggest a dissociation between striatal subregions and symptom domains, with elevated dopamine synthesis capacity in the associative striatum contributing to positive symptoms while blunted prediction error signaling in the ventral striatum related to negative symptoms.

scholarly journals Contextual Prediction Errors Reorganize Naturalistic Episodic Memories in Time

2021 ◽  
Author(s):  
Fahd Yazin ◽  
Moumita Das ◽  
Arpan Banerjee ◽  
Dipanjan Roy
Keyword(s):  
Declarative Memory ◽  
Prediction Errors ◽  
Decision Threshold ◽  
Time Dimension ◽  
Drift Diffusion ◽  
Temporal Domain ◽  
Sequence Memory ◽  
Episodic Memories ◽  
Adaptive Role

Abstract Episodic memories are contextual experiences ordered in time. This is underpinned by associative binding between events within the same contexts. The role of prediction errors in declarative memory is well established but has not been investigated in the time dimension of complex episodic memories. Here we combine these two properties of episodic memory, extend them into the temporal domain and demonstrate that prediction errors in different naturalistic contexts lead to changes in the temporal ordering of event structures in them. The wrongly predicted older sequences were weakened despite reactivating them after. Interestingly, the newly encoded sequences with prediction errors, seen once, showed as high accuracy as control sequences which were viewed repeatedly without change. Drift-diffusion modelling revealed a lower decision threshold for the newer sequences compared to older sequences, reflected by their faster recall. Moreover participants’ adjustments to their decision threshold could significantly predict their relative speed of sequence memory recall. This suggests a temporally distinct and adaptive role for prediction errors in learning and reorganizing episodic sequences.

scholarly journals Causal evidence supporting the proposal that dopamine transients function as a temporal difference prediction error

2019 ◽  
Author(s):  
Etienne JP Maes ◽  
Melissa J Sharpe ◽  
Matthew P.H. Gardner ◽  
Chun Yun Chang ◽  
Geoffrey Schoenbaum ◽  
...  
Keyword(s):  
Prediction Error ◽  
Second Order ◽  
Prediction Errors ◽  
Temporal Difference ◽  
Reward Prediction ◽  
Order Conditioning ◽  
Central Tenet

Reward-evoked dopamine is well-established as a prediction error. However the central tenet of temporal difference accounts – that similar transients evoked by reward-predictive cues also function as errors – remains untested. To address this, we used two phenomena, second-order conditioning and blocking, in order to examine the role of dopamine in prediction error versus reward prediction. We show that optogenetically-shunting dopamine activity at the start of a reward-predicting cue prevents second-order conditioning without affecting blocking. These results support temporal difference accounts by providing causal evidence that cue-evoked dopamine transients function as prediction errors.

Sign in / Sign up

Export Citation Format

Share Document