scholarly journals Discrete coding of stimulus value, reward expectation, and reward prediction error in the dorsal striatum

2015 ◽  
Vol 114 (5) ◽  
pp. 2600-2615 ◽  
Author(s):  
Kei Oyama ◽  
Yukina Tateyama ◽  
István Hernádi ◽  
Philippe N. Tobler ◽  
Toshio Iijima ◽  
...  
Keyword(s):  
Prediction Error ◽  
Sensory Information ◽  
Dorsal Striatum ◽  
Striatal Neurons ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Reward Probability ◽  
Stimulus Value ◽  
Reward Contingencies ◽  
Reward Information

To investigate how the striatum integrates sensory information with reward information for behavioral guidance, we recorded single-unit activity in the dorsal striatum of head-fixed rats participating in a probabilistic Pavlovian conditioning task with auditory conditioned stimuli (CSs) in which reward probability was fixed for each CS but parametrically varied across CSs. We found that the activity of many neurons was linearly correlated with the reward probability indicated by the CSs. The recorded neurons could be classified according to their firing patterns into functional subtypes coding reward probability in different forms such as stimulus value, reward expectation, and reward prediction error. These results suggest that several functional subgroups of dorsal striatal neurons represent different kinds of information formed through extensive prior exposure to CS-reward contingencies.

scholarly journals Reward Prediction Error Coding in Dorsal Striatal Neurons

2010 ◽  
Vol 30 (34) ◽  
pp. 11447-11457 ◽  
Author(s):  
K. Oyama ◽  
I. Hernadi ◽  
T. Iijima ◽  
K.-I. Tsutsui
Keyword(s):  
Prediction Error ◽  
Striatal Neurons ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Error Coding

scholarly journals Differential reinforcement encoding along the hippocampal long axis helps resolve the explore–exploit dilemma

2020 ◽  
Vol 11 (1) ◽  
Author(s):  
Alexandre Y. Dombrovski ◽  
Beatriz Luna ◽  
Michael N. Hallquist
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Differential Reinforcement ◽  
Cognitive Maps ◽  
Learning Task ◽  
Natural Environments ◽  
Reward Prediction Error ◽  
Reward Function ◽  
Reward Prediction ◽  
Reward Information

Abstract When making decisions, should one exploit known good options or explore potentially better alternatives? Exploration of spatially unstructured options depends on the neocortex, striatum, and amygdala. In natural environments, however, better options often cluster together, forming structured value distributions. The hippocampus binds reward information into allocentric cognitive maps to support navigation and foraging in such spaces. Here we report that human posterior hippocampus (PH) invigorates exploration while anterior hippocampus (AH) supports the transition to exploitation on a reinforcement learning task with a spatially structured reward function. These dynamics depend on differential reinforcement representations in the PH and AH. Whereas local reward prediction error signals are early and phasic in the PH tail, global value maximum signals are delayed and sustained in the AH body. AH compresses reinforcement information across episodes, updating the location and prominence of the value maximum and displaying goal cell-like ramping activity when navigating toward it.

Different Neural Correlates of Reward Expectation and Reward Expectation Error in the Putamen and Caudate Nucleus During Stimulus-Action-Reward Association Learning

2006 ◽  
Vol 95 (2) ◽  
pp. 948-959 ◽  
Author(s):  
Masahiko Haruno ◽  
Mitsuo Kawato
Keyword(s):  
Caudate Nucleus ◽  
Prediction Error ◽  
Dorsal Striatum ◽  
Neural Correlates ◽  
Functional Difference ◽  
Sensory Stimuli ◽  
Reward Prediction Error ◽  
Association Learning ◽  
Reward Prediction ◽  
Fmri Study

To select appropriate behaviors leading to rewards, the brain needs to learn associations among sensory stimuli, selected behaviors, and rewards. Recent imaging and neural-recording studies have revealed that the dorsal striatum plays an important role in learning such stimulus-action-reward associations. However, the putamen and caudate nucleus are embedded in distinct cortico-striatal loop circuits, predominantly connected to motor-related cerebral cortical areas and frontal association areas, respectively. This difference in their cortical connections suggests that the putamen and caudate nucleus are engaged in different functional aspects of stimulus-action-reward association learning. To determine whether this is the case, we conducted an event-related and computational model–based functional MRI (fMRI) study with a stochastic decision-making task in which a stimulus-action-reward association must be learned. A simple reinforcement learning model not only reproduced the subject's action selections reasonably well but also allowed us to quantitatively estimate each subject's temporal profiles of stimulus-action-reward association and reward-prediction error during learning trials. These two internal representations were used in the fMRI correlation analysis. The results revealed that neural correlates of the stimulus-action-reward association reside in the putamen, whereas a correlation with reward-prediction error was found largely in the caudate nucleus and ventral striatum. These nonuniform spatiotemporal distributions of neural correlates within the dorsal striatum were maintained consistently at various levels of task difficulty, suggesting a functional difference in the dorsal striatum between the putamen and caudate nucleus during stimulus-action-reward association learning.

Reward prediction error coding in striatal neurons

2010 ◽  
Vol 68 ◽  
pp. e288-e289
Author(s):  
Kei Oyama ◽  
Istvan Hernadi ◽  
Toshio Iijima ◽  
Ken-Ichiro Tsutsui
Keyword(s):  
Prediction Error ◽  
Striatal Neurons ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Error Coding

scholarly journals Differential reinforcement encoding along the hippocampal long axis helps resolve the explore/exploit dilemma

2020 ◽  
Author(s):  
Alexandre Y. Dombrovski ◽  
Beatriz Luna ◽  
Michael N. Hallquist
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Differential Reinforcement ◽  
Cognitive Maps ◽  
Learning Task ◽  
Natural Environments ◽  
Reward Prediction Error ◽  
Reward Function ◽  
Reward Prediction ◽  
Reward Information

ABSTRACTWhen making decisions, should one exploit known good options or explore potentially better alternatives? Exploration of spatially unstructured options depends on the neocortex, striatum, and amygdala. In natural environments, however, better options often cluster together, forming structured value distributions. The hippocampus binds reward information into allocentric cognitive maps to support navigation and foraging in such spaces. Using a reinforcement learning task with a spatially structured reward function, we show that human posterior hippocampus (PH) invigorates exploration while anterior hippocampus (AH) supports the transition to exploitation. These dynamics depend on differential reinforcement representations in the PH and AH. Whereas local reward prediction error signals are early and phasic in the PH tail, global value maximum signals are delayed and sustained in the AH body. AH compresses reinforcement information across episodes, updating the location and prominence of the value maximum and displaying goal cell-like ramping activity when navigating toward it.

scholarly journals A VTA GABAergic computational model of dissociated reward prediction error computation in classical conditioning

2020 ◽  
Author(s):  
Pramod Kaushik ◽  
Jérémie Naudé ◽  
Surampudi Bapi Raju ◽  
Frédéric Alexandre
Keyword(s):  
Classical Conditioning ◽  
Computational Model ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Dopamine Neurons ◽  
System Level ◽  
Reward Prediction Error ◽  
Twin Peaks ◽  
Reward Prediction ◽  
Error Computation

AbstractClassical Conditioning is a fundamental learning mechanism where the Ventral Striatum is generally thought to be the source of inhibition to Ventral Tegmental Area (VTA) Dopamine neurons when a reward is expected. However, recent evidences point to a new candidate in VTA GABA encoding expectation for computing the reward prediction error in the VTA. In this system-level computational model, the VTA GABA signal is hypothesised to be a combination of magnitude and timing computed in the Peduncolopontine and Ventral Striatum respectively. This dissociation enables the model to explain recent results wherein Ventral Striatum lesions affected the temporal expectation of the reward but the magnitude of the reward was intact. This model also exhibits other features in classical conditioning namely, progressively decreasing firing for early rewards closer to the actual reward, twin peaks of VTA dopamine during training and cancellation of US dopamine after training.

T92. Time Scales of Encoding the Reward Prediction Error in Youth: Representation of Past Events

2018 ◽  
Vol 83 (9) ◽  
pp. S164
Author(s):  
Hanna Keren ◽  
Nathan Fox ◽  
Ellen Leibenluft ◽  
Daniel S. Pine ◽  
Argyris Stringaris
Keyword(s):  
Time Scales ◽  
Prediction Error ◽  
Reward Prediction Error ◽  
Reward Prediction

scholarly journals O5.1. STRIATAL DOPAMINE AND REDUCED REWARD PREDICTION ERROR SIGNALING IN UNMEDICATED SCHIZOPHRENIA PATIENTS

2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S11-S11
Author(s):  
Teresa Katthagen ◽  
Jakob Kaminski ◽  
Andreas Heinz ◽  
Ralph Buchert ◽  
Florian Schlagenhauf
Keyword(s):  
Prediction Error ◽  
Negative Symptoms ◽  
Ventral Striatum ◽  
Striatal Dopamine ◽  
Dopamine Synthesis ◽  
Positive Symptoms ◽  
Healthy Controls ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Significant Difference

Abstract Background Increased striatal dopamine synthesis capacity (DSC) has consistently been reported in patients with schizophrenia (Sz). However, the functional mechanism translating this into behavior and symptoms remains unclear. It has been proposed that heightened striatal dopamine may blunt dopaminergic reward prediction error (RPE) signaling during reinforcement learning. Methods In this study, we investigated striatal DSC and RPEs and their association in unmedicated Sz and healthy controls. 23 healthy controls (HC) and 20 unmedicated Sz took part in an FDOPA-PET scan measuring DSC and underwent fMRI scanning, where they performed a reversal learning paradigm. We compared groups regarding DSC und neural RPE signals and probed the respective correlation (23 HC and 16 Sz for both measures). Results There was no significant difference between HC and Sz in DSC. Taking into account comorbid alcohol abuse revealed that only patients without such abuse showed elevated DSC in the associative and sensorimotor striatum, while those with abuse did not differ from HC. Patients performed worse during learning, accompanied by a reduced RPE signal in the ventral striatum. In HC, the DSC in the limbic striatum correlated with higher RPE signaling, while there was no significant association in patients. DSC in the associative striatum correlated with higher positive symptoms, and blunted RPE signaling was associated with negative symptoms. Discussion Our results suggest that dopamine modulation of RPE is impaired in schizophrenia. Furthermore, we observed a dissociation with elevated DSC in the associative and sensorimotor striatum contributing to positive symptoms and blunted RPE in the ventral striatum to negative symptoms.

Sign in / Sign up

Export Citation Format

Share Document