scholarly journals All That Glitters … Dissociating Attention and Outcome Expectancy From Prediction Errors Signals

2010 ◽  
Vol 104 (2) ◽  
pp. 587-595 ◽  
Author(s):  
Matthew R. Roesch ◽  
Donna J. Calu ◽  
Guillem R. Esber ◽  
Geoffrey Schoenbaum
Keyword(s):  
Orbitofrontal Cortex ◽  
Basolateral Amygdala ◽  
Ventral Striatum ◽  
Recent Literature ◽  
Neural Correlates ◽  
Dopamine Neurons ◽  
Outcome Expectancy ◽  
Prediction Errors ◽  
Signal Prediction ◽  
Unexpected Outcomes

Initially reported in dopamine neurons, neural correlates of prediction errors have now been shown in a variety of areas, including orbitofrontal cortex, ventral striatum, and amygdala. Yet changes in neural activity to an outcome or cues that precede it can reflect other processes. We review the recent literature and show that although activity in dopamine neurons appears to signal prediction errors, similar activity in orbitofrontal cortex, basolateral amygdala, and ventral striatum does not. Instead, increased firing in basolateral amygdala to unexpected outcomes likely reflects attention, whereas activity in orbitofrontal cortex and ventral striatum is unaffected by prior expectations and may provide information on outcome expectancy. These results have important implications for how these areas interact to facilitate learning and guide behavior.

scholarly journals Dopamine transients delivered in learning contexts do not act as model-free prediction errors

2019 ◽  
Author(s):  
Melissa J. Sharpe ◽  
Hannah M. Batchelor ◽  
Lauren E. Mueller ◽  
Chun Yun Chang ◽  
Etienne J.P. Maes ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Associative Learning ◽  
Prediction Error ◽  
Error Term ◽  
Neural Correlates ◽  
Dopamine Neurons ◽  
Prediction Errors ◽  
Model Free ◽  
Reward Prediction ◽  
Excess Value

AbstractDopamine neurons fire transiently in response to unexpected rewards. These neural correlates are proposed to signal the reward prediction error described in model-free reinforcement learning algorithms. This error term represents the unpredicted or ‘excess’ value of the rewarding event. In model-free reinforcement learning, this value is then stored as part of the learned value of any antecedent cues, contexts or events, making them intrinsically valuable, independent of the specific rewarding event that caused the prediction error. In support of equivalence between dopamine transients and this model-free error term, proponents cite causal optogenetic studies showing that artificially induced dopamine transients cause lasting changes in behavior. Yet none of these studies directly demonstrate the presence of cached value under conditions appropriate for associative learning. To address this gap in our knowledge, we conducted three studies where we optogenetically activated dopamine neurons while rats were learning associative relationships, both with and without reward. In each experiment, the antecedent cues failed to acquired value and instead entered into value-independent associative relationships with the other cues or rewards. These results show that dopamine transients, constrained within appropriate learning situations, support valueless associative learning.

scholarly journals Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral Striatum

Neuron ◽  
2016 ◽  
Vol 91 (1) ◽  
pp. 182-193 ◽  
Author(s):  
Yuji K. Takahashi ◽  
Angela J. Langdon ◽  
Yael Niv ◽  
Geoffrey Schoenbaum
Keyword(s):  
Ventral Striatum ◽  
Dopamine Neurons ◽  
Prediction Errors ◽  
Temporal Specificity ◽  
Reward Prediction

The basal ganglia

2020 ◽  
pp. 468-496
Author(s):  
Edmund T. Rolls
Keyword(s):  
Basal Ganglia ◽  
Orbitofrontal Cortex ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Dopamine Neurons ◽  
Stimulus Response ◽  
Cortical Areas ◽  
Related Information ◽  
Single Output ◽  
Selection Of

The basal ganglia include the striatum (caudate, putamen, and ventral striatum) which receive from all cortical areas, and which project via the globus pallidus and substantia nigra back to the neocortex. The basal ganglia are implicated in stimulus-response habit learning, which may be provided by a reinforcement learning signal received by dopamine neurons responding to reward prediction error. The dopamine neurons may receive reward-related information from the orbitofrontal cortex, via the ventral striatum and habenula. The network mechanisms in the basal ganglia implement selection of a single output for behaviour, which is highly adaptive, by mutual direct inhibition between neurons.

scholarly journals A VTA GABAergic computational model of dissociated reward prediction error computation in classical conditioning

2020 ◽  
Author(s):  
Pramod Kaushik ◽  
Jérémie Naudé ◽  
Surampudi Bapi Raju ◽  
Frédéric Alexandre
Keyword(s):  
Classical Conditioning ◽  
Computational Model ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Dopamine Neurons ◽  
System Level ◽  
Reward Prediction Error ◽  
Twin Peaks ◽  
Reward Prediction ◽  
Error Computation

AbstractClassical Conditioning is a fundamental learning mechanism where the Ventral Striatum is generally thought to be the source of inhibition to Ventral Tegmental Area (VTA) Dopamine neurons when a reward is expected. However, recent evidences point to a new candidate in VTA GABA encoding expectation for computing the reward prediction error in the VTA. In this system-level computational model, the VTA GABA signal is hypothesised to be a combination of magnitude and timing computed in the Peduncolopontine and Ventral Striatum respectively. This dissociation enables the model to explain recent results wherein Ventral Striatum lesions affected the temporal expectation of the reward but the magnitude of the reward was intact. This model also exhibits other features in classical conditioning namely, progressively decreasing firing for early rewards closer to the actual reward, twin peaks of VTA dopamine during training and cancellation of US dopamine after training.

scholarly journals Midbrain dopamine neurons signal aversion in a reward-context-dependent manner

eLife ◽  
2016 ◽  
Vol 5 ◽  
Author(s):  
Hideyuki Matsumoto ◽  
Ju Tian ◽  
Naoshige Uchida ◽  
Mitsuko Watabe-Uchida
Keyword(s):  
Dopamine Neurons ◽  
Prediction Errors ◽  
Dependent Manner ◽  
Value Prediction ◽  
Signal Value ◽  
Aversive Events ◽  
High Reward ◽  
Midbrain Dopamine ◽  
Dopamine Signaling ◽  
Reward And Punishment

Dopamine is thought to regulate learning from appetitive and aversive events. Here we examined how optogenetically-identified dopamine neurons in the lateral ventral tegmental area of mice respond to aversive events in different conditions. In low reward contexts, most dopamine neurons were exclusively inhibited by aversive events, and expectation reduced dopamine neurons’ responses to reward and punishment. When a single odor predicted both reward and punishment, dopamine neurons’ responses to that odor reflected the integrated value of both outcomes. Thus, in low reward contexts, dopamine neurons signal value prediction errors (VPEs) integrating information about both reward and aversion in a common currency. In contrast, in high reward contexts, dopamine neurons acquired a short-latency excitation to aversive events that masked their VPE signaling. Our results demonstrate the importance of considering the contexts to examine the representation in dopamine neurons and uncover different modes of dopamine signaling, each of which may be adaptive for different environments.

scholarly journals Separable Dorsal Raphe Dopamine Projections Mediate Sociability and Valence

2021 ◽  
Author(s):  
Kay Tye ◽  
Gillian Matthews ◽  
Mackenzie Lemieux ◽  
Elizabeth Brewer ◽  
Raymundo Miranda ◽  
...  
Keyword(s):  
Basolateral Amygdala ◽  
Ex Vivo ◽  
Affective State ◽  
Well Being ◽  
Dorsal Raphe ◽  
Receptor Expression ◽  
Dopamine Neurons ◽  
Bed Nucleus ◽  
Distributed Target ◽  
Dorsal Raphé

Abstract Affiliative social connections facilitate well-being and survival in numerous species. Engaging in social interactions requires positive and negative motivational drive, elicited through coordinated activity across neural circuits. However, the identity, interconnectivity, and functional encoding of social information within these circuits remains poorly understood. Here, we focused on downstream projections of dorsal raphe nucleus (DRN) dopamine neurons (DRNDAT), which we previously implicated in ‘negative drive’-induced social motivation. We show that three prominent DRNDAT projections – to the bed nucleus of the stria terminalis (BNST), central amygdala (CeA), and posterior basolateral amygdala (BLP) – play separable roles in behavior, despite substantial collateralization. Photoactivation of the DRNDAT-CeA projection promoted social behavior and photoactivation of the DRNDAT-BNST projection promoted exploratory behavior, while the DRNDAT-BLP projection supported place avoidance, suggesting a negative affective state. Downstream regions showed diverse, region-specific, receptor expression, poising DRNDAT neurons to act through dopamine, neuropeptide, and glutamate transmission. Furthermore, we show ex vivo that the effect of DRNDAT photostimulation on downstream neuron excitability was predicted by baseline cell properties, suggesting cell-type-specific modulation. Collectively, these data indicate that DRNDAT neurons may bias behavior via precise modulation of cellular activity in broadly-distributed target structures.

scholarly journals Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

2015 ◽  
Vol 113 (1) ◽  
pp. 200-205 ◽  
Author(s):  
Kenneth T. Kishida ◽  
Ignacio Saez ◽  
Terry Lohrenz ◽  
Mark R. Witcher ◽  
Adrian W. Laxton ◽  
...  
Keyword(s):  
Parkinson’S Disease ◽  
Decision Making ◽  
Parkinson's Disease ◽  
Animal Models ◽  
Large Body ◽  
Reward Processing ◽  
Dopamine Neurons ◽  
Model Organisms ◽  
Mammalian Brain ◽  
Prediction Errors

In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson’s disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson’s disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.

B.7 - 5-HT2A RECEPTORS IN THE ORBITOFRONTAL CORTEX AND THE BASOLATERAL AMYGDALA MODULATE IMPULSIVE DECISION-MAKING IN RATS

2013 ◽  
Vol 24 ◽  
pp. e28
Author(s):  
Lena Wischhof ◽  
Kerstin Wernecke ◽  
Ellen Irrsack ◽  
Malte Feja ◽  
Michael Koch
Keyword(s):  
Decision Making ◽  
Orbitofrontal Cortex ◽  
Basolateral Amygdala
Sign in / Sign up

Export Citation Format

Share Document