scholarly journals Forebrain dopamine value signals arise independently from midbrain dopamine cell firing

2018 ◽  
Author(s):  
Ali Mohebi ◽  
Jeffrey Pettibone ◽  
Arif Hamid ◽  
Jenny-Marie Wong ◽  
Robert Kennedy ◽  
...  
Keyword(s):  
Dopamine Release ◽  
Prediction Errors ◽  
Tonic Firing ◽  
Reward Prediction ◽  
Approach Behaviors ◽  
Cell Firing ◽  
Transient Error ◽  
Midbrain Dopamine ◽  
Adaptive Decision Making ◽  
Dopamine Cell

The mesolimbic dopamine projection from the ventral tegmental area (VTA) to nucleus accumbens (NAc) is a key pathway for reward-driven learning, and for the motivation to work for more rewards. VTA dopamine cell firing can encode reward prediction errors (RPEs1,2), vital learning signals in computational theories of adaptive behavior. However, NAc dopamine release more closely resembles reward expectation (value), a motivational signal that invigorates approach behaviors3-7. This discrepancy might be due to distinct behavioral contexts: VTA dopamine cells have been recorded under head-fixed conditions, while NAc dopamine release has been measured in actively-moving subjects. Alternatively the mismatch may reflect changes in the tonic firing of dopamine cells8, or a fundamental dissociation between firing and release. Here we directly compare dopamine cell firing and release in the same adaptive decision-making task. We show that dopamine release covaries with reward expectation in two specific forebrain hotspots, NAc core and ventral prelimbic cortex. Yet the firing rates of optogenetically-identified VTA dopamine cells did not correlate with reward expectation, but instead showed transient, error-like responses to unexpected cues. We conclude that critical motivation-related dopamine dynamics do not arise from VTA dopamine cell firing, and may instead reflect local influences over forebrain dopamine varicosities.

Effects of prefrontal cortex microinjection of neurotensin-(8-13) on midbrain dopamine and non-dopamine cell firing

2000 ◽  
Vol 876 (1-2) ◽  
pp. 196-200 ◽  
Author(s):  
Maria D Fatigati ◽  
Roberta M Anderson ◽  
Pierre-Paul Rompré
Keyword(s):  
Prefrontal Cortex ◽  
Cell Firing ◽  
Midbrain Dopamine ◽  
Dopamine Cell

Noradrenergic modulation of midbrain dopamine cell firing elicited by stimulation of the locus coeruleus in the rat

1993 ◽  
Vol 93 (1) ◽  
pp. 11-25 ◽  
Author(s):  
J. Grenhoff ◽  
M. Nisell ◽  
S. Ferr� ◽  
G. Aston-Jones ◽  
T. H. Svensson
Keyword(s):  
Locus Coeruleus ◽  
Cell Firing ◽  
Midbrain Dopamine ◽  
Noradrenergic Modulation ◽  
Dopamine Cell ◽  
Stimulation Of

scholarly journals Rethinking dopamine as generalized prediction error

2017 ◽  
Author(s):  
Matthew P.H. Gardner ◽  
Geoffrey Schoenbaum ◽  
Samuel J. Gershman
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Dopamine Neurons ◽  
Prediction Errors ◽  
Model Free ◽  
Reward Prediction ◽  
Midbrain Dopamine ◽  
Sensory Prediction ◽  
Lines Of Evidence ◽  
Midbrain Dopamine Neurons

AbstractMidbrain dopamine neurons are commonly thought to report a reward prediction error, as hypothesized by reinforcement learning theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signaling errors in both sensory and reward predictions, dopamine supports a form of reinforcement learning that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and reward prediction errors, while also accounting for new data suggesting a role for these signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.

scholarly journals Dopamine release drives motivation, independently from dopamine cell firing

2019 ◽  
Vol 45 (1) ◽  
pp. 220-220
Author(s):  
Ali Mohebi ◽  
Joshua D. Berke
Keyword(s):  
Dopamine Release ◽  
Cell Firing ◽  
Dopamine Cell

scholarly journals Signed and unsigned reward prediction errors dynamically enhance learning and memory

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
Nina Rouhani ◽  
Yael Niv
Keyword(s):  
Reinforcement Learning ◽  
Locus Coeruleus ◽  
Learning And Memory ◽  
Learning Rate ◽  
Prediction Errors ◽  
Learning Models ◽  
The Past ◽  
Reward Prediction ◽  
Midbrain Dopamine ◽  
Reinforcement Learning Models

Memory helps guide behavior, but which experiences from the past are prioritized? Classic models of learning posit that events associated with unpredictable outcomes as well as, paradoxically, predictable outcomes, deploy more attention and learning for those events. Here, we test reinforcement learning and subsequent memory for those events, and treat signed and unsigned reward prediction errors (RPEs), experienced at the reward-predictive cue or reward outcome, as drivers of these two seemingly contradictory signals. By fitting reinforcement learning models to behavior, we find that both RPEs contribute to learning by modulating a dynamically changing learning rate. We further characterize the effects of these RPE signals on memory, and show that both signed and unsigned RPEs enhance memory, in line with midbrain dopamine and locus-coeruleus modulation of hippocampal plasticity, thereby reconciling separate findings in the literature.

scholarly journals A history-derived reward prediction error signal in ventral pallidum

2019 ◽  
Author(s):  
David J. Ottenheimer ◽  
Bilal A. Bari ◽  
Elissa Sutlief ◽  
Kurt M. Fraser ◽  
Tabitha H. Kim ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Computational Models ◽  
Ventral Pallidum ◽  
Neural Population ◽  
Learning Activity ◽  
Prediction Errors ◽  
Dopamine System ◽  
Reward Seeking ◽  
Reward Prediction ◽  
Midbrain Dopamine

ABSTRACTLearning from past interactions with the environment is critical for adaptive behavior. Within the framework of reinforcement learning, the nervous system builds expectations about future reward by computing reward prediction errors (RPEs), the difference between actual and predicted rewards. Correlates of RPEs have been observed in the midbrain dopamine system, which is thought to locally compute this important variable in service of learning. However, the extent to which RPE signals may be computed upstream of the dopamine system is largely unknown. Here, we quantify history-based RPE signals in the ventral pallidum (VP), an input region to the midbrain dopamine system implicated in reward-seeking behavior. We trained rats to associate cues with future delivery of reward and fit computational models to predict individual neuron firing rates at the time of reward delivery. We found that a subset of VP neurons encoded RPEs and did so more robustly than nucleus accumbens, an input to VP. VP RPEs predicted trial-by-trial task engagement, and optogenetic inhibition of VP reduced subsequent task-related reward seeking. Consistent with reinforcement learning, activity of VP RPE cells adapted when rewards were delivered in blocks. We further found that history- and cue-based RPEs were largely separate across the VP neural population. The presence of behaviorally-instructive RPE signals in the VP suggests a pivotal role for this region in value-based computations.

scholarly journals Occasion setters determine responses of putative dopamine neurons to discriminative stimuli

2019 ◽  
Author(s):  
Luca Aquili ◽  
Eric M. Bowman ◽  
Robert Schmidt
Keyword(s):  
Single Unit ◽  
Population Level ◽  
Dopamine Neurons ◽  
Prediction Errors ◽  
Occasion Setting ◽  
Population Response ◽  
Unit Level ◽  
Reward Prediction ◽  
Midbrain Dopamine ◽  
The Relationship

AbstractMidbrain dopamine (DA) neurons are involved in the processing of rewards and reward-predicting stimuli, possibly analogous to reinforcement learning reward prediction errors. Here we studied the activity of putative DA neurons (n=41) recorded in the ventral tegmental area of rats (n=6) performing a behavioural task involving occasion setting. In this task an occasion setter (OS) indicated that the relationship between a discriminative stimulus (DS) and reinforcement is in effect, so that reinforcement of bar pressing occurred only after the OS (tone or houselight) was followed by the DS (houselight or tone). We found that responses of putative DA cells to the DS were enhanced when preceded by the OS, as were behavioural responses to obtain rewards. Surprisingly though, we did not find a population response of putative DA neurons to the OS, contrary to predictions of standard temporal-difference models of DA neurons. However, despite the absence of a population response, putative DA neurons exhibited a heterogeneous response on a single unit level, so that some units increased and others decreased their activity as a response to the OS. Similarly, putative non-DA cells did not respond to the DS on a population level, but with heterogeneous responses on a single unit level. The heterogeneity in the responses of putative DA cells may reflect how DA neurons encode context and point to local differences in DA signalling.

scholarly journals Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework

eLife ◽  
2016 ◽  
Vol 5 ◽  
Author(s):  
Brian F Sadacca ◽  
Joshua L Jones ◽  
Geoffrey Schoenbaum
Keyword(s):  
Dopamine Neurons ◽  
Prediction Errors ◽  
Temporal Difference ◽  
Use Value ◽  
Value Prediction ◽  
Reward Prediction ◽  
Common Framework ◽  
Midbrain Dopamine ◽  
Midbrain Dopamine Neurons ◽  
Do So

Midbrain dopamine neurons have been proposed to signal reward prediction errors as defined in temporal difference (TD) learning algorithms. While these models have been extremely powerful in interpreting dopamine activity, they typically do not use value derived through inference in computing errors. This is important because much real world behavior – and thus many opportunities for error-driven learning – is based on such predictions. Here, we show that error-signaling rat dopamine neurons respond to the inferred, model-based value of cues that have not been paired with reward and do so in the same framework as they track the putative cached value of cues previously paired with reward. This suggests that dopamine neurons access a wider variety of information than contemplated by standard TD models and that, while their firing conforms to predictions of TD models in some cases, they may not be restricted to signaling errors from TD predictions.

scholarly journals Rethinking dopamine as generalized prediction error

2018 ◽  
Vol 285 (1891) ◽  
pp. 20181645 ◽  
Author(s):  
Matthew P. H. Gardner ◽  
Geoffrey Schoenbaum ◽  
Samuel J. Gershman
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Dopamine Neurons ◽  
Prediction Errors ◽  
Model Free ◽  
Reward Prediction ◽  
Midbrain Dopamine ◽  
Sensory Prediction ◽  
Lines Of Evidence ◽  
Midbrain Dopamine Neurons

Midbrain dopamine neurons are commonly thought to report a reward prediction error (RPE), as hypothesized by reinforcement learning (RL) theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here, we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signalling errors in both sensory and reward predictions, dopamine supports a form of RL that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and RPEs, while also accounting for new data suggesting a role for these signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.

Sign in / Sign up

Export Citation Format

Share Document