The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning

Motivational and perceptual disturbances co-occur in psychosis and have been linked to aberrations in reward learning and sensory gating, respectively. Although traditionally studied independently, when viewed through a predictive coding framework, these processes can both be linked to dysfunction in striatal dopaminergic prediction error signaling. This study examined whether reward learning and sensory gating are correlated in individuals with psychotic disorders, and whether nicotine—a psychostimulant that amplifies phasic striatal dopamine firing—is a common modulator of these two processes. We recruited 183 patients with psychotic disorders (79 schizophrenia, 104 psychotic bipolar disorder) and 129 controls and assessed reward learning (behavioral probabilistic reward task), sensory gating (P50 event-related potential), and smoking history. Reward learning and sensory gating were correlated across the sample. Smoking influenced reward learning and sensory gating in both patient groups; however, the effects were in opposite directions. Specifically, smoking was associated with improved performance in individuals with schizophrenia but impaired performance in individuals with psychotic bipolar disorder. These findings suggest that reward learning and sensory gating are linked and modulated by smoking. However, disorder-specific associations with smoking suggest that nicotine may expose pathophysiological differences in the architecture and function of prediction error circuitry in these overlapping yet distinct psychotic disorders.

Download Full-text

Baseline reward processing and ventrostriatal dopamine function are associated with pramipexole response in depression

Brain ◽

10.1093/brain/awaa002 ◽

2020 ◽

Vol 143 (2) ◽

pp. 701-710 ◽

Cited By ~ 6

Author(s):

Alexis E Whitton ◽

Jenna M Reinen ◽

Mark Slifstein ◽

Yuen-Siang Ang ◽

Patrick J McGrath ◽

...

Keyword(s):

Major Depressive Disorder ◽

Depressive Disorder ◽

Prediction Error ◽

Dopamine Agonists ◽

Dopamine Release ◽

Reward Processing ◽

Reward Learning ◽

Symptom Improvement ◽

Major Depressive ◽

Depressed Group

Abstract The efficacy of dopamine agonists in treating major depressive disorder has been hypothesized to stem from effects on ventrostriatal dopamine and reward function. However, an important question is whether dopamine agonists are most beneficial for patients with reward-based deficits. This study evaluated whether measures of reward processing and ventrostriatal dopamine function predicted response to the dopamine agonist, pramipexole (ClinicalTrials.gov Identifier: NCT02033369). Individuals with major depressive disorder (n = 26) and healthy controls (n = 26) (mean ± SD age = 26.5 ± 5.9; 50% female) first underwent assessments of reward learning behaviour and ventrostriatal prediction error signalling (measured using functional MRI). 11C-(+)-PHNO PET before and after oral amphetamine was used to assess ventrostriatal dopamine release. The depressed group then received open-label pramipexole treatment for 6 weeks (0.5 mg/day titrated to a maximum daily dose of 2.5 mg). Symptoms were assessed weekly, and reward learning was reassessed post-treatment. At baseline, relative to controls, the depressed group showed lower reward learning (P = 0.02), a trend towards blunted reward-related prediction error signals (P = 0.07), and a trend towards increased amphetamine-induced dopamine release (P = 0.07). Despite symptom improvements following pramipexole (Cohen’s d ranging from 0.51 to 2.16 across symptom subscales), reward learning did not change after treatment. At a group level, baseline reward learning (P = 0.001) and prediction error signalling (P = 0.004) were both associated with symptom improvement, albeit in a direction opposite to initial predictions: patients with stronger pretreatment reward learning and reward-related prediction error signalling improved most. Baseline D2/3 receptor availability (P = 0.02) and dopamine release (P = 0.05) also predicted improvements in clinical functioning, with lower D2/3 receptor availability and lower dopamine release predicting greater improvements. Although these findings await replication, they suggest that measures of reward-related mesolimbic dopamine function may hold promise for identifying depressed individuals likely to respond favourably to dopaminergic pharmacotherapy.

Download Full-text

Neural Mechanisms of Reward Prediction Error in Autism Spectrum Disorder

Autism Research and Treatment ◽

10.1155/2019/5469191 ◽

2019 ◽

Vol 2019 ◽

pp. 1-10 ◽

Cited By ~ 3

Author(s):

Maya G. Mosner ◽

R. Edward McLaurin ◽

Jessica L. Kinard ◽

Shabnam Hakimi ◽

Jacob Parelman ◽

...

Keyword(s):

Prediction Error ◽

Autism Spectrum ◽

Neural Mechanisms ◽

Reward Learning ◽

Prediction Errors ◽

Frontal Pole ◽

Neural Basis ◽

Reward Prediction ◽

Behavioral Impairments ◽

Adults With Asd

Few studies have explored neural mechanisms of reward learning in ASD despite evidence of behavioral impairments of predictive abilities in ASD. To investigate the neural correlates of reward prediction errors in ASD, 16 adults with ASD and 14 typically developing controls performed a prediction error task during fMRI scanning. Results revealed greater activation in the ASD group in the left paracingulate gyrus during signed prediction errors and the left insula and right frontal pole during thresholded unsigned prediction errors. Findings support atypical neural processing of reward prediction errors in ASD in frontostriatal regions critical for prediction coding and reward learning. Results provide a neural basis for impairments in reward learning that may contribute to traits common in ASD (e.g., intolerance of unpredictability).

Download Full-text

Akt1 deficiency modulates reward learning and reward prediction error in mice

Genes Brain & Behavior ◽

10.1111/j.1601-183x.2011.00759.x ◽

2012 ◽

Vol 11 (2) ◽

pp. 157-169 ◽

Cited By ~ 8

Author(s):

Y.-C. Chen ◽

Y.-W. Chen ◽

Y.-F. Hsu ◽

W.-T. Chang ◽

C. K. Hsiao ◽

...

Keyword(s):

Prediction Error ◽

Reward Learning ◽

Reward Prediction Error ◽

Reward Prediction

Download Full-text

Reward Prediction Error Signal Enhanced by Striatum-Amygdala Interaction Explains the Acceleration of Probabilistic Reward Learning by Emotion

Journal of Neuroscience ◽

10.1523/jneurosci.3400-12.2013 ◽

2013 ◽

Vol 33 (10) ◽

pp. 4487-4493 ◽

Cited By ~ 37

Author(s):

N. Watanabe ◽

M. Sakagami ◽

M. Haruno

Keyword(s):

Prediction Error ◽

Reward Learning ◽

Error Signal ◽

Reward Prediction Error ◽

Reward Prediction

Download Full-text

Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans

Scientific Reports ◽

10.1038/s41598-017-04507-w ◽

2017 ◽

Vol 7 (1) ◽

Cited By ~ 15

Author(s):

Elsa Fouragnan ◽

Filippo Queirazza ◽

Chris Retzler ◽

Karen J. Mullinger ◽

Marios G. Philiastides

Keyword(s):

Prediction Error ◽

Reward Learning

Download Full-text

Neural Signatures of Reward and Sensory Prediction Error in Motor Learning

10.1101/262576 ◽

2018 ◽

Cited By ~ 4

Author(s):

Dimitrios J. Palidis ◽

Joshua G.A. Cashaback ◽

Paul L. Gribble

Keyword(s):

Visual Feedback ◽

Prediction Error ◽

Motor Adaptation ◽

Learning Task ◽

Reward Learning ◽

P300 Amplitude ◽

Error Feedback ◽

Learning Mechanisms ◽

Visuomotor Rotation ◽

Motor Commands

AbstractAt least two distinct processes have been identified by which motor commands are adapted according to movement-related feedback: reward based learning and sensory error based learning. In sensory error based learning, mappings between sensory targets and motor commands are recalibrated according to sensory error feedback. In reward based learning, motor commands are associated with subjective value, such that successful actions are reinforced. We designed two tasks to isolate reward and sensory error based motor adaptation, and recorded electroencephalography (EEG) from humans to identify and dissociate the neural correlates of reward and sensory error processing. We designed a visuomotor rotation task to isolate sensory error based learning which was induced by altered visual feedback of hand position. In a reward learning task, we isolated reward based learning induced by binary reward feedback that was decoupled from the visual target. We found that a fronto-central event related potential called the feedback related negativity (FRN) was elicited specifically by reward feedback but not sensory error feedback. A more posterior component called the P300 was evoked by feedback in both tasks. In the visuomotor rotation task, P300 amplitude was increased by sensory error induced by perturbed visual feedback, and was correlated with learning rate. In the reward learning task, P300 amplitude was increased by reward relative to non reward and by surprise regardless of feedback valence. We propose that during motor adaptation, the FRN might specifically mark reward prediction error while the P300 might reflect processing which is modulated more generally by prediction error.New and NoteworthyWe studied the event related potentials evoked by feedback stimuli during motor adaptation tasks that isolate reward and sensory error learning mechanisms. We found that the feedback related negativity was specifically elicited by reward feedback, while the P300 was observed in both tasks. These results reveal neural processes associated with different learning mechanisms and elucidate which classes of errors, from a computational standpoint, elicit the FRN and P300.

Download Full-text

Meta-reinforcement learning in a thalamo-orbitofrontal circuit

10.1101/2020.04.28.066878 ◽

2020 ◽

Author(s):

Vijay Mohan K Namboodiri ◽

Taylor Hobbs ◽

Ivan Trujillo Pisanty ◽

Rhiana C Simon ◽

Garret D Stuber

Keyword(s):

Reinforcement Learning ◽

Orbitofrontal Cortex ◽

Calcium Imaging ◽

Prediction Error ◽

Reward Learning ◽

Two Photon ◽

Simultaneous Imaging ◽

Reward Prediction ◽

Biological Substrate ◽

The Difference

Learning to predict rewards is essential for the survival of animals. Contemporary views suggest that such learning is driven by a reward prediction error—the difference between received and predicted rewards. Here we show using two-photon calcium imaging and optogenetics in mice that a different class of reward learning signals exists within the orbitofrontal cortex (OFC). Specifically, the reward responses of many OFC neurons exhibit plasticity consistent with filtering out rewards that are less salient for learning (such as predicted rewards, or, unpredicted rewards available in a context containing highly salient aversive stimuli). We show using quasi-simultaneous imaging and optogenetics that this reward response plasticity is sculpted by medial thalamic inputs to OFC. These results provide a biological substrate for emerging theoretical views of meta-reinforcement learning in prefrontal cortex.

Download Full-text