Neural Coding of Reward-Prediction Error Signals During Classical Conditioning With Attractive Faces

2007 ◽  
Vol 97 (4) ◽  
pp. 3036-3045 ◽  
Author(s):  
Signe Bray ◽  
John O'Doherty
Keyword(s):  
Neural Coding ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Human Subjects ◽  
Visual Stimuli ◽  
Dopamine Neurons ◽  
Error Signal ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Attractive Female

Attractive faces can be considered to be a form of visual reward. Previous imaging studies have reported activity in reward structures including orbitofrontal cortex and nucleus accumbens during presentation of attractive faces. Given that these stimuli appear to act as rewards, we set out to explore whether it was possible to establish conditioning in human subjects by pairing presentation of arbitrary affectively neutral stimuli with subsequent presentation of attractive and unattractive faces. Furthermore, we scanned human subjects with functional magnetic resonance imaging (fMRI) while they underwent this conditioning procedure to determine whether a reward-prediction error signal is engaged during learning with attractive faces as is known to be the case for learning with other types of reward such as juice and money. Subjects showed changes in behavioral ratings to the conditioned stimuli (CS) when comparing post- to preconditioning evaluations, notably for those CSs paired with attractive female faces. We used a simple Rescorla-Wagner learning model to generate a reward-prediction error signal and entered this into a regression analysis with the fMRI data. We found significant prediction error-related activity in the ventral striatum during conditioning with attractive compared with unattractive faces. These findings suggest that an arbitrary stimulus can acquire conditioned value by being paired with pleasant visual stimuli just as with other types of reward such as money or juice. This learning process elicits a reward-prediction error signal in a main target structure of dopamine neurons: the ventral striatum. The findings we describe here may provide insights into the neural mechanisms tapped into by advertisers seeking to influence behavioral preferences by repeatedly exposing consumers to simple associations between products and rewarding visual stimuli such as pretty faces.

scholarly journals A VTA GABAergic computational model of dissociated reward prediction error computation in classical conditioning

2020 ◽  
Author(s):  
Pramod Kaushik ◽  
Jérémie Naudé ◽  
Surampudi Bapi Raju ◽  
Frédéric Alexandre
Keyword(s):  
Classical Conditioning ◽  
Computational Model ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Dopamine Neurons ◽  
System Level ◽  
Reward Prediction Error ◽  
Twin Peaks ◽  
Reward Prediction ◽  
Error Computation

AbstractClassical Conditioning is a fundamental learning mechanism where the Ventral Striatum is generally thought to be the source of inhibition to Ventral Tegmental Area (VTA) Dopamine neurons when a reward is expected. However, recent evidences point to a new candidate in VTA GABA encoding expectation for computing the reward prediction error in the VTA. In this system-level computational model, the VTA GABA signal is hypothesised to be a combination of magnitude and timing computed in the Peduncolopontine and Ventral Striatum respectively. This dissociation enables the model to explain recent results wherein Ventral Striatum lesions affected the temporal expectation of the reward but the magnitude of the reward was intact. This model also exhibits other features in classical conditioning namely, progressively decreasing firing for early rewards closer to the actual reward, twin peaks of VTA dopamine during training and cancellation of US dopamine after training.

scholarly journals Recent advances in understanding the role of phasic dopamine activity

F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 1680 ◽  
Author(s):  
Wolfram Schultz
Keyword(s):  
Consumer Choice ◽  
Prediction Error ◽  
Behavioral Activation ◽  
Early Response ◽  
Dopamine Neurons ◽  
Error Signal ◽  
Spatial Choice ◽  
Response Component ◽  
Reward Prediction Error ◽  
Reward Prediction

The latest animal neurophysiology has revealed that the dopamine reward prediction error signal drives neuronal learning in addition to behavioral learning and reflects subjective reward representations beyond explicit contingency. The signal complies with formal economic concepts and functions in real-world consumer choice and social interaction. An early response component is influenced by physical impact, reward environment, and novelty but does not fully code prediction error. Some dopamine neurons are activated by aversive stimuli, which may reflect physical stimulus impact or true aversiveness, but they do not seem to code general negative value or aversive prediction error. The reward prediction error signal is complemented by distinct, heterogeneous, smaller and slower changes reflecting sensory and motor contributors to behavioral activation, such as substantial movement (as opposed to precise motor control), reward expectation, spatial choice, vigor, and motivation. The different dopamine signals seem to defy a simple unifying concept and should be distinguished to better understand phasic dopamine functions.

scholarly journals Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice

eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
William Menegas ◽  
Benedicte M Babayan ◽  
Naoshige Uchida ◽  
Mitsuko Watabe-Uchida
Keyword(s):  
Classical Conditioning ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Dopamine Neurons ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Strong Excitation ◽  
Dopamine Signaling ◽  
Predicted Values ◽  
Over Time

Dopamine neurons are thought to encode novelty in addition to reward prediction error (the discrepancy between actual and predicted values). In this study, we compared dopamine activity across the striatum using fiber fluorometry in mice. During classical conditioning, we observed opposite dynamics in dopamine axon signals in the ventral striatum (‘VS dopamine’) and the posterior tail of the striatum (‘TS dopamine’). TS dopamine showed strong excitation to novel cues, whereas VS dopamine showed no responses to novel cues until they had been paired with a reward. TS dopamine cue responses decreased over time, depending on what the cue predicted. Additionally, TS dopamine showed excitation to several types of stimuli including rewarding, aversive, and neutral stimuli whereas VS dopamine showed excitation only to reward or reward-predicting cues. Together, these results demonstrate that dopamine novelty signals are localized in TS along with general salience signals, while VS dopamine reliably encodes reward prediction error.

scholarly journals O5.1. STRIATAL DOPAMINE AND REDUCED REWARD PREDICTION ERROR SIGNALING IN UNMEDICATED SCHIZOPHRENIA PATIENTS

2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S11-S11
Author(s):  
Teresa Katthagen ◽  
Jakob Kaminski ◽  
Andreas Heinz ◽  
Ralph Buchert ◽  
Florian Schlagenhauf
Keyword(s):  
Prediction Error ◽  
Negative Symptoms ◽  
Ventral Striatum ◽  
Striatal Dopamine ◽  
Dopamine Synthesis ◽  
Positive Symptoms ◽  
Healthy Controls ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Significant Difference

Abstract Background Increased striatal dopamine synthesis capacity (DSC) has consistently been reported in patients with schizophrenia (Sz). However, the functional mechanism translating this into behavior and symptoms remains unclear. It has been proposed that heightened striatal dopamine may blunt dopaminergic reward prediction error (RPE) signaling during reinforcement learning. Methods In this study, we investigated striatal DSC and RPEs and their association in unmedicated Sz and healthy controls. 23 healthy controls (HC) and 20 unmedicated Sz took part in an FDOPA-PET scan measuring DSC and underwent fMRI scanning, where they performed a reversal learning paradigm. We compared groups regarding DSC und neural RPE signals and probed the respective correlation (23 HC and 16 Sz for both measures). Results There was no significant difference between HC and Sz in DSC. Taking into account comorbid alcohol abuse revealed that only patients without such abuse showed elevated DSC in the associative and sensorimotor striatum, while those with abuse did not differ from HC. Patients performed worse during learning, accompanied by a reduced RPE signal in the ventral striatum. In HC, the DSC in the limbic striatum correlated with higher RPE signaling, while there was no significant association in patients. DSC in the associative striatum correlated with higher positive symptoms, and blunted RPE signaling was associated with negative symptoms. Discussion Our results suggest that dopamine modulation of RPE is impaired in schizophrenia. Furthermore, we observed a dissociation with elevated DSC in the associative and sensorimotor striatum contributing to positive symptoms and blunted RPE in the ventral striatum to negative symptoms.

scholarly journals Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons

2018 ◽  
Author(s):  
Rachel S. Lee ◽  
Marcelo G. Mattar ◽  
Nathan F. Parker ◽  
Ilana B. Witten ◽  
Nathaniel D. Daw
Keyword(s):  
Prediction Error ◽  
Dopamine Neurons ◽  
Movement Direction ◽  
Related Activity ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Trial Basis ◽  
Midbrain Dopamine ◽  
Shed Light ◽  
Dorsomedial Striatum

AbstractAlthough midbrain dopamine (DA) neurons have been thought to primarily encode reward prediction error (RPE), recent studies have also found movement-related DAergic signals. For example, we recently reported that DA neurons in mice projecting to dorsomedial striatum are modulated by choices contralateral to the recording side. Here, we introduce, and ultimately reject, a candidate resolution for the puzzling RPE vs movement dichotomy, by showing how seemingly movement-related activity might be explained by an action-specific RPE. By considering both choice and RPE on a trial-by-trial basis, we find that DA signals are modulated by contralateral choice in a manner that is distinct from RPE, implying that choice encoding is better explained by movement direction. This fundamental separation between RPE and movement encoding may help shed light on the diversity of functions and dysfunctions of the DA system.

Sign in / Sign up

Export Citation Format

Share Document