"Learning in reverse: Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner".

For over two decades, midbrain dopamine was considered synonymous with the prediction error in temporal-difference reinforcement learning. Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received allowing for the calculation of prediction errors. Phasic firing of dopamine neurons was proposed to reflect this computation, facilitating the backpropagation of value from the predicted reward to the reward-predictive stimulus, thus reducing future prediction errors. There are two critical assumptions of this proposal: 1) that dopamine errors can only facilitate learning about scalar value and not more complex features of predicted rewards, and 2) that the dopamine signal can only be involved in anticipatory learning in which cues or actions precede rewards. Recent work has challenged the first assumption, demonstrating that phasic dopamine signals across species are involved in learning about more complex features of the predicted outcomes, in a manner that transcends this value computation. Here, we tested the validity of the second assumption. Specifically, we examined whether phasic midbrain dopamine activity would be necessary for backward conditioning- when a neutral cue reliably follows a rewarding outcome. Using a specific Pavlovian-to-Instrumental Transfer (PIT) procedure, we show rats learn both excitatory and inhibitory components of a backward association, and that this association entails knowledge of the specific identity of the reward and cue. We demonstrate that brief optogenetic inhibition of ventral tegmental area dopamine (VTA DA) neurons timed to the transition between the reward and cue, reduces both of these components of backward conditioning. These findings suggest VTA DA neurons are capable of facilitating associations between contiguously occurring events, regardless of the content of those events. We conclude that these data are in line with suggestions that the VTA DA error acts as a universal teaching signal. This may provide insight into why dopamine function has been implicated in a myriad of psychological disorders that are characterized by very distinct reinforcement-learning deficits.

Associative mechanisms involved in specific Pavlovian-to-instrumental transfer in human learning tasks

Four experiments compared the effect of forward and backward conditioning procedures on the ability of conditioned stimuli (CS) to elevate instrumental responding in a Pavlovian-to-instrumental transfer (PIT) task. Two responses were each trained with one distinct outcome (R1->O1, R2->O2), either concurrently (Experiment 1) or separately (Experiments 2, 3 and 4). Then, in Experiments 1 and 2, four CSs were either followed or preceded by one outcome (A->O1, B->O2, O1->C, O2->D). In Experiment 3, each CS was preceded and followed by an outcome: for one group of participants, both outcomes were identical (e.g., O1->A->O1, O2->B->O2), but for the other, they were different (e.g., O1->A->O2, O2->B->O1). In Experiment 4, two CSs were preceded and followed by identical outcomes, and two CSs by different outcomes. In the PIT tests, participants performed R1 and R2 in the presence and absence of the CSs. In Experiments 1 and 2, only the CSs followed by outcomes in Pavlovian training elevated responding. In Experiments 3 and 4, all the CSs elevated responding but based on the outcome that followed them in training. These results support the stimulus-outcome-response (S-O-R) mechanism of specific PIT, according to which CSs elevate responding via activation of its associated outcome representation.

Glucose Intake Alters Expression of Neuropeptides Derived from Proopiomelanocortin in the Lateral Hypothalamus and the Nucleus Accumbens in Fructose Preference Rats

To study the neuroendocrine mechanism of sugar preference, we investigated the role of glucose feeding in the regulation of expression levels of neuropeptides derived from proopiomelanocortin (POMC) in the lateral hypothalamus (LH) and nucleus accumbens (NAc) in fructose preference rats. Fructose preference rats were induced by using the lithium chloride backward conditioning procedure. The fructose preference was confirmed by the two-bottle test. The drinking behavior of rats was assessed by the fructose concentration gradient test. The preference of 10% glucose or 0.1% saccharine was assessed, and the expression levels of neuropeptides derived from POMC in the LH and the NAc in fructose preference rats were measured by Western blot analysis. Fructose preference rats displayed a greater fructose preference than control rats. Furthermore, fructose preference rats preferred glucose solution rather than saccharine solution, while control rats preferred saccharine solution rather than glucose solution. The expression levels of neuropeptides derived from POMC in the LH and the NAc were changed by glucose but not saccharine intake. In summary, the data suggests that glucose intake increases the expression of neuropeptides derived from POMC in the LH and the NAc in fructose preference rats.