Dorsal Bed Nucleus of the Stria Terminalis-Subcortical Output Circuits Encode Positive Bias in Pavlovian Fear and Reward

Opposite emotions like fear and reward states often utilize the same brain regions. The bed nucleus of the stria terminalis (BNST) comprises one hub for processing fear and reward processes. However, it remains unknown how dorsal BNST (dBNST) circuits process these antagonistic behaviors. Here, we exploited a combined Pavlovian fear and reward conditioning task that exposed mice to conditioned tone stimuli (CS)s, either paired with sucrose delivery or footshock unconditioned stimuli (US). Pharmacological inactivation identified the dorsal BNST as a crucial element for both fear and reward behavior. Deep brain calcium imaging revealed opposite roles of two distinct dBNST neuronal output pathways to the periaqueductal gray (PAG) or paraventricular hypothalamus (PVH). dBNST neural activity profiles differentially process valence and Pavlovian behavior components: dBNST-PAG neurons encode fear CS, whereas dBNST-PVH neurons encode reward responding. Optogenetic activation of BNST-PVH neurons increased reward seeking, whereas dBNST-PAG neurons attenuated freezing. Thus, dBNST-PVH or dBNST-PAG circuitry encodes oppositely valenced fear and reward states, while simultaneously triggering an overall positive affective response bias (increased reward seeking while reducing fear responses). We speculate that this mechanism amplifies reward responding and suppresses fear responses linked to BNST dysfunction in stress and addictive behaviors.

Specialized coding patterns among dorsomedial prefrontal neuronal ensembles predict conditioned reward seeking

Non-overlapping cell populations within dorsomedial prefrontal cortex (dmPFC), defined by gene expression or projection target, control dissociable aspects of reward seeking through unique activity patterns. However, even within these defined cell populations considerable cell-to-cell variability is found, suggesting that greater resolution is needed to understand information processing in dmPFC. Here we use two-photon calcium imaging in awake, behaving mice to monitor the activity of dmPFC excitatory neurons throughout Pavlovian reward conditioning. We characterize five unique neuronal ensembles that each encode specialized information related to a sucrose reward, reward-predictive cues, and behavioral responses to those cues. The ensembles differentially emerge across daily training sessions - and stabilize after learning - in a manner that improves the predictive validity of dmPFC activity dynamics for deciphering variables related to behavioral conditioning. Our results characterize the complex dmPFC neuronal ensemble dynamics that stably predict reward availability and initiation of conditioned reward seeking following cue-reward learning.

A bidirectional corticoamygdala circuit for the encoding and retrieval of detailed reward memories

Adaptive reward-related decision making often requires accurate and detailed representation of potential available rewards. Environmental reward-predictive stimuli can facilitate these representations, allowing one to infer which specific rewards might be available and choose accordingly. This process relies on encoded relationships between the cues and the sensory-specific details of the reward they predict. Here we interrogated the function of the basolateral amygdala (BLA) and its interaction with the lateral orbitofrontal cortex (lOFC) in the ability to learn such stimulus-outcome associations and use these memories to guide decision making. Using optical recording and inhibition approaches, Pavlovian cue-reward conditioning, and the outcome-selective Pavlovian-to-instrumental transfer (PIT) test in male rats, we found that the BLA is robustly activated at the time of stimulus-outcome learning and that this activity is necessary for sensory-specific stimulus-outcome memories to be encoded, so they can subsequently influence reward choices. Direct input from the lOFC was found to support the BLA in this function. Based on prior work, activity in BLA projections back to the lOFC was known to support the use of stimulus-outcome memories to influence decision making. By multiplexing optogenetic and chemogenetic inhibition we performed a serial circuit disconnection and found that the lOFCàBLA and BLAàlOFC pathways form a functional circuit regulating the encoding (lOFCàBLA) and subsequent use (BLAàlOFC) of the stimulus-dependent, sensory-specific reward memories that are critical for adaptive, appetitive decision making.

A bidirectional corticoamygdala circuit for the encoding and retrieval of detailed reward memories

Adaptive reward-related decision making often requires accurate and detailed representation of potential available rewards. Environmental reward-predictive stimuli can facilitate these representations, allowing one to infer which specific rewards might be available and choose accordingly. This process relies on encoded relationships between the cues and the sensory-specific details of the reward they predict. Here we interrogated the function of the basolateral amygdala (BLA) and its interaction with the lateral orbitofrontal cortex (lOFC) in the ability to learn such stimulus-outcome associations and use these memories to guide decision making. Using optical recording and inhibition approaches, Pavlovian cue-reward conditioning, and an outcome-selective Pavlovian-to-instrumental transfer (PIT) test in male rats, we found that the BLA is robustly activated at the time of stimulus-outcome learning and that this activity is necessary for sensory-specific stimulus-outcome memories to be encoded, so that they can subsequently influence reward choices. Direct input from the lOFC was found to support the BLA in this function. Based on prior work, activity in BLA projections back to the lOFC was known to support the use of stimulus-outcome memories to influence decision making. By multiplexing optogenetic and chemogenetic inhibition to perform a serial circuit disconnection, we found that activity in lOFC→BLA projections regulates the encoding of the same components of the stimulus-outcome memory that are later used to allow cues to guide choice via activity in BLA→lOFC projections. Thus, the lOFC→BLA→lOFC circuit regulates the encoding (lOFC→BLA) and subsequent use (BLA→lOFC) of the stimulus-dependent, sensory-specific reward memories that are critical for adaptive, appetitive decision making.

Visual Cues Predictive of Behaviorally Neutral Outcomes Evoke Persistent but Not Interval Timing Activity in V1, Whereas Aversive Conditioning Suppresses This Activity

Cue-evoked persistent activity is neural activity that persists beyond stimulation of a sensory cue and has been described in many regions of the brain, including primary sensory areas. Nonetheless, the functional role that persistent activity plays in primary sensory areas is enigmatic. However, one form of persistent activity in a primary sensory area is the representation of time between a visual stimulus and a water reward. This “reward timing activity”—observed within the primary visual cortex—has been implicated in informing the timing of visually cued, reward-seeking actions. Although rewarding outcomes are sufficient to engender interval timing activity within V1, it is unclear to what extent cue-evoked persistent activity exists outside of reward conditioning, and whether temporal relationships to other outcomes (such as behaviorally neutral or aversive outcomes) are able to engender timing activity. Here we describe the existence of cue-evoked persistent activity in mouse V1 following three conditioning strategies: pseudo-conditioning (where unpaired, monocular visual stimuli are repeatedly presented to an animal), neutral conditioning (where monocular visual stimuli are paired with a binocular visual stimulus, at a delay), and aversive conditioning (where monocular visual stimuli are paired with a tail shock, at a delay). We find that these conditioning strategies exhibit persistent activity that takes one of three forms, a sustained increase of activity; a sustained decrease of activity; or a delayed, transient peak of activity, as previously observed following conditioning with delayed reward. However, these conditioning strategies do not result in visually cued interval timing activity, as observed following appetitive conditioning. Moreover, we find that neutral conditioning increases the magnitude of cue-evoked responses whereas aversive conditioning strongly diminished both the response magnitude and the prevalence of cue-evoked persistent activity. These results demonstrate that cue-evoked persistent activity within V1 can exist outside of conditioning visual stimuli with delayed outcomes and that this persistent activity can be uniquely modulated across different conditioning strategies using unconditioned stimuli of varying behavioral relevance. Together, these data extend our understanding of cue-evoked persistent activity within a primary sensory cortical network and its ability to be modulated by salient outcomes.

Synaptic inhibition in the lateral habenula shapes reward anticipation

The nervous system can associate neutral cues with rewards to promote appetitive adaptive behaviors. The lateral habenula (LHb) contributes to such behaviors as rewards and reward-predictive cues inhibit this structure and engage LHb-to-dopamine circuits. However, the mechanistic understanding of reward encoding within the LHb remains unknown. We report that, in mice, acquisition of anticipatory licking in a reward-conditioning task potentiates postsynaptic GABAergic transmission, leaving excitatory synapses unaffected. Conversely, LHb-targeted manipulations of postsynaptic GABAergic function via pharmacological blockade or impairment of GABAA receptor trafficking decrease anticipatory licking. Hence, inhibitory signaling within LHb enables the expression of appetitive behaviors.

Nucleus-accumbens dopamine tracks aversive stimulus duration and prediction but not value or prediction error

ABSTRACTThe role of dopamine in processing aversive stimuli is under debate: Credits range from no involvement at all, to acting as a punishment-prediction error (PPE) signal. Here, we systematically investigated dopamine release in the nucleus-accumbens core (NAC), which is closely linked to reward-prediction errors, in rats that were exposed to white noise (WN), a versatile, underutilized aversive stimulus, and its predictive cues. Both induced a negative dopamine ramp, followed by slow signal recovery upon stimulus cessation. In contrast to reward conditioning, dopamine was unaffected by WN value, context valence, or probabilistic contingencies, and the WN dopamine-response shifted only partially towards its predictive cue. However, unpredicted WN provoked slower post-stimulus signal recovery than predicted WN. Despite differing signal qualities, dopamine responses to simultaneous presentation of rewarding and aversive stimuli were additive. Together, our findings indicate that instead of a PPE, NAC dopamine primarily tracks prediction and duration of punishment.

Prior Reward Conditioning Dampens Hippocampal and Striatal Responses during an Associative Memory Task

Offering reward during encoding typically leads to better memory [Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S.,Knutson, B., & Gabrieli, J. D. E. Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron, 50, 507–517, 2006]. Whether such memory benefit persists when tested in a different task context remains, however, largely understudied [Wimmer, G. E., & Buechel, C. Reactivation of reward-related patterns from single past episodes supports memory-based decision making. Journal of Neuroscience, 36, 2868–2880, 2016]. Here, we ask whether reward at encoding leads to a generalized advantage across learning episodes, a question of high importance for any everyday life applications, from education to patient rehabilitation. Although we confirmed that offering monetary reward increased responses in the ventral striatum and pleasantness judgments for pictures used as stimuli, this immediate beneficial effect of reward did not carry over to a subsequent and different picture–location association memory task during which no reward was delivered. If anything, a trend for impaired memory accuracy was observed for the initially high-rewarded pictures as compared to low-rewarded ones. In line with this trend in behavioral performance, fMRI activity in reward (i.e., ventral striatum) and in memory (i.e., hippocampus) circuits was reduced during the encoding of new associations using previously highly rewarded pictures (compared to low-reward pictures). These neural effects extended to new pictures from same, previously highly rewarded semantic category. Twenty-four hours later, delayed recall of associations involving originally highly rewarded items was accompanied by decreased functional connectivity between the hippocampus and two brain regions implicated in value-based learning, the ventral striatum and the ventromedial pFC. We conclude that acquired reward value elicits a downward value-adjustment signal in the human reward circuit when reactivated in a novel nonrewarded context, with a parallel disengagement of memory–reward (hippocampal–striatal) networks, likely to undermine new associative learning. Although reward is known to promote learning, here we show how it may subsequently hinder hippocampal and striatal responses during new associative memory formation.

The minimal behavioral time window for reward conditioning in the nucleus accumbens of mice

The temporal precision of reward-reinforcement learning is determined by the minimal time window of the reward action—theoretically known as the eligibility trace. In animal studies, however, such a minimal time window and its origin have not been well understood. Here, we used head-restrained mice to accurately control the timing of sucrose water as an unconditioned stimulus (US); we found that the reinforcement effect of the US occurred only within 1 s after a short tone of a conditioned stimulus (CS). The conditioning required the dopamine D1 receptor and CaMKII signaling in the nucleus accumbens (NAc). The time window was not reduced by replacing CS with optogenetic stimulation of the synaptic inputs to the NAc, which is in agreement with previous reports on the effective dopamine timing of NAc synapses. Thus, our data suggest that the minimal reward time window is 1 s, and is formed in the NAc.