Systematically investigating the role of context on effect replicability in reinstatement of fear in humans

Context is crucial in guiding behavior in an ever changing world and contextual information plays a crucial role in associative learning processes. For instance, the return of fear (RoF) after reinstatement (i.e, re-exposure the unconditioned stimulus (US) after successful fear extinction) is context dependent and is suggested to occur only when either extinction and test, or extinction and reinstatement context are identical, not when US re-exposure (i.e., reinstatement) occurs in a context different from extinction and test. Human adaptions of reinstatement paradigms have resulted in mixed findings: CS specific as well as unspecific RoF or unexpected “reinstated” conditioned responding in no reinstatement US control groups. Here, we systematically investigate the role of context on reinstatement-induced RoF in a human differential fear conditioning paradigm using subjective and psychophysiological measures in a large sample (N=212) including reinstatement and control groups. Overall, response patterns in reinstatement-groups mirrored results from single-cue rodent work. Yet, only generalized, not differential RoF was observed. Remarkably, depending on outcome measure RoF was also observed under identical experimental context conditions without US-re-exposure, underlining effects of contextual change beyond the reinstatement-US and challenging reinstatement research in human subjects and what we think we know about the mechanisms behind the reinstatement-phenomenon.

Sex differences in behavioral responding and dopamine release during early Pavlovian learning

Learning associations between cues and rewards requires the mesolimbic dopamine system. The dopamine response to cues signals differences in reward value in well-trained animals. These value-related dopamine responses are absent during early learning when cues signal differences in the reward rate, which suggests cue-evoked dopamine release conveys differences between outcomes only after extensive training. However, it is unclear if this lack of value coding by cue-evoked dopamine release during early learning is unique to when cues signal differences in reward rate, or if this is also evident when cues signal differences in other value-related parameters such as reward size. To address this, we utilized a Pavlovian conditioning task in which one audio cue was associated with a small reward (one pellet) and another audio cue was associated with a large reward (three pellets). We performed fast-scan cyclic voltammetry to record changes in dopamine release in the nucleus accumbens of male and female rats throughout early learning. Cue-evoked dopamine release did not encode differences in reward value, and there were no differences in this response between males and females. However, female rats exhibited higher levels of conditioned responding and a faster latency to respond. Reward-evoked dopamine release scaled with reward size in both sexes, though there were transient sex differences in the dynamics of this response. We additionally identified sex differences in the number of post-reward head entries. Collectively these data illustrate sustained sex differences in behavioral responding as well as transient sex differences in reward-evoked dopamine release.

Corticostriatal suppression of appetitive Pavlovian conditioned responding

ABSTRACTThe capacity to suppress learned responses is essential for animals to adapt in dynamic environments. Extinction is a process by which animals learn to suppress conditioned responding when an expected outcome is omitted. The infralimbic cortex (IL) to nucleus accumbens shell (NAcS) neural circuit is implicated in suppressing conditioned responding after extinction, especially in the context of operant cocaine-seeking behaviour. However, the role of the IL-to-NAcS neural circuit in the extinction of responding to appetitive Pavlovian cues is unknown and the psychological mechanisms involved in response suppression following extinction are unclear. We trained rats to associate a 10 s auditory conditioned stimulus (CS; 14 trials per session) with a sucrose unconditioned stimulus (US; 0.2 mL per CS) in a specific context and then, following extinction in a different context, precipitated a renewal of CS responding by presenting the CS alone in the original Pavlovian conditioning context. Unilateral, optogenetic stimulation of the IL-to-NAcS circuit selectively during CS trials suppressed renewal. In a separate experiment, IL-to-NAcS stimulation suppressed CS responding regardless of prior extinction and impaired extinction retrieval. Finally, IL-to-NAcS stimulation during the CS did not suppress the acquisition of Pavlovian conditioning but was required for the subsequent expression of CS responding. These results are consistent with multiple studies showing that the IL-to-NAcS neural circuit is involved in the suppression of operant cocaine-seeking, extending these findings to appetitive Pavlovian cues. The suppression of appetitive Pavlovian responding following IL-to-NAcS circuit stimulation does not, however, appear to require an extinction-dependent process.SIGNIFICANCE STATEMENTExtinction is a form of inhibitory learning through which animals learn to suppress conditioned responding in the face of non-reinforcement. We investigated the role of infralimbic (IL) cortex inputs to the nucleus accumbens shell (NAcS) in the extinction of responding to appetitive Pavlovian cues and the psychological mechanisms involved in response suppression following extinction. Using in vivo optogenetics, we found that stimulating the IL-to-NAcS neural circuit suppressed context-induced renewal of conditioned responding after extinction. In a separate experiment, stimulating the IL-to-NAcS circuit suppressed conditioned responding in an extinction-independent manner. These findings can be leveraged by future research aimed at understanding how corticostriatal circuits contribute to behavioural flexibility and mental disorders that involve the suppression of learned behaviours.

The Role of Context Conditioning in the Reinstatement of Responding to an Alcohol-Predictive Conditioned Stimulus

Re-exposure to an unconditioned stimulus (US) can reinstate extinguished conditioned responding elicited by a conditioned stimulus (CS). We tested the hypothesis that the reinstatement of responding to an appetitive CS is driven by an excitatory association formed between the US and the context that the US was ingested in during US re-exposure. Male, Long-Evans rats were acclimated to drinking alcohol (15%, v/v) in the home-cage, then trained to associate an auditory CS with an alcohol-US that was delivered into a fluid port for oral intake. During subsequent extinction sessions, the CS was presented as before, but without alcohol. After extinction, rats were re-exposed to alcohol as in training, but without the CS (US re-exposure). 24 h later at test, the CS was presented as in training, but without alcohol. First, we tested the effect of extinguishing the context-alcohol association, formed during alcohol re-exposure, on reinstatement. Conducting four context extinction sessions across four days (spaced extinction) after the US re-exposure session did not impact reinstatement. However, four context extinction sessions conducted across two days (massed extinction) prevented reinstatement. Next, we conducted alcohol re-exposure in a context that either differed from, or was the same as, the test context. One alcohol re-exposure session in a different context did not affect reinstatement, however, three alcohol re-exposure sessions in a different context significantly reduced reinstatement during the first CS trial. These results partially support the view that a context-US association formed during US re-exposure drives the reinstatement of responding to an appetitive, alcohol-predictive CS.

The amygdala instructs insular feedback for affective learning

Affective responses depend on assigning value to environmental predictors of threat or reward. Neuroanatomically, this affective value is encoded at both cortical and subcortical levels. However, the purpose of this distributed representation across functional hierarchies remains unclear. Using fMRI in mice, we mapped a discrete cortico-limbic loop between insular cortex (IC), central amygdala (CE), and nucleus basalis of Meynert (NBM), which decomposes the affective value of a conditioned stimulus (CS) into its salience and valence components. In IC, learning integrated unconditioned stimulus (US)-evoked bodily states into CS valence. In turn, CS salience in the CE recruited these CS representations bottom-up via the cholinergic NBM. This way, the CE incorporated interoceptive feedback from IC to improve discrimination of CS valence. Consequently, opto-/chemogenetic uncoupling of hierarchical information flow disrupted affective learning and conditioned responding. Dysfunctional interactions in the IC↔CE/NBM network may underlie intolerance to uncertainty, observed in autism and related psychiatric conditions.

The probability of reinforcement per trial affects post-trial responding and subsequent extinction, but not within-trial responding

During magazine approach conditioning, rats do not discriminate between a conditioned stimulus (CS) that is consistently reinforced with food and a CS that is occasionally (partially) reinforced, as long as the CSs have the same overall reinforcement rate per second. This implies that rats are indifferent to the probability of reinforcement per trial. However, in the same rats, the per-trial reinforcement rate will affect subsequent extinction—responding extinguishes more rapidly for a CS that was consistently reinforced than for a partially reinforced CS. Here, we trained rats with consistently and partially reinforced CSs that were matched for overall reinforcement rate per second. We measured conditioned responding both during and immediately after the CSs. Differences in the per-trial probability of reinforcement did not affect the acquisition of responding during the CS, but did affect subsequent extinction of that responding, and also affected the post-CS response rates during conditioning. Indeed, CSs with the same probability of reinforcement per trial evoked the same amount of post-CS responding even when they differed in overall reinforcement rate and thus evoked different amounts of responding during the CS. We conclude that reinforcement rate per second controls rats’ acquisition of responding during the CS, but at the same time rats also learn specifically about the probability of reinforcement per trial. The latter learning affects the rats’ expectation of reinforcement as an outcome of the trial, which influences their ability to detect retrospectively that an opportunity for reinforcement was missed, and in turn drives extinction.

Pavlovian conditioning under partial reinforcement: The effects of non-reinforced trials versus cumulative CS duration

A core feature of associative models, such as those proposed by Allan Wagner (Rescorla & Wagner, 1972; Wagner, 1981), is that conditioning proceeds in a trial-by-trial fashion, with increments and decrements in associative strength occurring on each occasion that the conditioned stimulus (CS) is present either with or without the unconditioned stimulus (US). A very different approach has been taken by theories that assume animals continuously accumulate information about the total length of time spent waiting for the US both during the CS and in the absence of the CS (e.g., Gallistel & Gibbon, 2000). Here we describe three experiments using within-subject designs that tested between trial-based and time-accumulation accounts of the acquisition of conditioned responding using magazine approach conditioning in rats. We found that responding was affected by the total (cumulative) duration of exposure to the CS without the US rather than the number of trials on which the CS occurred without the US. We also found that exposure to the CS without the US had the same effect on conditioning whether that exposure occurred shortly (60 s) before each CS-US pairing or whether it occurred long (240 s) before each pairing. These findings are more consistent with time-accumulation models of conditioning than trial-based models like the Rescorla-Wagner model and Wagner’s (1981) Sometimes Opponent Process model. We discuss these findings in relation to other evidence that favours trial-based models rather than time-accumulation models.

The Partial Reinforcement Extinction Effect Depends on Learning about Non-Reinforced Trials Rather than Reinforcement Rate

Conditioned responding extinguishes more slowly after partial (inconsistent) reinforcement than after consistent reinforcement. This Partial Reinforcement Extinction Effect (PREE) is usually attributed to learning about nonreinforcement during the partial schedule. An alternative explanation attributes it to any difference in the rate of reinforcement, arguing that animals can detect the change to nonreinforcement more quickly after a denser schedule than a leaner schedule. Experiments 1a and 1b compared extinction of magazine responding to a conditioned stimulus (CS) reinforced with one food pellet per trial and a CS reinforced with two pellets per trial. Despite the difference in reinforcement rate, there was no reliable difference in extinction. Both experiments did demonstrate the conventional PREE comparing a partial CS (50% reinforced) with a consistent CS. Experiments 2 and 3 tested whether the PREE depends specifically on learning about nonreinforced trials during partial reinforcement. Rats were trained with two CS configurations, A and AX. One was partially reinforced, the other consistently reinforced. When AX was partial and A consistent, responding to AX extinguished more slowly than to A. When AX was consistent and A was partial, there was no difference in their extinction. Therefore, pairing X with partial reinforcement allowed rats to show a PREE to AX that did not generalise to A. Pairing A with partial reinforcement meant that rats showed a PREE to A that generalised to AX. Thus, the PREE depends on learning about nonreinforced trials during partial reinforcement and is not due to any difference in per-trial probability of reinforcement

Dopamine release and its control over early Pavlovian learning differs between the NAc core and medial NAc shell

Dopamine neurons respond to cues to reflect the value of associated outcomes. These cue-evoked dopamine responses can encode the relative rate of reward in rats with extensive Pavlovian training. Specifically, a cue that always follows the previous reward by a short delay (high reward rate) evokes a larger dopamine response in the nucleus accumbens (NAc) core relative to a distinct cue that always follows the prior reward by a long delay (low reward rate). However, it was unclear if these reward rate dopamine signals are evident during early Pavlovian training sessions and across NAc subregions. To address this, we performed fast-scan cyclic voltammetry recordings of dopamine levels to track the pattern of cue- and reward-evoked dopamine signals in the NAc core and medial NAc shell. We identified regional differences in the progression of cue-evoked dopamine signals across training. However, the dopamine response to cues did not reflect the reward rate in either the NAc core or the medial NAc shell during early training sessions. Pharmacological experiments found that dopamine-sensitive conditioned responding emerged in the NAc core before the medial NAc shell. Together, these findings illustrate regional differences in NAc dopamine release and its control over behavior during early Pavlovian learning.