Behavioral and Neurobiological Mechanisms of Pavlovian and Instrumental Extinction Learning

2020 ◽  
Author(s):  
Mark E. Bouton ◽  
Stephen Maren ◽  
Gavan P McNally
Keyword(s):  
Neural Circuit ◽  
Original Learning ◽  
Cortical Inhibition ◽  
Extinction Learning ◽  
Fully Integrated ◽  
Operant Responding ◽  
Prefrontal Cortical ◽  
Pavlovian Extinction ◽  
Inhibitory Learning ◽  
Operant Extinction

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning, but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in fear retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

scholarly journals Conditional Control of Instrumental Avoidance by Context Following Extinction

2021 ◽  
Vol 15 ◽  
Author(s):  
Vincent D. Campese ◽  
Lauren A. Brannigan ◽  
Joseph E. LeDoux
Keyword(s):  
Hierarchical Control ◽  
Response Rates ◽  
Training Context ◽  
Extinction Learning ◽  
Retention Testing ◽  
Conditional Control ◽  
Within Subjects ◽  
Pavlovian Extinction ◽  
Contrast Response

Using rodents, three training arrangements (i.e., ABB vs. ABA, AAA vs. AAB and ABB vs. ABC) explored whether extinction influences the expression of avoidance in a manner controlled by context. Retention testing following extinction showed that more avoidance responding (i.e., renewal) was observed when extinguished cues were tested outside of the context where they had undergone extinction. In contrast, response rates were significantly lower when stimuli were tested within the context where extinction learning had occurred. These findings add to the emerging literature assessing the role of Pavlovian extinction processes in the development of instrumental avoidance responding by demonstrating conditional control over extinguished responding by context. This study was conducted using a within-subjects approach that minimized the potential for context-outcome associations to bias responding, and thus, reflects hierarchical control over behavior based on the specific associative status of each tested cue in each training context.

scholarly journals Complex dynamics of choice during operant extinction and a “simple” associative learning account

2020 ◽  
Author(s):  
José R. Donoso ◽  
Julian Packheiser ◽  
Roland Pusch ◽  
Zhiyin Lederer ◽  
Thomas Walther ◽  
...  
Keyword(s):  
Computational Model ◽  
Associative Learning ◽  
Operant Conditioning ◽  
Complex Dynamics ◽  
Rule Learning ◽  
Learning Curves ◽  
Learning Mechanisms ◽  
Learning Session ◽  
Extinction Learning ◽  
Operant Extinction

AbstractExtinction learning, the process of ceasing an acquired behavior in response to altered reinforcement contingencies, is essential for survival in a changing environment. So far, research has mostly neglected the learning dynamics and variability of behavior during extinction learning and instead focused on a few response types that were studied by population averages. Here, we take a different approach by analyzing the trial-by-trial dynamics of operant extinction learning in both pigeons and a computational model. The task involved discriminant operant conditioning in context A, extinction in context B, and a return to context A to test the context-dependent return of the conditioned response (ABA renewal). By studying single learning curves across animals under repeated sessions of this paradigm, we uncovered a rich variability of behavior during extinction learning: (1) Pigeons prefer the unrewarded alternative choice in one-third of the sessions, predominantly during the very first extinction session an animal encountered. (2) In later sessions, abrupt transitions of behavior at the onset of context B emerge, and (3) the renewal effect decays as sessions progress. While these results could be interpreted in terms of rule learning mechanisms, we show that they can be parsimoniously accounted for by a computational model based only on associative learning between stimuli and actions. Our work thus demonstrates the critical importance of studying the trial-by-trial dynamics of learning in individual sessions, and the unexpected power of “simple” associative learning processes.Significance StatementOperant conditioning is essential for the discovery of purposeful actions, but once a stimulus-response association is acquired, the ability to extinguish it in response to altered reward contingencies is equally important. These processes also play a fundamental role in the development and treatment of pathological behaviors such as drug addiction, overeating and gambling. Here we show that extinction learning is not limited to the cessation of a previously reinforced response, but also drives the emergence of complex and variable choices that change from learning session to learning session. At first sight, these behavioral changes appear to reflect abstract rule learning, but we show in a computational model that they can emerge from “simple” associative learning.

scholarly journals The accumulation of N6-methyl-2’-deoxyadenosine in DNA drives activity-induced gene expression and is required for the extinction of conditioned fear

2016 ◽  
Author(s):  
Xiang Li ◽  
Qiongyi Zhao ◽  
Wei Wei ◽  
Quan Lin ◽  
Christophe Magnan ◽  
...  
Keyword(s):  
Gene Expression ◽  
Cortical Neurons ◽  
Conditioned Fear ◽  
Fear Extinction ◽  
Adult Brain ◽  
Extinction Learning ◽  
Prefrontal Cortical ◽  
Genome Wide ◽  
Extinction Memory ◽  
The Brain

Here we report that the recently discovered mammalian DNA modification N6-methyl-2’-deoxyadenosine (m6dA) is dynamically regulated in primary cortical neurons, and accumulates along promoters and coding sequences within the genome of activated prefrontal cortical neurons of adult C57/BI6 mice in response to fear extinction learning. The deposition of m6dA is generally associated with increased genome-wide occupancy of the mammalian m6dA methyltransferase, N6amt1, and this correlates with fear extinction learning-induced gene expression. Of particular relevance for fear extinction memory, the accumulation of m6dA is associated with an active chromatin state and the recruitment of transcriptional machinery to the brain-derived neurotrophic factor (Bdnf) P4 promoter, which is required for Bdnf exon IV mRNA expression and for the extinction of conditioned fear. These results expand the scope of DNA modifications in the adult brain and highlight changes in m6dA as a novel neuroepigenetic mechanism associated with activity-induced gene expression and the formation of fear extinction memory.

scholarly journals Dissociable roles of medial orbitofrontal cortex in human operant extinction learning

NeuroImage ◽  
2008 ◽  
Vol 43 (4) ◽  
pp. 748-755 ◽  
Author(s):  
Elizabeth C. Finger ◽  
Derek G.V. Mitchell ◽  
Matthew Jones ◽  
R.J.R. Blair
Keyword(s):  
Orbitofrontal Cortex ◽  
Extinction Learning ◽  
Human Operant ◽  
Operant Extinction

scholarly journals Temporal regulation of motor behavior on a modified forelimb dexterity test in mice

2020 ◽  
Author(s):  
Hisham Mohammed ◽  
Yue Li ◽  
Paola Di Grazia ◽  
Amanda Bernstein ◽  
Sydney Agger ◽  
...  
Keyword(s):  
Task Difficulty ◽  
Neural Circuit ◽  
Motor Behavior ◽  
High Variability ◽  
Cortical Inhibition ◽  
Significant Percentage ◽  
Temporal Regulation ◽  
Single Pellet ◽  
Circuit Function ◽  
Task Acquisition

ABSTRACTHand and arm manual dexterity is a hallmark of humans and non-human primates. While rodents are less dexterous than primates, they provide powerful models for testing neural circuit function in behavioral output, including dexterous behaviors. In rodents, the single pellet reach task has been used extensively to study both dexterous forelimb motor learning as well as recovery from injury; however, mice exhibit high variability in task acquisition in comparison to rats and a significant percentage fail to learn the task. We have created a recessed version of the task that requires greater dexterity. This subtle modification increases both task difficulty as well as the proportion of mice that show an improvement with training. Furthermore, motor cortex inactivation shows a greater effect on the execution of the recessed forelimb reach task, with distinct effects on reach targeting vs grasping components depending on the timing of inhibitory activation. Kinematic analysis revealed differences in reach targeting upon transient cortical inhibition prior to reach onset. In summary, the recessed single pellet reach task provides a robust assessment of forelimb dexterity in mice and a tool for studying skilled motor acquisition and execution.

scholarly journals Sign-tracking behavior is difficult to extinguish and resistant to multiple cognitive enhancers

2019 ◽  
Author(s):  
Christopher J. Fitzpatrick ◽  
Trevor Geary ◽  
Justin F. Creeden ◽  
Jonathan D. Morrow
Keyword(s):  
Drug Addiction ◽  
Addiction Treatment ◽  
Spontaneous Recovery ◽  
Extinction Learning ◽  
Cognitive Enhancers ◽  
Pavlovian Conditioned Approach ◽  
Conditioned Approach ◽  
Pavlovian Extinction ◽  
Tracking Behavior ◽  
Drug Addiction Treatment

AbstractThe attribution of incentive-motivational value to drug-related cues underlies relapse and craving in drug addiction. One method of addiction treatment, cue-exposure therapy, utilizes repeated presentations of drug-related cues in the absence of drug (i.e., extinction learning); however, its efficacy has been limited due to an incomplete understanding of extinction and relapse processes after cues have been imbued with incentive-motivational value. To investigate this, we used a Pavlovian conditioned approach procedure to screen for rats that attribute incentive-motivational value to reward-related cues (sign-trackers; STs) or those that do not (goal-trackers; GTs). In Experiment 1, rats underwent Pavlovian extinction followed by reinstatement and spontaneous recovery tests. For comparison, a separate group of rats underwent PCA training followed by operant conditioning, extinction, and tests of reinstatement and spontaneous recovery. In Experiment 2, three cognitive enhancers (sodium butyrate, D-cycloserine, and fibroblast growth factor 2) were administered following extinction training to facilitate extinction learning. STs but not GTs displayed enduring resistance to Pavlovian, but not operant, extinction and were more susceptible to spontaneous recovery. In addition, none of the cognitive enhancers tested affected extinction learning. These results expand our understanding of extinction learning by demonstrating that there is individual variation in extinction and relapse processes and highlight potential difficulties in applying extinction-based therapies to drug addiction treatment in the clinic.

Sign in / Sign up

Export Citation Format

Share Document