Altered Reinforcement Learning From Reward and Punishment in Anorexia Nervosa: Evidence From Computational Modeling

ABSTRACT Objectives: Anorexia nervosa (AN) is associated with altered sensitivity to reward and punishment. Few studies have investigated whether this results in aberrant learning. The ability to learn from rewarding and aversive experiences is essential for flexibly adapting to changing environments, yet individuals with AN tend to demonstrate cognitive inflexibility, difficulty set-shifting and altered decision-making. Deficient reinforcement learning may contribute to repeated engagement in maladaptive behavior. Methods: This study investigated learning in AN using a probabilistic associative learning task that separated learning of stimuli via reward from learning via punishment. Forty-two individuals with Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 restricting-type AN were compared to 38 healthy controls (HCs). We applied computational models of reinforcement learning to assess group differences in learning, thought to be driven by violations in expectations, or prediction errors (PEs). Linear regression analyses examined whether learning parameters predicted BMI at discharge. Results: AN had lower learning rates than HC following both positive and negative PE (p < .02), and were less likely to exploit what they had learned. Negative PE on punishment trials predicted lower discharge BMI (p < .001), suggesting individuals with more negative expectancies about avoiding punishment had the poorest outcome. Conclusions: This is the first study to show lower rates of learning in AN following both positive and negative outcomes, with worse punishment learning predicting less weight gain. An inability to modify expectations about avoiding punishment might explain persistence of restricted eating despite negative consequences, and suggests that treatments that modify negative expectancy might be effective in reducing food avoidance in AN.

Download Full-text

Reward and punishment reversal learning in major depressive disorder

10.31234/osf.io/aqgx3 ◽

2020 ◽

Author(s):

Dahlia Mukherjee ◽

Alexandre Leo Stephen Filipowicz ◽

Khoi D. Vo ◽

Theodore Sattherwaite ◽

Joe Kable

Keyword(s):

Major Depressive Disorder ◽

Reinforcement Learning ◽

Depressive Disorder ◽

Computational Modeling ◽

Reversal Learning ◽

Performance Metrics ◽

Learning Task ◽

Major Depressive ◽

Learning Rates ◽

Reward And Punishment

Depression has been associated with impaired reward and punishment processing, but the specific nature of these deficits is less understood and still widely debated. We analyzed reinforcement-based decision-making in individuals diagnosed with major depressive disorder (MDD) to identify the specific decision mechanisms contributing to poorer performance. Individuals with MDD (n = 64) and matched healthy controls (n = 64) performed a probabilistic reversal learning task in which they used feedback to identify which of two stimuli had the highest probability of reward (reward condition) or lowest probability of punishment (punishment condition). Learning differences were characterized using a hierarchical Bayesian reinforcement learning model. While both groups showed reinforcement learning-like behavior, depressed individuals made fewer optimal choices and adjusted more slowly to reversals in both the reward and punishment conditions. Our computational modeling analysis found that depressed individuals showed lower learning rates and, to a lesser extent, lower value sensitivity in both the reward and punishment conditions. Learning rates also predicted depression more accurately than simple performance metrics. These results demonstrate that depression is characterized by a hyposensitivity to positive outcomes, which influences the rate at which depressed individuals learn from feedback, but not a hypersensitivity to negative outcomes as has previously been suggested. Additionally, we demonstrate that computational modeling provides a more precise characterization of the dynamics contributing to these learning deficits, and offers stronger insights into the mechanistic processes affected by depression.

Download Full-text

Model-based prioritization for acquiring protection

10.31234/osf.io/nv5u4 ◽

2021 ◽

Author(s):

Sarah M. Tashjian ◽

Toby Wise ◽

dean mobbs

Keyword(s):

Reinforcement Learning ◽

Computational Modeling ◽

Learning Rate ◽

Behavioral Task ◽

Decision Strategies ◽

Total N ◽

Model Based Control ◽

Model Based ◽

Computational Architecture ◽

Reward And Punishment

Protection, or the mitigation of harm, often involves the capacity to prospectively plan the actions needed to combat a threat. The computational architecture of decisions involving protection remains unclear, as well as whether these decisions differ from other positive prospective actions. Here we examine effects of valence and context by comparing protection to reward, which occurs in a different context but is also positively valenced, and punishment, which also occurs in an aversive context but differs in valence. We applied computational modeling across three independent studies (Total N=600) using five iterations of a ‘two-step’ behavioral task to examine model-based reinforcement learning for protection, reward, and punishment in humans. Decisions motivated by acquiring safety via protection evoked a higher degree of model-based control than acquiring reward and avoiding punishment, with no significant differences in learning rate. The context-valence asymmetry characteristic of protection increased deployment of flexible decision strategies, suggesting model-based control depends on the context in which outcomes are encountered as well as the valence of the outcome.

Download Full-text

High dietary restraint improves performance on a food-motivated probabilistic selection task

10.31232/osf.io/wk4bt ◽

2019 ◽

Cited By ~ 2

Author(s):

Jennifer R Sadler ◽

Grace Elisabeth Shearrer ◽

Nichollette Acosta ◽

Kyle Stanley Burger

Keyword(s):

Reinforcement Learning ◽

Instrumental Conditioning ◽

Food Reinforcement ◽

Dietary Restraint ◽

Memory Task ◽

Brain Response ◽

Anthropometric Measures ◽

Instrumental Task ◽

The Impact ◽

Reward And Punishment

BACKGROUND: Dietary restraint represents an individual’s intent to limit their food intake and has been associated with impaired passive food reinforcement learning. However, the impact of dietary restraint on an active, response dependent learning is poorly understood. In this study, we tested the relationship between dietary restraint and food reinforcement learning using an active, instrumental conditioning task. METHODS: A sample of ninety adults completed a response-dependent instrumental conditioning task with reward and punishment using sweet and bitter tastes. Brain response via functional MRI was measured during the task. Participants also completed anthropometric measures, reward/motivation related questionnaires, and a working memory task. Dietary restraint was assessed via the Dutch Restrained Eating Scale. RESULTS: Two groups were selected from the sample: high restraint (n=29, score >2.5) and low restraint (n=30; score <1.85). High restraint was associated with significantly higher BMI (p=0.003) and lower N-back accuracy (p=0.045). The high restraint group also was marginally better at the instrumental conditioning task (p=0.066, r=0.37). High restraint was also associated with significantly greater brain response in the intracalcarine cortex (MNI: 15, -69, 12; k=35, pfwe< 0.05) to bitter taste, compared to neutral taste.CONCLUSIONS: High restraint was associated with improved performance on an instrumental task testing how individuals learn from reward and punishment. This may be mediated by greater brain response in the primary visual cortex, which has been associated with mental representation. Results suggest that dietary restraint does not impair response-dependent reinforcement learning.

Download Full-text

Affect, reward, and punishment in anorexia nervosa: a narrative overview

Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity ◽

10.1007/s40519-018-0588-9 ◽

2018 ◽

Vol 23 (6) ◽

pp. 731-737 ◽

Cited By ~ 3

Author(s):

Margarita Sala ◽

Amy H. Egbert ◽

Jason M. Lavender ◽

Andrea B. Goldschmidt

Keyword(s):

Anorexia Nervosa ◽

Reward And Punishment

Download Full-text

The Tortoise and the Hare: Interactions between Reinforcement Learning and Working Memory

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_01238 ◽

2018 ◽

Vol 30 (10) ◽

pp. 1422-1432 ◽

Cited By ~ 16

Author(s):

Anne G. E. Collins

Keyword(s):

Working Memory ◽

Reinforcement Learning ◽

Computational Modeling ◽

Behavioral Experiment ◽

Prediction Errors ◽

Multiple Systems ◽

Memory Interference ◽

Neural Representations ◽

The Cost

Learning to make rewarding choices in response to stimuli depends on a slow but steady process, reinforcement learning, and a fast and flexible, but capacity-limited process, working memory. Using both systems in parallel, with their contributions weighted based on performance, should allow us to leverage the best of each system: rapid early learning, supplemented by long-term robust acquisition. However, this assumes that using one process does not interfere with the other. We use computational modeling to investigate the interactions between the two processes in a behavioral experiment and show that working memory interferes with reinforcement learning. Previous research showed that neural representations of reward prediction errors, a key marker of reinforcement learning, were blunted when working memory was used for learning. We thus predicted that arbitrating in favor of working memory to learn faster in simple problems would weaken the reinforcement learning process. We tested this by measuring performance in a delayed testing phase where the use of working memory was impossible, and thus participant choices depended on reinforcement learning. Counterintuitively, but confirming our predictions, we observed that associations learned most easily were retained worse than associations learned slower: Using working memory to learn quickly came at the cost of long-term retention. Computational modeling confirmed that this could only be accounted for by working memory interference in reinforcement learning computations. These results further our understanding of how multiple systems contribute in parallel to human learning and may have important applications for education and computational psychiatry.

Download Full-text