scholarly journals Signals of anticipation of reward and of mean reward rates in the human brain

2019 ◽  
Author(s):  
Roberto Viviani ◽  
Lisa Dommes ◽  
Julia Bosch ◽  
Michael Steffens ◽  
Anna Paul ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Substantia Nigra ◽  
Human Brain ◽  
Learning Theory ◽  
Functional Neuroimaging ◽  
Theoretical Models ◽  
Prediction Errors ◽  
Deep Layers ◽  
Signal Changes ◽  
Reinforcement Learning Model

AbstractTheoretical models of dopamine function stemming from reinforcement learning theory have emphasized the importance of prediction errors, which signal changes in the expectation of impending rewards. Much less is known about the effects of mean reward rates, which may be of motivational significance due to their role in computing the optimal effort put into exploiting reward opportunities. Here, we used a reinforcement learning model to design three functional neuroimaging studies and disentangle the effects of changes in reward expectations and mean reward rates, showing recruitment of specific regions in the brainstem regardless of prediction errors. While changes in reward expectations activated ventral striatal areas as in previous studies, mean reward rates preferentially modulated the substantia nigra/ventral tegmental area, deep layers of the superior colliculi, and a posterior pontomesencephalic region. These brainstem structures may work together to set motivation and attentional efforts levels according to perceived reward opportunities.

scholarly journals Increased ventromedial prefrontal cortex activity in adolescence benefits prosocial reinforcement learning

2021 ◽  
Author(s):  
Bianca Westhoff ◽  
Neeltje E. Blankenstein ◽  
Elisabeth Schreuders ◽  
Eveline A. Crone ◽  
Anna C. K. van Duijvenvoorde
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Functional Neuroimaging ◽  
Learning Task ◽  
Developmental Trajectory ◽  
Ventromedial Prefrontal Cortex ◽  
Prediction Errors ◽  
Age Related ◽  
Probabilistic Reinforcement ◽  
Neuroimaging Study

AbstractLearning which of our behaviors benefit others contributes to social bonding and being liked by others. An important period for the development of (pro)social behavior is adolescence, in which peers become more salient and relationships intensify. It is, however, unknown how learning to benefit others develops across adolescence and what the underlying cognitive and neural mechanisms are. In this functional neuroimaging study, we assessed learning for self and others (i.e., prosocial learning) and the concurring neural tracking of prediction errors across adolescence (ages 9-21, N=74). Participants performed a two-choice probabilistic reinforcement learning task in which outcomes resulted in monetary consequences for themselves, an unknown other, or no one. Participants from all ages were able to learn for themselves and others, but learning for others showed a more protracted developmental trajectory. Prediction errors for self were observed in the ventral striatum and showed no age-related differences. However, prediction error coding for others was specifically observed in the ventromedial prefrontal cortex and showed age-related increases. These results reveal insights into the computational mechanisms of learning for others across adolescence, and highlight that learning for self and others show different age-related patterns.

scholarly journals Acute stress impairs reward learning in men

2020 ◽  
Author(s):  
Joana Carvalheiro ◽  
Vasco A. Conceição ◽  
Ana Mesquita ◽  
Ana Seara-Cardoso
Keyword(s):  
Reinforcement Learning ◽  
Acute Stress ◽  
Learning Task ◽  
Reward Learning ◽  
Prediction Errors ◽  
How People Learn ◽  
And Control ◽  
The Impact ◽  
Reinforcement Learning Model ◽  
Better Than

AbstractAcute stress is ubiquitous in everyday life, but the extent to which acute stress affects how people learn from the outcomes of their choices is still poorly understood. Here, we investigate how acute stress impacts reward and punishment learning in men using a reinforcement-learning task. Sixty-two male participants performed the task whilst under stress and control conditions. We observed that acute stress impaired participants’ choice performance towards monetary gains, but not losses. To unravel the mechanism(s) underlying such impairment, we fitted a reinforcement-learning model to participants’ trial-by-trial choices. Computational modeling indicated that under acute stress participants learned more slowly from positive prediction errors — when the outcomes were better than expected — consistent with stress-induced dopamine disruptions. Such mechanistic understanding of how acute stress impairs reward learning is particularly important given the pervasiveness of stress in our daily life and the impact that stress can have on our wellbeing and mental health.

scholarly journals Neurocomputational mechanisms of prosocial learning and links to empathy

2016 ◽  
Vol 113 (35) ◽  
pp. 9763-9768 ◽  
Author(s):  
Patricia L. Lockwood ◽  
Matthew A. J. Apps ◽  
Vincent Valton ◽  
Essi Viding ◽  
Jonathan P. Roiser
Keyword(s):  
Reinforcement Learning ◽  
Prosocial Behavior ◽  
Learning Theory ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Posterior Portion ◽  
Subgenual Anterior Cingulate Cortex ◽  
Actual Outcome ◽  
The Difference ◽  
Do So

Reinforcement learning theory powerfully characterizes how we learn to benefit ourselves. In this theory, prediction errors—the difference between a predicted and actual outcome of a choice—drive learning. However, we do not operate in a social vacuum. To behave prosocially we must learn the consequences of our actions for other people. Empathy, the ability to vicariously experience and understand the affect of others, is hypothesized to be a critical facilitator of prosocial behaviors, but the link between empathy and prosocial behavior is still unclear. During functional magnetic resonance imaging (fMRI) participants chose between different stimuli that were probabilistically associated with rewards for themselves (self), another person (prosocial), or no one (control). Using computational modeling, we show that people can learn to obtain rewards for others but do so more slowly than when learning to obtain rewards for themselves. fMRI revealed that activity in a posterior portion of the subgenual anterior cingulate cortex/basal forebrain (sgACC) drives learning only when we are acting in a prosocial context and signals a prosocial prediction error conforming to classical principles of reinforcement learning theory. However, there is also substantial variability in the neural and behavioral efficiency of prosocial learning, which is predicted by trait empathy. More empathic people learn more quickly when benefitting others, and their sgACC response is the most selective for prosocial learning. We thus reveal a computational mechanism driving prosocial learning in humans. This framework could provide insights into atypical prosocial behavior in those with disorders of social cognition.

scholarly journals Behavioural and computational evidence for memory consolidation biased by reward-prediction errors

2019 ◽  
Author(s):  
Emma L. Roscow ◽  
Matthew W. Jones ◽  
Nathan F. Lepora
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Learning Task ◽  
Male Rats ◽  
Adaptive Behaviour ◽  
Prediction Errors ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Per Se ◽  
Reinforcement Learning Model

AbstractNeural activity encoding recent experiences is replayed during sleep and rest to promote consolidation of the corresponding memories. However, precisely which features of experience influence replay prioritisation to optimise adaptive behaviour remains unclear. Here, we trained adult male rats on a novel maze-based rein-forcement learning task designed to dissociate reward outcomes from reward-prediction errors. Four variations of a reinforcement learning model were fitted to the rats’ behaviour over multiple days. Behaviour was best predicted by a model incorporating replay biased by reward-prediction error, compared to the same model with no replay; random replay or reward-biased replay produced poorer predictions of behaviour. This insight disentangles the influences of salience on replay, suggesting that reinforcement learning is tuned by post-learning replay biased by reward-prediction error, not by reward per se. This work therefore provides a behavioural and theoretical toolkit with which to measure and interpret replay in striatal, hippocampal and neocortical circuits.

scholarly journals Associability-modulated loss learning is increased in posttraumatic stress disorder

eLife ◽  
2018 ◽  
Vol 7 ◽  
Author(s):  
Vanessa M Brown ◽  
Lusha Zhu ◽  
John M Wang ◽  
B Christopher Frueh ◽  
Brooks King-Casas ◽  
...  
Keyword(s):  
Posttraumatic Stress Disorder ◽  
Reinforcement Learning ◽  
Posttraumatic Stress ◽  
Stress Disorder ◽  
Functional Neuroimaging ◽  
Parameter Estimates ◽  
Prediction Errors ◽  
Behavioral Choices ◽  
Cardinal Symptom ◽  
Unexpected Outcomes

Disproportionate reactions to unexpected stimuli in the environment are a cardinal symptom of posttraumatic stress disorder (PTSD). Here, we test whether these heightened responses are associated with disruptions in distinct components of reinforcement learning. Specifically, using functional neuroimaging, a loss-learning task, and a computational model-based approach, we assessed the mechanistic hypothesis that overreactions to stimuli in PTSD arise from anomalous gating of attention during learning (i.e., associability). Behavioral choices of combat-deployed veterans with and without PTSD were fit to a reinforcement learning model, generating trial-by-trial prediction errors (signaling unexpected outcomes) and associability values (signaling attention allocation to the unexpected outcomes). Neural substrates of associability value and behavioral parameter estimates of associability updating, but not prediction error, increased with PTSD during loss learning. Moreover, the interaction of PTSD severity with neural markers of associability value predicted behavioral choices. These results indicate that increased attention-based learning may underlie aspects of PTSD and suggest potential neuromechanistic treatment targets.

Reward Prediction Errors Drive Declarative Learning Irrespective of Agency

2020 ◽  
Author(s):  
Kate Ergo ◽  
Luna De Vilder ◽  
Esther De Loof ◽  
Tom Verguts
Keyword(s):  
Learning Theory ◽  
Learning Effect ◽  
Steady Increase ◽  
Prediction Errors ◽  
Experimental Paradigm ◽  
Reward Prediction ◽  
Declarative Learning

Recent years have witnessed a steady increase in the number of studies investigating the role of reward prediction errors (RPEs) in declarative learning. Specifically, in several experimental paradigms RPEs drive declarative learning; with larger and more positive RPEs enhancing declarative learning. However, it is unknown whether this RPE must derive from the participant’s own response, or whether instead any RPE is sufficient to obtain the learning effect. To test this, we generated RPEs in the same experimental paradigm where we combined an agency and a non-agency condition. We observed no interaction between RPE and agency, suggesting that any RPE (irrespective of its source) can drive declarative learning. This result holds implications for declarative learning theory.

Cancer and Cancer Treatment-Related Cognitive Impairment

2019 ◽  
pp. 60-83
Author(s):  
James C.  Root ◽  
Elizabeth Ryan ◽  
Tim A. Ahles
Keyword(s):  
Cancer Survivors ◽  
Cancer Treatment ◽  
Clinical Presentation ◽  
Functional Neuroimaging ◽  
Theoretical Models ◽  
Future Research ◽  
Cognitive Changes ◽  
Central Nervous System Cancer ◽  
Pharmacologic Treatments

As the population of cancer survivors has grown into the millions, there is increasing emphasis on understanding how late effects of treatment impact survivors’ ability return to work/school, ability to function and live independently, and overall quality of life. Cognitive changes are one of the most feared problems among cancer survivors. This chapter describes the growing literature examining cognitive changes associated with non-central nervous system cancer and cancer treatment. Typical elements of cancer treatment are discussed, followed by a description of clinical presentation, self-reported and objectively assessed cognitive findings, and results of structural and functional neuroimaging research. Genetic and other risk factors for cognitive decline following treatment are identified and discussed, together with biomarkers and animal models of treatment-related effects. This is followed by a discussion of behavioral and pharmacologic treatments. Finally, challenges and recommendations for future research are provided to help guide subsequent research and theoretical models.

Sign in / Sign up

Export Citation Format

Share Document