scholarly journals Increased Ventromedial Prefrontal Cortex Activity in Adolescence Benefits Prosocial Reinforcement Learning

2021 ◽  
pp. 101018
Author(s):  
Bianca Westhoff ◽  
Neeltje E. Blankenstein ◽  
Elisabeth Schreuders ◽  
Eveline A. Crone ◽  
Anna C.K. van Duijvenvoorde
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Ventromedial Prefrontal Cortex

scholarly journals Diffusion modeling reveals reinforcement learning impairments in gambling disorder that are linked to attenuated ventromedial prefrontal cortex value representations

2020 ◽  
Author(s):  
Antonius Wiehler ◽  
Jan Peters
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Gambling Disorder ◽  
Ventromedial Prefrontal Cortex ◽  
Response Threshold ◽  
Boundary Separation ◽  
Learning Impairments ◽  
Model Based ◽  
Learning Rates ◽  
Over Time

AbstractGambling disorder is associated with deficits in classical feedback-based learning tasks, but the computational mechanisms underlying such learning impairments are still poorly understood. Here, we examined this question using a combination of computational modeling and functional resonance imaging (fMRI) in gambling disorder participants (n=23) and matched controls (n=19). Participants performed a simple reinforcement learning task with two pairs of stimuli (80% vs. 20% reinforcement rates per pair). As predicted, gamblers made significantly fewer selections of the optimal stimulus, while overall response times (RTs) were not significantly different between groups. We then used comprehensive modeling using reinforcement learning drift diffusion models (RLDDMs) in combination with hierarchical Bayesian parameter estimation to shed light on the computational underpinnings of this performance impairment. In both groups, an RLDDM in which both non-decision time and response threshold (boundary separation) changed over the course of the experiment accounted for the data best. The model showed good parameter recovery, and posterior predictive checks revealed that in both groups, the model reproduced the evolution of both accuracy and RTs over time. Examination of the group-wise posterior distributions revealed that the learning impairment in gamblers was attributable to both reduced learning rates and a more rapid reduction in boundary separation over time, compared to controls. Furthermore, gamblers also showed substantially shorter non-decision times. Model-based imaging analyses then revealed that value representations in gamblers in the ventromedial prefrontal cortex were attenuated compared to controls, and these effects were partly associated with model-based learning rates. Exploratory analyses revealed that a more anterior ventromedial prefrontal cortex cluster showed attenuations in value representations in proportion to gambling disorder severity in gamblers. Taken together, our findings reveal computational mechanisms underlying reinforcement learning impairments in gambling disorder, and confirm the ventromedial prefrontal cortex and as a critical neural hub in this disorder.

scholarly journals Value generalization in human avoidance learning

2017 ◽  
Author(s):  
Agnes Norbury ◽  
Trevor W. Robbins ◽  
Ben Seymour
Keyword(s):  
Decision Making ◽  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Avoidance Learning ◽  
Anterior Insula ◽  
Ventromedial Prefrontal Cortex ◽  
Sensory Cortex ◽  
Primary Sensory Cortex ◽  
Human Volunteers ◽  
Mediate Generalization

SummaryGeneralization during aversive decision-making allows us to avoid a broad range of potential threats following experience with a limited set of exemplars. However, over-generalization, resulting in excessive and inappropriate avoidance, has been implicated in a variety of psychological disorders. Here, we use reinforcement learning modelling to dissect out different contributions to the generalization of instrumental avoidance in two groups of human volunteers (N=26, N=482). We found that generalization of avoidance could be parsed into perceptual and value-based processes, and further, that value-based generalization could be subdivided into that relating to aversive and neutral feedback - with corresponding circuits including primary sensory cortex, anterior insula, and ventromedial prefrontal cortex, respectively. Further, generalization from aversive, but not neutral, feedback was associated with self-reported anxiety and intrusive thoughts. These results reveal a set of distinct mechanisms that mediate generalization in avoidance learning, and show how specific individual differences within them can yield anxiety.

scholarly journals Increased ventromedial prefrontal cortex activity in adolescence benefits prosocial reinforcement learning

2021 ◽  
Author(s):  
Bianca Westhoff ◽  
Neeltje E. Blankenstein ◽  
Elisabeth Schreuders ◽  
Eveline A. Crone ◽  
Anna C. K. van Duijvenvoorde
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Functional Neuroimaging ◽  
Learning Task ◽  
Developmental Trajectory ◽  
Ventromedial Prefrontal Cortex ◽  
Prediction Errors ◽  
Age Related ◽  
Probabilistic Reinforcement ◽  
Neuroimaging Study

AbstractLearning which of our behaviors benefit others contributes to social bonding and being liked by others. An important period for the development of (pro)social behavior is adolescence, in which peers become more salient and relationships intensify. It is, however, unknown how learning to benefit others develops across adolescence and what the underlying cognitive and neural mechanisms are. In this functional neuroimaging study, we assessed learning for self and others (i.e., prosocial learning) and the concurring neural tracking of prediction errors across adolescence (ages 9-21, N=74). Participants performed a two-choice probabilistic reinforcement learning task in which outcomes resulted in monetary consequences for themselves, an unknown other, or no one. Participants from all ages were able to learn for themselves and others, but learning for others showed a more protracted developmental trajectory. Prediction errors for self were observed in the ventral striatum and showed no age-related differences. However, prediction error coding for others was specifically observed in the ventromedial prefrontal cortex and showed age-related increases. These results reveal insights into the computational mechanisms of learning for others across adolescence, and highlight that learning for self and others show different age-related patterns.

scholarly journals Pure Correlates of Exploration and Exploitation in the Human Brain

2017 ◽  
Author(s):  
Tommy C. Blanchard ◽  
Samuel J. Gershman
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Human Brain ◽  
Anterior Cingulate Cortex ◽  
Fundamental Problem ◽  
Cingulate Cortex ◽  
Ventromedial Prefrontal Cortex ◽  
Anterior Cingulate ◽  
Exploration And Exploitation ◽  
Exploration Exploitation

AbstractBalancing exploration and exploitation is a fundamental problem in reinforcement learning. Previous neuroimaging studies of the exploration-exploitation dilemma could not completely disentangle these two processes, making it difficult to unambiguously identify their neural signatures. We overcome this problem using a task in which subjects can either observe (pure exploration) or bet (pure exploitation). Insula and dorsal anterior cingulate cortex showed significantly greater activity on observe trials compared to bet trials, suggesting that these regions play a role in driving exploration. A model-based analysis of task performance suggested that subjects chose to observe until a critical evidence threshold was reached. We observed a neural signature of this evidence accumulation process in ventromedial prefrontal cortex. These findings support theories positing an important role for anterior cingulate cortex in exploration, while also providing a new perspective on the roles of insula and ventromedial prefrontal cortex.Significance StatementSitting down at a familiar restaurant, you may choose to order an old favorite or sample a new dish. In reinforcement learning theory, this is known as the exploration-exploitation dilemma. The optimal solution is known to be intractable; therefore, humans must use heuristic strategies. Behavioral studies have revealed several candidate strategies, but identifying the neural mechanisms underlying these strategies is complicated due to the fact that exploration and exploitation are not perfectly dissociable in standard tasks. Using an “observe or bet” task, we identify for the first time pure neural correlates of exploration and exploitation in the human brain.

scholarly journals Value generalization in human avoidance learning

eLife ◽  
2018 ◽  
Vol 7 ◽  
Author(s):  
Agnes Norbury ◽  
Trevor W Robbins ◽  
Ben Seymour
Keyword(s):  
Decision Making ◽  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Avoidance Learning ◽  
Anterior Insula ◽  
Ventromedial Prefrontal Cortex ◽  
Sensory Cortex ◽  
Primary Sensory Cortex ◽  
Human Volunteers ◽  
Mediate Generalization

Generalization during aversive decision-making allows us to avoid a broad range of potential threats following experience with a limited set of exemplars. However, over-generalization, resulting in excessive and inappropriate avoidance, has been implicated in a variety of psychological disorders. Here, we use reinforcement learning modelling to dissect out different contributions to the generalization of instrumental avoidance in two groups of human volunteers (N = 26, N = 482). We found that generalization of avoidance could be parsed into perceptual and value-based processes, and further, that value-based generalization could be subdivided into that relating to aversive and neutral feedback − with corresponding circuits including primary sensory cortex, anterior insula, amygdala and ventromedial prefrontal cortex. Further, generalization from aversive, but not neutral, feedback was associated with self-reported anxiety and intrusive thoughts. These results reveal a set of distinct mechanisms that mediate generalization in avoidance learning, and show how specific individual differences within them can yield anxiety.

Age differences in ventromedial prefrontal cortex functional connectivity during socioemotional content processing

2020 ◽  
Vol 48 (7) ◽  
pp. 1-19
Author(s):  
Ryan T. Daley ◽  
Holly J. Bowen ◽  
Eric C. Fields ◽  
Angela Gutchess ◽  
Elizabeth A. Kensinger
Keyword(s):  
Prefrontal Cortex ◽  
Functional Connectivity ◽  
Inferior Frontal Gyrus ◽  
Age Groups ◽  
Emotional Content ◽  
Ventromedial Prefrontal Cortex ◽  
Anterior Cingulate ◽  
Supramarginal Gyrus ◽  
Age Related ◽  
Content Processing

Self-relevance effects are often confounded by the presence of emotional content, rendering it difficult to determine how brain networks functionally connected to the ventromedial prefrontal cortex (vmPFC) are affected by the independent contributions of self-relevance and emotion. This difficulty is complicated by age-related changes in functional connectivity between the vmPFC and other default mode network regions, and regions typically associated with externally oriented networks. We asked groups of younger and older adults to imagine placing emotional and neutral objects in their home or a stranger's home. An age-invariant vmPFC cluster showed increased activation for self-relevant and emotional content processing. Functional connectivity analyses revealed age × self-relevance interactions in vmPFC connectivity with the anterior cingulate cortex. There were also age × emotion interactions in vmPFC functional connectivity with the anterior insula, orbitofrontal gyrus, inferior frontal gyrus, and supramarginal gyrus. Interactions occurred in regions with the greatest differences between the age groups, as revealed by conjunction analyses. Implications of the findings are discussed.

Sign in / Sign up

Export Citation Format

Share Document