Context-Dependent Reinforcement Learning

2015 ◽  
Author(s):  
Oussama H. Hamid
Keyword(s):  
Reinforcement Learning ◽  
Context Dependent

scholarly journals Neurostimulation Reveals Context-Dependent Arbitration Between Model-Based and Model-Free Reinforcement Learning

2019 ◽  
Vol 29 (11) ◽  
pp. 4850-4862 ◽  
Author(s):  
Sebastian Weissengruber ◽  
Sang Wan Lee ◽  
John P O’Doherty ◽  
Christian C Ruff
Keyword(s):  
Reinforcement Learning ◽  
Neural Mechanism ◽  
Ventrolateral Prefrontal Cortex ◽  
Control Behavior ◽  
Model Based ◽  
Model Free ◽  
Context Dependent ◽  
The Brain ◽  
Over Time ◽  
Environmental Demands

Abstract While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems. The opposing neural interventions resulted in a bidirectional shift of control between MB and MF learning. Stimulation also affected the sensitivity of the arbitration mechanism itself, as it changed how often subjects switched between the dominant system over time. Both of these effects depended on varying task contexts that either favored MB or MF control, indicating that this arbitration mechanism is not context-invariant but flexibly incorporates information about current environmental demands.

scholarly journals Flexibility in valenced reinforcement learning computations across development

2021 ◽  
Author(s):  
Kate Nussenbaum ◽  
Juan A. Velez ◽  
Bradli T. Washington ◽  
Hannah E. Hamling ◽  
Catherine A. Hartley
Keyword(s):  
Reinforcement Learning ◽  
Mixed Race ◽  
Positive Outcomes ◽  
Negative Outcomes ◽  
Optimal Integration ◽  
Adolescents And Adults ◽  
Context Dependent ◽  
Expected Outcomes ◽  
Better Than

Optimal integration of positive and negative outcomes during learning varies depending on an environment’s reward statistics. The present study investigated the extent to which children, adolescents, and adults (N = 142 8 - 25 year-olds, 55% female, 42% White, 31% Asian, 17% mixed race, and 8% Black) adapt their weighting of better-than-expected and worse-than-expected outcomes when learning from reinforcement. Participants made a series of choices across two contexts: one in which weighting positive outcomes more heavily than negative outcomes led to better performance, and one in which the reverse was true. Reinforcement learning modeling revealed that across age, participants shifted their valence biases in accordance with the structure of the environment. Exploratory analyses revealed increases in context-dependent flexibility with age.

scholarly journals Context-dependent outcome encoding in human reinforcement learning

2021 ◽  
Vol 41 ◽  
pp. 144-151
Author(s):  
Stefano Palminteri ◽  
Maël Lebreton
Keyword(s):  
Reinforcement Learning ◽  
Context Dependent

scholarly journals Context-dependent reinforcement learning impairment in depression

2021 ◽  
Author(s):  
henri Vandendriessche ◽  
Amel Demmou ◽  
Sophie Bavard ◽  
Julien Yadak ◽  
Cédric Lemogne ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Positive Reinforcement ◽  
Original Learning ◽  
Expected Value ◽  
Learning Performance ◽  
Transfer Phase ◽  
Learning Context ◽  
Context Analysis ◽  
The Poor ◽  
Context Dependent

Backgrounds:Value-based decision-making impairment in depression is a complex phenomenon: while some studies did find evidence of blunted reward learning and reward-related signals in the brain, others indicate no effect. Here we test whether such reward sensitivity deficits are dependent on the overall value of the decision problem.Methods:We used a two-armed bandit task that includes two different contexts: one ‘rich’ context where both options were associated with an overall positive expected value and a ‘poor’ context where options were associated with overall negative expected value. We tested patients (N=30) undergoing a major depressive episode and age, gender and socio-economically matched controls (N=26). To assess whether differences in learning performance were due to a decision or a value-update process, we also analysed performance in a transfer phase, performed immediately after the learning phase. ResultsHealthy subjects showed similar learning performance in the ‘rich’ and the ‘poor’ contexts, while patients showed reduced learning in the ‘poor’ context. Analysis of the transfer phase showed that the context-dependent deficit in patients generalized when options were extrapolated from their original learning context, thus suggesting that the effect of depression has to be traced to the outcome encoding, rather than the decision phase.ConclusionsOur results illustrate that reinforcement learning deficits in depression are complex and depend on the value of the context. We show that depressive patients have a specific trouble in contexts with an overall negative state value, supporting the relevance of setting up patients in a spiral of positive reinforcement.

scholarly journals A Biologically Plausible Architecture of the Striatum to Solve Context-Dependent Reinforcement Learning Tasks

2017 ◽  
Vol 11 ◽  
Author(s):  
Sabyasachi Shivkumar ◽  
Vignesh Muralidharan ◽  
V. Srinivasa Chakravarthy
Keyword(s):  
Reinforcement Learning ◽  
Learning Tasks ◽  
Context Dependent
Sign in / Sign up

Export Citation Format

Share Document