scholarly journals Putting value in context: A role for context memory in decisions for reward

2015 ◽  
Author(s):  
Aaron M. Bornstein ◽  
Kenneth A. Norman
Keyword(s):  
Reinforcement Learning ◽  
Choice Behavior ◽  
Model Free ◽  
Episodic Memories ◽  
Made In ◽  
Context Memory

AbstractHow does experience inform decisions? In episodic sampling, decisions are guided by a few episodic memories of past choices. This process can yield choice patterns similar to model-free Reinforcement Learning (RL); however, samples can vary from trial to trial, causing decisions to vary. Here, we show that context retrieved during episodic sampling can cause choice behavior to deviate sharply from the predictions of RL. Specifically, we show that, when a given memory is sampled, choices (in the present) are influenced by the properties of other decisions made in the same context as the sampled event. This effect is mediated by fMRI measures of context retrieval on each trial, suggesting a mechanism whereby cues trigger retrieval of context, which then triggers retrieval of other decisions from that context. This result establishes a new avenue by which experience can guide choice, and as such has broad implications for the study of decisions.

scholarly journals Cognitive Control Predicts Use of Model-based Reinforcement Learning

2015 ◽  
Vol 27 (2) ◽  
pp. 319-333 ◽  
Author(s):  
A. Ross Otto ◽  
Anya Skatova ◽  
Seth Madlon-Kay ◽  
Nathaniel D. Daw
Keyword(s):  
Reinforcement Learning ◽  
Cognitive Control ◽  
Choice Behavior ◽  
Contextual Information ◽  
Neural Substrates ◽  
Model Based ◽  
Controlled Choice ◽  
Model Free ◽  
Predict Model ◽  
Computational Processes

Accounts of decision-making and its neural substrates have long posited the operation of separate, competing valuation systems in the control of choice behavior. Recent theoretical and experimental work suggest that this classic distinction between behaviorally and neurally dissociable systems for habitual and goal-directed (or more generally, automatic and controlled) choice may arise from two computational strategies for reinforcement learning (RL), called model-free and model-based RL, but the cognitive or computational processes by which one system may dominate over the other in the control of behavior is a matter of ongoing investigation. To elucidate this question, we leverage the theoretical framework of cognitive control, demonstrating that individual differences in utilization of goal-related contextual information—in the service of overcoming habitual, stimulus-driven responses—in established cognitive control paradigms predict model-based behavior in a separate, sequential choice task. The behavioral correspondence between cognitive control and model-based RL compellingly suggests that a common set of processes may underpin the two behaviors. In particular, computational mechanisms originally proposed to underlie controlled behavior may be applicable to understanding the interactions between model-based and model-free choice behavior.

Shaping Model-Free Reinforcement-Learning with Model-Based Pseudorewards

2018 ◽  
Author(s):  
Paul Krueger ◽  
Thomas Griffiths
Keyword(s):  
Reinforcement Learning ◽  
Model Based ◽  
Model Free

Model-Based and Model-Free Social Cognition

2019 ◽  
Author(s):  
Leor M Hackel ◽  
Jeffrey Jordan Berg ◽  
Björn Lindström ◽  
David Amodio
Keyword(s):  
Reinforcement Learning ◽  
Social Cognition ◽  
Learning Strategies ◽  
Memory Systems ◽  
Learning Task ◽  
Financial Advisors ◽  
Model Based ◽  
Model Free ◽  
Systems Model ◽  
Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

Model-Free Event-Triggered Optimal Consensus Control of Multiple Euler-Lagrange Systems via Reinforcement Learning

2020 ◽  
pp. 1-1
Author(s):  
Saiwei Wang ◽  
Xin Jin ◽  
Shuai Mao ◽  
Athanasios V. Vasilakos ◽  
Yang Tang
Keyword(s):  
Reinforcement Learning ◽  
Consensus Control ◽  
Model Free ◽  
Event Triggered

scholarly journals Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems

2021 ◽  
Author(s):  
Amarildo Likmeta ◽  
Alberto Maria Metelli ◽  
Giorgia Ramponi ◽  
Andrea Tirinzoni ◽  
Matteo Giuliani ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Real Life ◽  
User Preferences ◽  
Inverse Reinforcement Learning ◽  
Water Release ◽  
Reward Function ◽  
Model Free ◽  
Conflicting Objectives ◽  
Multiple Experts

AbstractIn real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.

scholarly journals Experimental evaluation of model-free reinforcement learning algorithms for continuous HVAC control

2021 ◽  
Vol 298 ◽  
pp. 117164
Author(s):  
Marco Biemann ◽  
Fabian Scheller ◽  
Xiufeng Liu ◽  
Lizhen Huang
Keyword(s):  
Reinforcement Learning ◽  
Experimental Evaluation ◽  
Learning Algorithms ◽  
Model Free ◽  
Hvac Control

A Model-Free Distributed Cooperative Frequency Control Strategy for MT-HVDC Systems Using Reinforcement Learning Method

2021 ◽  
Author(s):  
Zhong-Jie Hu ◽  
Zhi-Wei Liu ◽  
Chaojie Li ◽  
Tingwen Huang ◽  
Xiong Hu
Keyword(s):  
Reinforcement Learning ◽  
Control Strategy ◽  
Frequency Control ◽  
Learning Method ◽  
Model Free

scholarly journals A set-based model-free reinforcement learning design technique for nonlinear systems

2018 ◽  
Vol 51 (18) ◽  
pp. 37-42
Author(s):  
Martin Guay ◽  
Khalid Tourkey Atta
Keyword(s):  
Reinforcement Learning ◽  
Nonlinear Systems ◽  
Learning Design ◽  
Design Technique ◽  
Model Free
Sign in / Sign up

Export Citation Format

Share Document