Quantum reinforcement learning during human decision-making

Reinforcement learning (RL) models have advanced our understanding of how animals learn and make decisions, and how the brain supports some aspects of learning. However, the neural computations that are explained by RL algorithms fall short of explaining many sophisticated aspects of human decision making, including the generalization of learned information, one-shot learning, and the synthesis of task information in complex environments. Instead, these aspects of instrumental behavior are assumed to be supported by the brain’s executive functions (EF). We review recent findings that highlight the importance of EF in learning. Specifically, we advance the theory that EF sets the stage for canonical RL computations in the brain, providing inputs that broaden their flexibility and applicability. Our theory has important implications for how to interpret RL computations in the brain and behavior.

Download Full-text

Invited: Efficient Reinforcement Learning for Automating Human Decision-Making in SoC Design

2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC) ◽

10.1109/dac.2018.8465874 ◽

2018 ◽

Author(s):

Shankar Sadasivam ◽

Jinwon Lee ◽

Zhuo Chen ◽

Rajeev Jain

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Human Decision

Download Full-text

Reinforcement learning and instance-based learning approaches to modeling human decision making in a prognostic foraging task

2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) ◽

10.1109/devlrn.2015.7346127 ◽

2015 ◽

Author(s):

Suhas E. Chelian ◽

Jaehyon Paik ◽

Peter Pirolli ◽

Christian Lebiere ◽

Rajan Bhattacharyya

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Learning Approaches ◽

Instance Based Learning ◽

Human Decision ◽

Foraging Task

Download Full-text

Automatic discovery of interpretable planning strategies

Machine Learning ◽

10.1007/s10994-021-05963-2 ◽

2021 ◽

Author(s):

Julian Skirzyński ◽

Frederic Becker ◽

Falk Lieder

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Information Acquisition ◽

Decision Aids ◽

Decision Rules ◽

Sequential Decision ◽

Behavioral Experiments ◽

Good Decision ◽

Human Decision ◽

Planning Strategies

AbstractWhen making decisions, people often overlook critical information or are overly swayed by irrelevant information. A common approach to mitigate these biases is to provide decision-makers, especially professionals such as medical doctors, with decision aids, such as decision trees and flowcharts. Designing effective decision aids is a difficult problem. We propose that recently developed reinforcement learning methods for discovering clever heuristics for good decision-making can be partially leveraged to assist human experts in this design process. One of the biggest remaining obstacles to leveraging the aforementioned methods for improving human decision-making is that the policies they learn are opaque to people. To solve this problem, we introduce AI-Interpret: a general method for transforming idiosyncratic policies into simple and interpretable descriptions. Our algorithm combines recent advances in imitation learning and program induction with a new clustering method for identifying a large subset of demonstrations that can be accurately described by a simple, high-performing decision rule. We evaluate our new AI-Interpret algorithm and employ it to translate information-acquisition policies discovered through metalevel reinforcement learning. The results of three large behavioral experiments showed that providing the decision rules generated by AI-Interpret as flowcharts significantly improved people’s planning strategies and decisions across three different classes of sequential decision problems. Moreover, our fourth experiment revealed that this approach is significantly more effective at improving human decision-making than training people by giving them performance feedback. Finally, a series of ablation studies confirmed that our AI-Interpret algorithm was critical to the discovery of interpretable decision rules and that it is ready to be applied to other reinforcement learning problems. We conclude that the methods and findings presented in this article are an important step towards leveraging automatic strategy discovery to improve human decision-making. The code for our algorithm and the experiments is available at https://github.com/RationalityEnhancement/InterpretableStrategyDiscovery.

Download Full-text

A predictive reinforcement learning framework for modeling human decision making behavior

2009 14th International CSI Computer Conference ◽

10.1109/csicc.2009.5349626 ◽

2009 ◽

Author(s):

Rezvan Kianifar ◽

Farzad Towhidkhah

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Learning Framework ◽

Human Decision ◽

Decision Making Behavior

Download Full-text

Model-based hierarchical reinforcement learning and human action control

Philosophical Transactions of the Royal Society B Biological Sciences ◽

10.1098/rstb.2013.0480 ◽

2014 ◽

Vol 369 (1655) ◽

pp. 20130480 ◽

Cited By ~ 63

Author(s):

Matthew Botvinick ◽

Ari Weinstein

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Hierarchical Model ◽

Human Action ◽

Action Control ◽

Computational Framework ◽

Hierarchical Reinforcement Learning ◽

Model Based ◽

Human Decision ◽

Areas Of Interest

Recent work has reawakened interest in goal-directed or ‘model-based’ choice, where decisions are based on prospective evaluation of potential action outcomes. Concurrently, there has been growing attention to the role of hierarchy in decision-making and action control. We focus here on the intersection between these two areas of interest, considering the topic of hierarchical model-based control. To characterize this form of action control, we draw on the computational framework of hierarchical reinforcement learning, using this to interpret recent empirical findings. The resulting picture reveals how hierarchical model-based mechanisms might play a special and pivotal role in human decision-making, dramatically extending the scope and complexity of human behaviour.

Download Full-text

Understanding Human Decision Making in an Interactive Landslide Simulator Tool via Reinforcement Learning

Frontiers in Psychology ◽

10.3389/fpsyg.2020.499422 ◽

2021 ◽

Vol 11 ◽

Author(s):

Pratik Chaturvedi ◽

Varun Dutt

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Computational Models ◽

Random Model ◽

Learning Models ◽

Human Decision ◽

Ev Model ◽

Reinforcement Learning Models

Prior research has used an Interactive Landslide Simulator (ILS) tool to investigate human decision making against landslide risks. It has been found that repeated feedback in the ILS tool about damages due to landslides causes an improvement in human decisions against landslide risks. However, little is known on how theories of learning from feedback (e.g., reinforcement learning) would account for human decisions in the ILS tool. The primary goal of this paper is to account for human decisions in the ILS tool via computational models based upon reinforcement learning and to explore the model mechanisms involved when people make decisions in the ILS tool. Four different reinforcement-learning models were developed and evaluated in their ability to capture human decisions in an experiment involving two conditions in the ILS tool. The parameters of an Expectancy-Valence (EV) model, two Prospect-Valence-Learning models (PVL and PVL-2), a combination EV-PU model, and a random model were calibrated to human decisions in the ILS tool across the two conditions. Later, different models with their calibrated parameters were generalized to data collected in an experiment involving a new condition in ILS. When generalized to this new condition, the PVL-2 model’s parameters of both damage-feedback conditions outperformed all other RL models (including the random model). We highlight the implications of our results for decision making against landslide risks.

Download Full-text