The role of reinforcement learning models to assess decision-making in the Iowa Gambling Task under the influence of alcohol

AbstractWhat people report is, at times, the best evidence we have for what they experience. Newell & Shanks (N&S) do a service for debates regarding the role of unconscious influences on decision making by offering some sound methodological recommendations. We doubt, however, that those recommendations go far enough. For even if people have knowledge of the factors that influence their decisions, it does not follow that such knowledge is conscious, and plays a causal role, at the time the decision is made. Moreover, N&S fail to demonstrate that unconscious thought plays no role at all in decision making. Indeed, such a claim is quite implausible. In making these points we comment on their discussion of the literature on expertise acquisition and the Iowa Gambling Task.

Download Full-text

The Orbitofrontal Cortex Represents Advantageous Choice in the Iowa Gambling Task

10.21203/rs.3.rs-523769/v1 ◽

2021 ◽

Author(s):

Rujing Zha ◽

Peng Li ◽

Ying Li ◽

Nan Li ◽

Meijun Gao ◽

...

Keyword(s):

Decision Making ◽

Orbitofrontal Cortex ◽

Iowa Gambling Task ◽

Real Life ◽

Binary Choice ◽

Learning Tools ◽

Gambling Task ◽

Complex Decision Making ◽

Complex Decision

Abstract A good-based model proposes that the orbitofrontal cortex (OFC) represents binary choice outcome, i.e., the chosen good. Previous studies have found that the OFC represents the binary choice outcome in decision-making tasks involving commodity type, cost, risk, and delay. Real-life decisions are often complex and involve uncertainty, rewards, and penalties; however, whether the OFC represents binary choice outcomes in a such decision-making situation, e.g., Iowa gambling task (IGT), remains unclear. Here, we propose that the OFC represents binary choice outcome, i.e., advantageous choice versus disadvantageous choice, in the IGT. We propose two hypotheses: first, the activity pattern in the human OFC represents an advantageous choice; and second, choice induces an OFC-related functional network. Using functional magnetic resonance imaging and advanced machine learning tools, we found that the OFC represented an advantageous choice in the IGT. The OFC representation of advantageous choice was related to decision-making performance. Choice modulated the functional connectivity between the OFC and the superior medial gyrus. In conclusion, the OFC represents an advantageous choice during the IGT. In the framework of a good-based model, the results extend the role of the OFC to complex decision-making when making a binary choice.

Download Full-text

Value-free reinforcement learning: Policy optimization as a minimal model of operant behavior

10.31234/osf.io/ew58m ◽

2021 ◽

Author(s):

Daniel Bennett ◽

Yael Niv ◽

Angela Langdon

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Operant Behavior ◽

Intermediate Step ◽

Learning Models ◽

Policy Gradient ◽

Reinforcement Learning Models ◽

Policy Optimization ◽

Value Learning ◽

Gradient Models

Reinforcement learning is a powerful framework for modelling the cognitive and neural substrates of learning and decision making. Contemporary research in cognitive neuroscience and neuroeconomics typically uses value-based reinforcement-learning models, which assume that decision-makers choose by comparing learned values for different actions. However, another possibility is suggested by a simpler family of models, called policy-gradient reinforcement learning. Policy-gradient models learn by optimizing a behavioral policy directly, without the intermediate step of value-learning. Here we review recent behavioral and neural findings that are more parsimoniously explained by policy-gradient models than by value-based models. We conclude that, despite the ubiquity of `value' in reinforcement-learning models of decision making, policy-gradient models provide a lightweight and compelling alternative model of operant behavior.

Download Full-text

Understanding Human Decision Making in an Interactive Landslide Simulator Tool via Reinforcement Learning

Frontiers in Psychology ◽

10.3389/fpsyg.2020.499422 ◽

2021 ◽

Vol 11 ◽

Author(s):

Pratik Chaturvedi ◽

Varun Dutt

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Computational Models ◽

Random Model ◽

Learning Models ◽

Human Decision ◽

Ev Model ◽

Reinforcement Learning Models

Prior research has used an Interactive Landslide Simulator (ILS) tool to investigate human decision making against landslide risks. It has been found that repeated feedback in the ILS tool about damages due to landslides causes an improvement in human decisions against landslide risks. However, little is known on how theories of learning from feedback (e.g., reinforcement learning) would account for human decisions in the ILS tool. The primary goal of this paper is to account for human decisions in the ILS tool via computational models based upon reinforcement learning and to explore the model mechanisms involved when people make decisions in the ILS tool. Four different reinforcement-learning models were developed and evaluated in their ability to capture human decisions in an experiment involving two conditions in the ILS tool. The parameters of an Expectancy-Valence (EV) model, two Prospect-Valence-Learning models (PVL and PVL-2), a combination EV-PU model, and a random model were calibrated to human decisions in the ILS tool across the two conditions. Later, different models with their calibrated parameters were generalized to data collected in an experiment involving a new condition in ILS. When generalized to this new condition, the PVL-2 model’s parameters of both damage-feedback conditions outperformed all other RL models (including the random model). We highlight the implications of our results for decision making against landslide risks.

Download Full-text

Role of Cortisol and Testosterone in Risky Decision-Making: Deciphering Male Decision-Making in the Iowa Gambling Task

Frontiers in Neuroscience ◽

10.3389/fnins.2021.631195 ◽

2021 ◽

Vol 15 ◽

Author(s):

Varsha Singh

Keyword(s):

Decision Making ◽

Working Memory ◽

Iowa Gambling Task ◽

Stress Hormones ◽

Hierarchical Regression ◽

Gambling Task ◽

Risky Decision ◽

Cognitive Benefits

Despite the widely observed high risk-taking behaviors in males, studies using the Iowa gambling task (IGT) have suggested that males choose safe long-term rewards over risky short-term rewards. The role of sex and stress hormones in male decision-making is examined in the initial uncertainty and the latter risk phase of the IGT. The task was tested at peak hormone activity, with breath counting to facilitate cortisol regulation and its cognitive benefits. Results from IGT decision-making before and after counting with saliva samples from two all-male groups (breath vs. number counting) indicated that cortisol declined independent of counting. IGT decision-making showed phase-specific malleability: alteration in the uncertainty phase and stability in the risk phase. Working memory showed alteration, whereas inhibition task performance remained stable, potentially aligning with the phase-specific demands of working memory and inhibition. The results of hierarchical regression for the uncertainty and risk trials indicated that testosterone improved the model fit, cortisol was detrimental for decision-making in uncertainty, and decision-making in the risk trials was benefitted by testosterone. Cortisol regulation accentuated hormones’ phase-specific effects on decision-making. Aligned with the dual-hormone hypothesis, sex, and stress hormones might jointly regulate male long-term decision-making in the IGT.

Download Full-text