scholarly journals Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Batel Yifrah ◽  
Ayelet Ramaty ◽  
Genela Morris ◽  
Avi Mendelsohn
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Declarative Memory ◽  
Contextual Information ◽  
Memory Performance ◽  
Relevant Information ◽  
Subjective Memory ◽  
Types Of Information ◽  
Reinforcement Learning Models ◽  
Implicit And Explicit

AbstractDecision making can be shaped both by trial-and-error experiences and by memory of unique contextual information. Moreover, these types of information can be acquired either by means of active experience or by observing others behave in similar situations. The interactions between reinforcement learning parameters that inform decision updating and memory formation of declarative information in experienced and observational learning settings are, however, unknown. In the current study, participants took part in a probabilistic decision-making task involving situations that either yielded similar outcomes to those of an observed player or opposed them. By fitting alternative reinforcement learning models to each subject, we discerned participants who learned similarly from experience and observation from those who assigned different weights to learning signals from these two sources. Participants who assigned different weights to their own experience versus those of others displayed enhanced memory performance as well as subjective memory strength for episodes involving significant reward prospects. Conversely, memory performance of participants who did not prioritize their own experience over others did not seem to be influenced by reinforcement learning parameters. These findings demonstrate that interactions between implicit and explicit learning systems depend on the means by which individuals weigh relevant information conveyed via experience and observation.

The Right to Information

2021 ◽  
pp. 64-103
Author(s):  
Helena U. Vrabec
Keyword(s):  
Decision Making ◽  
Data Processing ◽  
Relevant Information ◽  
Legal Basis ◽  
Control Rights ◽  
Data Flows ◽  
Right To Information ◽  
Types Of Information ◽  
The Right ◽  
Automated Decision Making

Chapter 4 addresses the right to information, the cornerstone of the system of control rights under the GDPR and the ePrivacy Directive. The types of information that are likely to provide data subjects the most relevant information about data processing in the context of the data-driven economy are analysed more thoroughly, e.g., the information about the legal basis for data processing, the information about the sources of data, and the details on automated decision-making. The chapter investigates the right to explanation and icons which seem to offer a new, promising option to exercise more control over modern data flows. In the ePrivacy area, the right to information plays an increasingly important role in regulating the use of cookies and similar tracking technologies. The chapter acknowledges that, despite some novel steps in the GDPR, entitlements that the law affords are undermined due to three groups of factors: psychological, technological, and economic.

scholarly journals Value-free reinforcement learning: Policy optimization as a minimal model of operant behavior

2021 ◽  
Author(s):  
Daniel Bennett ◽  
Yael Niv ◽  
Angela Langdon
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Operant Behavior ◽  
Intermediate Step ◽  
Learning Models ◽  
Policy Gradient ◽  
Reinforcement Learning Models ◽  
Policy Optimization ◽  
Value Learning ◽  
Gradient Models

Reinforcement learning is a powerful framework for modelling the cognitive and neural substrates of learning and decision making. Contemporary research in cognitive neuroscience and neuroeconomics typically uses value-based reinforcement-learning models, which assume that decision-makers choose by comparing learned values for different actions. However, another possibility is suggested by a simpler family of models, called policy-gradient reinforcement learning. Policy-gradient models learn by optimizing a behavioral policy directly, without the intermediate step of value-learning. Here we review recent behavioral and neural findings that are more parsimoniously explained by policy-gradient models than by value-based models. We conclude that, despite the ubiquity of `value' in reinforcement-learning models of decision making, policy-gradient models provide a lightweight and compelling alternative model of operant behavior.

scholarly journals Understanding Human Decision Making in an Interactive Landslide Simulator Tool via Reinforcement Learning

2021 ◽  
Vol 11 ◽  
Author(s):  
Pratik Chaturvedi ◽  
Varun Dutt
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Computational Models ◽  
Random Model ◽  
Learning Models ◽  
Human Decision ◽  
Ev Model ◽  
Reinforcement Learning Models

Prior research has used an Interactive Landslide Simulator (ILS) tool to investigate human decision making against landslide risks. It has been found that repeated feedback in the ILS tool about damages due to landslides causes an improvement in human decisions against landslide risks. However, little is known on how theories of learning from feedback (e.g., reinforcement learning) would account for human decisions in the ILS tool. The primary goal of this paper is to account for human decisions in the ILS tool via computational models based upon reinforcement learning and to explore the model mechanisms involved when people make decisions in the ILS tool. Four different reinforcement-learning models were developed and evaluated in their ability to capture human decisions in an experiment involving two conditions in the ILS tool. The parameters of an Expectancy-Valence (EV) model, two Prospect-Valence-Learning models (PVL and PVL-2), a combination EV-PU model, and a random model were calibrated to human decisions in the ILS tool across the two conditions. Later, different models with their calibrated parameters were generalized to data collected in an experiment involving a new condition in ILS. When generalized to this new condition, the PVL-2 model’s parameters of both damage-feedback conditions outperformed all other RL models (including the random model). We highlight the implications of our results for decision making against landslide risks.

scholarly journals Surprise acts as a reducer of outcome value in human reinforcement learning

2019 ◽  
Author(s):  
Motofumi Sumiya ◽  
Kentaro Katahira
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Simulation Analysis ◽  
Learning Task ◽  
Risk Averse ◽  
Choice Preference ◽  
Proposed Model ◽  
Modulation Parameter ◽  
Reinforcement Learning Models

Surprise occurs because of differences between a decision outcome and its predicted outcome (prediction error), regardless of whether the error is positive or negative. It has recently been postulated that surprise affects the reward value of the action outcome itself; studies have indicated that increasing surprise, as absolute value of prediction error, decreases the value of the outcome. However, how surprise affects the value of the outcome and subsequent decision making is unclear. We suggested that, on the assumption that surprise decreases the outcome value, agents will increase their risk averse choices when an outcome is often surprisal. Here, we propose the surprise-sensitive utility model, a reinforcement learning model that states that surprise decreases the outcome value, to explain how surprise affects subsequent decision-making. To investigate the assumption, we compared this model with previous reinforcement learning models on a risky probabilistic learning task with simulation analysis, and model selection with two experimental datasets with different tasks and population. We further simulated a simple decision-making task to investigate how parameters within the proposed model modulate the choice preference. As a result, we found the proposed model explains the risk averse choices in a manner similar to the previous models, and risk averse choices increased as the surprise-based modulation parameter of outcome value increased. The model fits these datasets better than the other models, with same free parameters, thus providing a more parsimonious and robust account for risk averse choices. These findings indicate that surprise acts as a reducer of outcome value and decreases the action value for risky choices in which prediction error often occurs.

Integration of Reinforcement Learning and Optimal Decision-Making Theories of the Basal Ganglia

2011 ◽  
Vol 23 (4) ◽  
pp. 817-851 ◽  
Author(s):  
Rafal Bogacz ◽  
Tobias Larsen
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Reaction Times ◽  
Learning Theories ◽  
Optimal Decision ◽  
Neural Responses ◽  
Sensory Evidence ◽  
Optimal Decision Making ◽  
Reinforcement Learning Models

This article seeks to integrate two sets of theories describing action selection in the basal ganglia: reinforcement learning theories describing learning which actions to select to maximize reward and decision-making theories proposing that the basal ganglia selects actions on the basis of sensory evidence accumulated in the cortex. In particular, we present a model that integrates the actor-critic model of reinforcement learning and a model assuming that the cortico-basal-ganglia circuit implements a statistically optimal decision-making procedure. The values of corico-striatal weights required for optimal decision making in our model differ from those provided by standard reinforcement learning models. Nevertheless, we show that an actor-critic model converges to the weights required for optimal decision making when biologically realistic limits on synaptic weights are introduced. We also describe the model's predictions concerning reaction times and neural responses during learning, and we discuss directions required for further integration of reinforcement learning and optimal decision-making theories.

Learning and Consolidation of Verbal Declarative Memory in Patients with Posttraumatic Stress Disorder

2010 ◽  
Vol 218 (2) ◽  
pp. 135-140 ◽  
Author(s):  
Slawomira J. Diener ◽  
Herta Flor ◽  
Michèle Wessa
Keyword(s):  
Posttraumatic Stress Disorder ◽  
Posttraumatic Stress ◽  
Verbal Memory ◽  
Stress Disorder ◽  
Declarative Memory ◽  
Memory Performance ◽  
Verbal Learning ◽  
Depression Scale ◽  
Learning Performance ◽  
Ptsd Symptoms

Impairments in declarative memory have been reported in posttraumatic stress disorder (PTSD). Fragmentation of explicit trauma-related memory has been assumed to impede the formation of a coherent memorization of the traumatic event and the integration into autobiographic memory. Together with a strong non-declarative memory that connects trauma reminders with a fear response the impairment in declarative memory is thought to be involved in the maintenance of PTSD symptoms. Fourteen PTSD patients, 14 traumatized subjects without PTSD, and 13 non-traumatized healthy controls (HC) were tested with the California Verbal Learning Test (CVLT) to assess verbal declarative memory. PTSD symptoms were assessed with the Clinician Administered PTSD Scale and depression with the Center of Epidemiological Studies Depression Scale. Several indices of the CVLT pointed to an impairment in declarative memory performance in PTSD, but not in traumatized persons without PTSD or HC. No group differences were observed if recall of memory after a time delay was set in relation to initial learning performance. In the PTSD group verbal memory performance correlated significantly with hyperarousal symptoms, after concentration difficulties were accounted for. The present study confirmed previous reports of declarative verbal memory deficits in PTSD. Extending previous results, we propose that learning rather than memory consolidation is impaired in PTSD patients. Furthermore, arousal symptoms may interfere with successful memory formation in PTSD.

Bayes factors for reinforcement-learning models of the Iowa gambling task.

Decision ◽  
2016 ◽  
Vol 3 (2) ◽  
pp. 115-131 ◽  
Author(s):  
Helen Steingroever ◽  
Ruud Wetzels ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Reinforcement Learning ◽  
Iowa Gambling Task ◽  
Bayes Factors ◽  
Gambling Task ◽  
Learning Models ◽  
Reinforcement Learning Models
Sign in / Sign up

Export Citation Format

Share Document