Towards Human–Robot Teams: Model-Based Analysis of Human Decision Making in Two-Alternative Choice Tasks With Social Feedback

Human decisions can be biased by irrelevant information. For example, choices between two preferred alternatives can be swayed by a third option that is inferior or unavailable. Previous work has identified three classic biases, known as the attraction, similarity and compromise effects, which arise during choices between economic alternatives defined by two attributes. However, the reliability, interrelationship, and computational origin of these three biases has been controversial. Here, a large cohort of human participants made incentive-compatible choices among assets that varied in price and quality. Instead of focusing on the three classic effects, we sampled decoy stimuli exhaustively across bidimensional multi-attribute space and constructed a full map of decoy influence on choices between two otherwise preferred target items. Our analysis revealed that the decoy influence map was highly structured even beyond the three classic biases. We identified a very simple model that can fully reproduce the decoy influence map and capture its variability in individual participants. This model reveals that the three decoy effects are not distinct phenomena but are all special cases of a more general principle, by which attribute values are repulsed away from the context provided by rival options. The model helps understand why the biases are typically correlated across participants and allows us to validate a new prediction about their interrelationship. This work helps to clarify the origin of three of the most widely-studied biases in human decision-making.

Download Full-text

Optimal Decision Making on the Basis of Evidence Represented in Spike Trains

Neural Computation ◽

10.1162/neco.2009.05-09-1025 ◽

2010 ◽

Vol 22 (5) ◽

pp. 1113-1148 ◽

Cited By ~ 10

Author(s):

Jiaxiang Zhang ◽

Rafal Bogacz

Keyword(s):

Decision Making ◽

Spike Trains ◽

Decision Procedures ◽

Optimal Decision ◽

Ratio Test ◽

Perceptual Decision Making ◽

Sensory Evidence ◽

Optimal Decision Making ◽

Alternative Choice ◽

Choice Tasks

Experimental data indicate that perceptual decision making involves integration of sensory evidence in certain cortical areas. Theoretical studies have proposed that the computation in neural decision circuits approximates statistically optimal decision procedures (e.g., sequential probability ratio test) that maximize the reward rate in sequential choice tasks. However, these previous studies assumed that the sensory evidence was represented by continuous values from gaussian distributions with the same variance across alternatives. In this article, we make a more realistic assumption that sensory evidence is represented in spike trains described by the Poisson processes, which naturally satisfy the mean-variance relationship observed in sensory neurons. We show that for such a representation, the neural circuits involving cortical integrators and basal ganglia can approximate the optimal decision procedures for two and multiple alternative choice tasks.

Download Full-text

Domestic dogs' gaze and behavior in 2-alternative choice tasks

10.31234/osf.io/pt3vh ◽

2021 ◽

Author(s):

Julia Espinosa ◽

Daphna Buchsbaum

Keyword(s):

Decision Making ◽

Visual Discrimination ◽

Visual Information ◽

Choice Behavior ◽

Choice Task ◽

Domestic Dogs ◽

The World ◽

Alternative Choice ◽

And Behavior ◽

Choice Tasks

Species such as humans rely on their excellent visual abilities to perceive and navigate the world. Dogs have co-habited with humans for millennia, yet we know little about how they gather and use visual information to guide decision-making. Across five experiments, we presented pet dogs (N=49) with two foods of unequal value in a 2-alternative choice task, and measured whether dogs showed preferential gazing, and whether visual attention patterns were associated with item choice. Overall, dogs looked significantly longer at the preferred (high value) food over the low value alternative. There was also evidence of item-dependent predictive gaze—dogs looked proportionally longer at the item they subsequently chose. Surprisingly, dogs’ choice behavior was only slightly above chance, despite visual discrimination. These results suggest that dogs use visual information in the environment to inform their choice behavior, but that other factors may also contribute to their decision-making.

Download Full-text

Urban Electric Vehicle Fast-Charging Demand Forecasting Model Based on Data-Driven Approach and Human Decision-Making Behavior

Energies ◽

10.3390/en13061412 ◽

2020 ◽

Vol 13 (6) ◽

pp. 1412 ◽

Cited By ~ 1

Author(s):

Qiang Xing ◽

Zhong Chen ◽

Ziqi Zhang ◽

Xiao Xu ◽

Tian Zhang ◽

...

Keyword(s):

Decision Making ◽

Electric Vehicles ◽

Demand Forecasting ◽

Model Based ◽

Fast Charging ◽

Human Decision ◽

Data Driven Approach ◽

Demand Forecasting Model ◽

Decision Making Behavior ◽

Charging Demand

Electric vehicles (EVs) have attracted growing attention in recent years. However, most existing research has not utilized actual traffic data and has not considered real psychological decision-making of owners in analyzing the charging demand. On this basis, an urban EV fast-charging demand forecasting model based on a data-driven approach and human decision-making behavior is presented in this paper. In this methodology, Didi ride-hailing order trajectory data are firstly taken as the original dataset. Through data mining and fusion technology, the regenerated data and rules of traffic operation are obtained. Then, the single EV model with driving and charging behavior parameters is established. Furthermore, a human behavior decision-making model based on Regret Theory is introduced, which comprises the utility of time consumption and charging cost to plan driving paths and recommend fast-charging stations for vehicles. The rules obtained from data mining together with established models are combined to construct the ‘Electric Vehicles–Power Grid–Traffic Network’ fusion architecture. At last, the actual urban traffic network in Nanjing is selected as an example to design the fast-charging demand load experiments in different scenarios. The results demonstrate that this proposed model is able to effectively predict the spatio-temporal distribution characteristics of urban fast-charging demands, and it more realistically simulates the decision-making psychology of owners’ charging behavior.

Download Full-text

The Actor-Critic Learning Is Behind the Matching Law: Matching Versus Optimal Behaviors

Neural Computation ◽

10.1162/neco.2008.20.1.227 ◽

2008 ◽

Vol 20 (1) ◽

pp. 227-251 ◽

Cited By ~ 31

Author(s):

Yutaka Sakai ◽

Tomoki Fukai

Keyword(s):

Decision Making ◽

Variable Interval ◽

Correct Choice ◽

Matching Law ◽

Neural Systems ◽

Schedules Of Reinforcement ◽

Neural Basis ◽

Schedule Task ◽

Alternative Choice ◽

Choice Tasks

The ability to make a correct choice of behavior from various options is crucial for animals' survival. The neural basis for the choice of behavior has been attracting growing attention in research on biological and artificial neural systems. Alternative choice tasks with variable ratio (VR) and variable interval (VI) schedules of reinforcement have often been employed in studying decision making by animals and humans. In the VR schedule task, alternative choices are reinforced with different probabilities, and subjects learn to select the behavioral response rewarded more frequently. In the VI schedule task, alternative choices are reinforced at different average intervals independent of the choice frequencies, and the choice behavior follows the so-called matching law. The two policies appear robustly in subjects' choice of behavior, but the underlying neural mechanisms remain unknown. Here, we show that these seemingly different policies can appear from a common computational algorithm known as actor-critic learning. We present experimentally testable variations of the VI schedule in which the matching behavior gives only a suboptimal solution to decision making and show that the actor-critic system exhibits the matching behavior in the steady state of the learning even when the matching behavior is suboptimal. However, it is found that the matching behavior can earn approximately the same reward as the optimal one in many practical situations.

Download Full-text

Model-based decision-making and its metacontrol in childhood

10.31234/osf.io/ervsb ◽

2020 ◽

Cited By ~ 1

Author(s):

Claire Rosalie Smid ◽

Wouter Kool ◽

Tobias U. Hauser ◽

Nikolaus Steinbeis

Keyword(s):

Decision Making ◽

Computational Cost ◽

Cost Benefit ◽

Optimal Decision ◽

Developmental Studies ◽

Free System ◽

Model Based ◽

Model Free ◽

Human Decision ◽

Optimal Decision Making

Human decision-making is underpinned by distinct systems that differ in their flexibility and associated computational cost. A widely accepted dichotomy distinguishes a flexible but costly model-based system and a cheap but rigid model-free system. Optimal decision-making requires adaptive arbitration between these two systems depending on environmental demands. Previous developmental studies suggest that model-based decision-making only emerges in adolescence. Here, we show that when using a paradigm more conducive to model-based decision-making, children as young as 5 years show contributions from a model-based system to their behaviour. Furthermore, we find that between the ages 5 to 11, children demonstrate increasing metacontrol, which is the engagement of cost-benefit arbitration over decision-making systems on a trial-by-trial basis. Our results suggest that model-based decision-making emerges much earlier than previously believed, while adaptive arbitration between computationally cheap and costly systems continues to undergo developmental changes during childhood.

Download Full-text

Model-based hierarchical reinforcement learning and human action control

Philosophical Transactions of the Royal Society B Biological Sciences ◽

10.1098/rstb.2013.0480 ◽

2014 ◽

Vol 369 (1655) ◽

pp. 20130480 ◽

Cited By ~ 63

Author(s):

Matthew Botvinick ◽

Ari Weinstein

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Hierarchical Model ◽

Human Action ◽

Action Control ◽

Computational Framework ◽

Hierarchical Reinforcement Learning ◽

Model Based ◽

Human Decision ◽

Areas Of Interest

Recent work has reawakened interest in goal-directed or ‘model-based’ choice, where decisions are based on prospective evaluation of potential action outcomes. Concurrently, there has been growing attention to the role of hierarchy in decision-making and action control. We focus here on the intersection between these two areas of interest, considering the topic of hierarchical model-based control. To characterize this form of action control, we draw on the computational framework of hierarchical reinforcement learning, using this to interpret recent empirical findings. The resulting picture reveals how hierarchical model-based mechanisms might play a special and pivotal role in human decision-making, dramatically extending the scope and complexity of human behaviour.

Download Full-text