scholarly journals Reward-driven changes in striatal pathway competition shape evidence evaluation in decision-making

2018 ◽  
Author(s):  
Kyle Dunovan ◽  
Catalina Vich ◽  
Matthew Clapp ◽  
Timothy Verstynen ◽  
Jonathan Rubin
Keyword(s):  
Decision Making ◽  
Basal Ganglia ◽  
Diffusion Model ◽  
Process Models ◽  
Decision Task ◽  
Drift Diffusion Model ◽  
Indirect Pathway ◽  
Drift Diffusion ◽  
Pathway Activity ◽  
Evidence Accumulation

AbstractCortico-basal-ganglia-thalamic (CBGT) networks are critical for adaptive decision-making, yet how changes to circuit-level properties impact cognitive algorithms remains unclear. Here we explore how dopaminergic plasticity at corticostriatal synapses alters competition between striatal pathways, impacting the evidence accumulation process during decision-making. Spike-timing dependent plasticity simulations showed that dopaminergic feedback based on rewards modified the ratio of direct and indirect corticostriatal weights within opposing action channels. Using the learned weight ratios in a full spiking CBGT network model, we simulated neural dynamics and decision outcomes in a reward-driven decision task and fit them with a drift diffusion model. Fits revealed that the rate of evidence accumulation varied with inter-channel differences in direct pathway activity while boundary height varied with overall indirect pathway activity. This multi-level modeling approach demonstrates how complementary learning and decision computations can emerge from corticostriatal plasticity.Author summaryCognitive process models such as reinforcement learning (RL) and the drift diffusion model (DDM) have helped to elucidate the basic algorithms underlying error-corrective learning and the evaluation of accumulating decision evidence leading up to a choice. While these relatively abstract models help to guide experimental and theoretical probes into associated phenomena, they remain uninformative about the actual physical mechanics by which learning and decision algorithms are carried out in a neurobiological substrate during adaptive choice behavior. Here we present an “upwards mapping” approach to bridging neural and cognitive models of value-based decision-making, showing how dopaminergic feedback alters the network-level dynamics of cortico-basal-ganglia-thalamic (CBGT) pathways during learning to bias behavioral choice towards more rewarding actions. By mapping “up” the levels of analysis, this approach yields specific predictions about aspects of neuronal activity that map to the quantities appearing in the cognitive decision-making framework.

scholarly journals A flexible framework for simulating and fitting generalized drift-diffusion models

2020 ◽  
Author(s):  
Maxwell Shinn ◽  
Norman H. Lam ◽  
John D. Murray
Keyword(s):  
Decision Making ◽  
Diffusion Model ◽  
Cognitive Neuroscience ◽  
Software Package ◽  
Planck Equation ◽  
Experimental Designs ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Flexible Framework ◽  
Decision Making Model

AbstractThe drift-diffusion model (DDM) is an important decision-making model in cognitive neuroscience. However, innovations in model form have been limited by methodological challenges. Here, we introduce the generalized drift-diffusion model (GDDM) framework for building, simulating, and fitting DDM extensions, and provide a software package which implements the framework. The GDDM framework augments traditional DDM parameters through arbitrary user-defined functions. Models are simulated numerically by directly solving the Fokker-Planck equation using efficient numerical methods, yielding a 100-fold or greater speedup over standard methodology. This speed allows GDDMs to be fit to data using maximum likelihood on the full response time (RT) distribution. We show that a GDDM fit with our framework explains a classic open dataset with better accuracy and fewer parameters than several DDMs implemented using the latest methodology. Overall, our framework will allow for decision-making model innovation and novel experimental designs.

scholarly journals Evidence or Confidence: What Really Accumulates During a Decision?

2021 ◽  
Author(s):  
Douglas G. Lee ◽  
Giovanni Pezzulo
Keyword(s):  
Decision Making ◽  
Empirical Evidence ◽  
Alternative Hypothesis ◽  
Decision Threshold ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Evidence Accumulation ◽  
Confidence Assessment ◽  
Standard Models ◽  
Post Hoc

Assessing one's confidence in one's choices is of paramount importance to making adaptive decisions, and it is thus no surprise that humans excel in this ability. However, standard models of decision-making, such as the drift-diffusion model (DDM), treat confidence assessment as a post-hoc or parallel process that does not directly influence the choice -- the latter depends only on accumulated evidence. Here, we pursue the alternative hypothesis that what is accumulated during a decision is confidence (that the to-be selected option is the best) rather than raw evidence. Accumulating confidence has the appealing consequence that the decision threshold corresponds to a desired level of confidence for the choice, and that confidence improvements can be traded off against the resources required to secure them. We show that most previous findings on perceptual and value-based decisions traditionally interpreted from an evidence-accumulation perspective can be explained more parsimoniously from our novel confidence-driven perspective. Furthermore, we show that our novel confidence-driven DDM (cDDM) naturally generalizes to any number of decisions -- which is notoriously extemporaneous using traditional DDM or related models. Finally, we discuss future empirical evidence that could be useful in adjudicating between these alternatives.

scholarly journals The drift diffusion model as the choice rule in inter-temporal and risky choice: a case study in medial orbitofrontal cortex lesion patients and controls

2019 ◽  
Author(s):  
Jan Peters ◽  
Mark D’Esposito
Keyword(s):  
Decision Making ◽  
Diffusion Model ◽  
Choice Model ◽  
Sequential Sampling ◽  
Temporal Discounting ◽  
Risky Choice ◽  
Choice Rule ◽  
Model Parameters ◽  
Drift Diffusion Model ◽  
Drift Diffusion

AbstractSequential sampling models such as the drift diffusion model have a long tradition in research on perceptual decision-making, but mounting evidence suggests that these models can account for response time distributions that arise during reinforcement learning and value-based decision-making. Building on this previous work, we implemented the drift diffusion model as the choice rule in inter-temporal choice (temporal discounting) and risky choice (probability discounting) using a hierarchical Bayesian estimation scheme. We validated our approach in data from nine patients with focal lesions to the ventromedial prefrontal cortex / medial orbitofrontal cortex (vmPFC/mOFC) and nineteen age- and education-matched controls. Choice model parameters estimated via standard softmax action selection were reliably reproduced using the drift diffusion model as the choice rule, both for temporal discounting and risky choice. Model comparison revealed that, for both tasks, the data were best accounted for by a variant of the drift diffusion model including a non-linear mapping from value-differences to trial-wise drift rates. Posterior predictive checks of the winning models revealed a reasonably good fit to individual participants reaction time distributions. We then applied this modeling framework and 1) reproduced our previous results regarding temporal discounting in vmPFC/mOFC patients and 2) showed in a previously unpublished data set on risky choice that vmPFC/mOFC patients exhibit increased risk-taking relative to controls. Analyses of diffusion model parameters revealed that vmPFC/mOFC damage abolished neither value sensitivity nor asymptote of the drift rate. Rather, it substantially increased non-decision times and reduced response caution during risky choice. Our results highlight that novel insights can be gained from applying sequential sampling models in studies of inter-temporal and risky decision-making in cognitive neuroscience.

scholarly journals Optimizing sequential decisions in the drift-diffusion model

2018 ◽  
Author(s):  
Khanh P. Nguyen ◽  
Krešimir Josić ◽  
Zachary P. Kilpatrick
Keyword(s):  
Decision Making ◽  
Ideal Observer ◽  
Natural Environments ◽  
Drift Diffusion Model ◽  
Sequential Decisions ◽  
Drift Diffusion ◽  
Evidence Accumulation ◽  
Temporal Correlations ◽  
Finite Sequences ◽  
Decision Times

AbstractTo make decisions organisms often accumulate information across multiple timescales. However, most experimental and modeling studies of decision-making focus on sequences of independent trials. On the other hand, natural environments are characterized by long temporal correlations, and evidence used to make a present choice is often relevant to future decisions. To understand decision-making under these conditions we analyze how a model ideal observer accumulates evidence to freely make choices across a sequence of correlated trials. We use principles of probabilistic inference to show that an ideal observer incorporates information obtained on one trial as an initial bias on the next. This bias decreases the time, but not the accuracy of the next decision. Furthermore, in finite sequences of trials the rate of reward is maximized when the observer deliberates longer for early decisions, but responds more quickly towards the end of the sequence. Our model also explains experimentally observed patterns in decision times and choices, thus providing a mathematically principled foundation for evidence-accumulation models of sequential decisions.

scholarly journals A drift diffusion model account of the semantic congruity effect in a classification paradigm

2017 ◽  
Vol 3 (1) ◽  
pp. 77-96 ◽  
Author(s):  
Angelo Pirrone ◽  
James A. R. Marshall ◽  
Tom Stafford
Keyword(s):  
Diffusion Model ◽  
Time Course ◽  
Target Stimulus ◽  
Congruity Effect ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Semantic Congruity ◽  
Evidence Accumulation ◽  
Model Account ◽  
Semantic Congruity Effect

The semantic congruity effect refers to the facilitation of judgements (i) when the direction of the comparison of two items coincides with the relative position of the items along the dimension comparison or (ii) when the relative size of a standard and a target stimulus coincides. For example, people are faster in judging 'which is bigger?' for two large items, than judging 'which is smaller?' for two large items (selection paradigm). Also, people are faster in judging a target stimulus as smaller when compared to a small standard, than when compared to a large standard, and vice versa (classification paradigm). We use the Drift Diffusion Model (DDM) to explain the time course of a semantic congruity effect in a classification paradigm. Formal modelling of semantic congruity allows the time course of the decision process to be described, using an established model of decision making. Moreover, although there have been attempts to explain the semantic congruity effect within evidence accumulation models, two possible accounts for the congruity effect have been proposed but their specific predictions have not been compared directly, using a model that could quantitatively account for both; a shift in the starting point of evidence accumulation or a change in the rate at which evidence is accumulated. With our computational investigation we provide evidence for the latter, while controlling for other possible explanations such as a variation in non-decision time or boundary separation, that have not been taken into account in the explanation of this phenomenon.

scholarly journals Motor response vigour and fixations reflect subjective preferences during intertemporal choice

2021 ◽  
Author(s):  
Elke Smith ◽  
Jan Peters
Keyword(s):  
Decision Making ◽  
Diffusion Model ◽  
Motor Response ◽  
Intertemporal Choice ◽  
Magnitude Effect ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Value Differences ◽  
Subjective Value ◽  
Subjective Utility

Value-based decision-making is of central interest in cognitive neuroscience and psychology, as well as in the context of neuropsychiatric disorders characterised by decision-making impairments. Studies examining (neuro-)computational mechanisms underlying choice behaviour typically focus on participants' decisions. However, there is increasing evidence that option valuation might also be reflected in motor response vigour and eye movements, implicit measures of subjective utility. To examine motor response vigour and visual fixation correlates of option valuation in intertemporal choice, we set up a task where the participants selected an option by pressing a grip force transducer, simultaneously tracking fixation shifts between options. As outlined in our preregistration (https://osf.io/k6jct), we used hierarchical Bayesian parameter estimation to model the choices assuming hyperbolic discounting, compared variants of the softmax and drift diffusion model, and assessed the relationship between response vigour and the estimated model parameters. The behavioural data were best explained by a drift diffusion model specifying a non-linear scaling of the drift rate by the subjective value differences. Replicating previous findings (Green et al., 1997; Wagner et al., 2020a), we found a magnitude effect for temporal discounting, such that higher rewards were discounted less. This magnitude effect was further reflected in response vigour, such that stronger forces were exerted in the high vs. the low magnitude condition. Bayesian hierarchical linear regression further revealed higher grip forces, faster response times and a lower number of fixation shifts for trials with higher subjective value differences. Our data suggest that subjective utility or implicit valuation is reflected in response vigour during intertemporal choice. Taking into account response vigour might thus provide deeper insight into decision-making, reward valuation and maladaptive changes in these processes, e.g. in the context of neuropsychiatric disorders.

scholarly journals Drift Diffusion Model of Animacy Categorization Task Can Detect Patients with Mild Cognitive Impairment and Mild Alzheimer's Disease

2021 ◽  
Author(s):  
Hamed Karimi ◽  
Haniye Marefat ◽  
Mahdiye Khanbagi ◽  
Alireza Karami ◽  
Zahra Vahabi
Keyword(s):  
Alzheimer’S Disease ◽  
Decision Making ◽  
Cognitive Impairment ◽  
Mild Cognitive Impairment ◽  
Diffusion Model ◽  
Computational Approach ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Categorization Task ◽  
Early Stages

Purpose: The process of neurodegeneration in Alzheimer's Disease (AD) is irreversible using current therapeutics. An earlier diagnosis of the disease can lead to earlier interventions, which will help patients sustain their cognitive abilities for longer. Individuals within the early stages of AD, shown to have trouble making confident and sounds decisions. Here we proposed a computational approach to quantify the decision-making ability in patients with mild cognitive impairment and mild AD. Materials and Methods: To study the quantified decision-making abilities at the early stages of the disease, we took advantage of a 2-Alternative Forced-Choice (2AFC) task. We applied the Drift Diffusion Model to determine whether the information accumulation process in a categorization task is altered in patients with mild cognitive impairment and mild AD. We implemented a classification model to detect cognitive impairment based on the Drift Diffusion Model's estimated parameters. Results: The results show a significant correlation of the classification score with the standard pen-and-paper tests, suggesting that the quantified decision-making parameters are undergoing significant change in patients with cognitive impairment. Conclusion: We confirmed that the decision-making ability deteriorates at the early stages of AD. We introduced a computational approach for measuring the decline in decision-making and used that measurement to distinguish patients from healthy individuals.

scholarly journals Decision-Making in the Human-Machine Interface

2021 ◽  
Vol 12 ◽  
Author(s):  
J. Benjamin Falandays ◽  
Samuel Spevack ◽  
Philip Pärnamets ◽  
Michael Spivey
Keyword(s):  
Decision Making ◽  
Diffusion Model ◽  
Human Machine Interface ◽  
Assistive Robotics ◽  
Clear Understanding ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Directional Bias ◽  
Model Simulations ◽  
Machine Interface

If our choices make us who we are, then what does that mean when these choices are made in the human-machine interface? Developing a clear understanding of how human decision making is influenced by automated systems in the environment is critical because, as human-machine interfaces and assistive robotics become even more ubiquitous in everyday life, many daily decisions will be an emergent result of the interactions between the human and the machine – not stemming solely from the human. For example, choices can be influenced by the relative locations and motor costs of the response options, as well as by the timing of the response prompts. In drift diffusion model simulations of response-prompt timing manipulations, we find that it is only relatively equibiased choices that will be successfully influenced by this kind of perturbation. However, with drift diffusion model simulations of motor cost manipulations, we find that even relatively biased choices can still show some influence of the perturbation. We report the results of a two-alternative forced-choice experiment with a computer mouse modified to have a subtle velocity bias in a pre-determined direction for each trial, inducing an increased motor cost to move the cursor away from the pre-designated target direction. With queries that have each been normed in advance to be equibiased in people’s preferences, the participant will often begin their mouse movement before their cognitive choice has been finalized, and the directional bias in the mouse velocity exerts a small but significant influence on their final choice. With queries that are not equibiased, a similar influence is observed. By exploring the synergies that are developed between humans and machines and tracking their temporal dynamics, this work aims to provide insight into our evolving decisions.

Sign in / Sign up

Export Citation Format

Share Document