scholarly journals The drift diffusion model as the choice rule in inter-temporal and risky choice: a case study in medial orbitofrontal cortex lesion patients and controls

2019 ◽  
Author(s):  
Jan Peters ◽  
Mark D’Esposito
Keyword(s):  
Decision Making ◽  
Diffusion Model ◽  
Choice Model ◽  
Sequential Sampling ◽  
Temporal Discounting ◽  
Risky Choice ◽  
Choice Rule ◽  
Model Parameters ◽  
Drift Diffusion Model ◽  
Drift Diffusion

AbstractSequential sampling models such as the drift diffusion model have a long tradition in research on perceptual decision-making, but mounting evidence suggests that these models can account for response time distributions that arise during reinforcement learning and value-based decision-making. Building on this previous work, we implemented the drift diffusion model as the choice rule in inter-temporal choice (temporal discounting) and risky choice (probability discounting) using a hierarchical Bayesian estimation scheme. We validated our approach in data from nine patients with focal lesions to the ventromedial prefrontal cortex / medial orbitofrontal cortex (vmPFC/mOFC) and nineteen age- and education-matched controls. Choice model parameters estimated via standard softmax action selection were reliably reproduced using the drift diffusion model as the choice rule, both for temporal discounting and risky choice. Model comparison revealed that, for both tasks, the data were best accounted for by a variant of the drift diffusion model including a non-linear mapping from value-differences to trial-wise drift rates. Posterior predictive checks of the winning models revealed a reasonably good fit to individual participants reaction time distributions. We then applied this modeling framework and 1) reproduced our previous results regarding temporal discounting in vmPFC/mOFC patients and 2) showed in a previously unpublished data set on risky choice that vmPFC/mOFC patients exhibit increased risk-taking relative to controls. Analyses of diffusion model parameters revealed that vmPFC/mOFC damage abolished neither value sensitivity nor asymptote of the drift rate. Rather, it substantially increased non-decision times and reduced response caution during risky choice. Our results highlight that novel insights can be gained from applying sequential sampling models in studies of inter-temporal and risky decision-making in cognitive neuroscience.

scholarly journals Dopamine and risky decision-making in pathological and problem gamblers

2019 ◽  
Author(s):  
Jan Peters ◽  
Taylor Vega ◽  
Dawn Weinstein ◽  
Jennifer Mitchell ◽  
Andrew Kayser
Keyword(s):  
Decision Making ◽  
Diffusion Model ◽  
Gambling Disorder ◽  
Risky Choice ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Problem Gamblers ◽  
Cohen’S D ◽  
Cohen's D

AbstractGambling disorder is a behavioral addiction that is associated with impairments in value-based decision-making such as increased temporal discounting and reduced risk-aversion. Dopamine regulates learning and decision-making by modulating information processing throughout fronto-striatal circuits. Although the role of alterations in dopamine neurotransmission in the etiology of gambling disorder is controversial, preliminary evidence suggests that specifically increasing frontal dopamine levels might improve cognitive functioning in pathological and problem gamblers. We therefore examined whether increasing frontal dopamine levels via the catechol-O-methyltransferase (COMT) inhibitor tolcapone would reduce risky choice in a group of pathological and problem gamblers (n=14) in a repeated-measures counter-balanced placebo-controlled double-blind study. Choice data were fit using hierarchical Bayesian parameter estimation and a modeling scheme that combined a risky choice model with the drift diffusion model to account for both choices and response time distributions. Model comparison revealed that the data were best accounted for by a variant of the drift diffusion model with a non-linear modulation of trial-wise drift rates by value differences, confirming recent findings. Contrary to our hypothesis, risk-taking was slightly increased under tolcapone vs. placebo (Cohen’s d = −.281). Examination of drug effects on diffusion model parameters revealed an increase in the value-dependency of the drift rate (Cohen’s d = .932) with a simultaneous reduction in the maximum drift rate (Cohen’s d = −1.84). These results add to previous work on the potential role of COMT inhibitors in behavioral addictions, and show no consistent beneficial effect of tolcapone on risky choice in gambling disorder. Modeling results add to mounting evidence for the applicability of diffusion models in value-based decision-making. Future work should focus on individual genetic, clinical and cognitive factors that might account for the heterogeneity in the effects of COMT inhibition.

scholarly journals A flexible framework for simulating and fitting generalized drift-diffusion models

2020 ◽  
Author(s):  
Maxwell Shinn ◽  
Norman H. Lam ◽  
John D. Murray
Keyword(s):  
Decision Making ◽  
Diffusion Model ◽  
Cognitive Neuroscience ◽  
Software Package ◽  
Planck Equation ◽  
Experimental Designs ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Flexible Framework ◽  
Decision Making Model

AbstractThe drift-diffusion model (DDM) is an important decision-making model in cognitive neuroscience. However, innovations in model form have been limited by methodological challenges. Here, we introduce the generalized drift-diffusion model (GDDM) framework for building, simulating, and fitting DDM extensions, and provide a software package which implements the framework. The GDDM framework augments traditional DDM parameters through arbitrary user-defined functions. Models are simulated numerically by directly solving the Fokker-Planck equation using efficient numerical methods, yielding a 100-fold or greater speedup over standard methodology. This speed allows GDDMs to be fit to data using maximum likelihood on the full response time (RT) distribution. We show that a GDDM fit with our framework explains a classic open dataset with better accuracy and fewer parameters than several DDMs implemented using the latest methodology. Overall, our framework will allow for decision-making model innovation and novel experimental designs.

scholarly journals Reward-driven changes in striatal pathway competition shape evidence evaluation in decision-making

2018 ◽  
Author(s):  
Kyle Dunovan ◽  
Catalina Vich ◽  
Matthew Clapp ◽  
Timothy Verstynen ◽  
Jonathan Rubin
Keyword(s):  
Decision Making ◽  
Basal Ganglia ◽  
Diffusion Model ◽  
Process Models ◽  
Decision Task ◽  
Drift Diffusion Model ◽  
Indirect Pathway ◽  
Drift Diffusion ◽  
Pathway Activity ◽  
Evidence Accumulation

AbstractCortico-basal-ganglia-thalamic (CBGT) networks are critical for adaptive decision-making, yet how changes to circuit-level properties impact cognitive algorithms remains unclear. Here we explore how dopaminergic plasticity at corticostriatal synapses alters competition between striatal pathways, impacting the evidence accumulation process during decision-making. Spike-timing dependent plasticity simulations showed that dopaminergic feedback based on rewards modified the ratio of direct and indirect corticostriatal weights within opposing action channels. Using the learned weight ratios in a full spiking CBGT network model, we simulated neural dynamics and decision outcomes in a reward-driven decision task and fit them with a drift diffusion model. Fits revealed that the rate of evidence accumulation varied with inter-channel differences in direct pathway activity while boundary height varied with overall indirect pathway activity. This multi-level modeling approach demonstrates how complementary learning and decision computations can emerge from corticostriatal plasticity.Author summaryCognitive process models such as reinforcement learning (RL) and the drift diffusion model (DDM) have helped to elucidate the basic algorithms underlying error-corrective learning and the evaluation of accumulating decision evidence leading up to a choice. While these relatively abstract models help to guide experimental and theoretical probes into associated phenomena, they remain uninformative about the actual physical mechanics by which learning and decision algorithms are carried out in a neurobiological substrate during adaptive choice behavior. Here we present an “upwards mapping” approach to bridging neural and cognitive models of value-based decision-making, showing how dopaminergic feedback alters the network-level dynamics of cortico-basal-ganglia-thalamic (CBGT) pathways during learning to bias behavioral choice towards more rewarding actions. By mapping “up” the levels of analysis, this approach yields specific predictions about aspects of neuronal activity that map to the quantities appearing in the cognitive decision-making framework.

Identifying Feigned Cognitive Impairment: Investigating the Utility of Diffusion Model Analyses

Assessment ◽  
2020 ◽  
pp. 107319112096231
Author(s):  
Elad Omer ◽  
Tomer Elbaum ◽  
Yoram Braw
Keyword(s):  
Cognitive Impairment ◽  
Diffusion Model ◽  
Cognitive Processes ◽  
Forced Choice ◽  
Model Parameters ◽  
Performance Validity ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Choice Performance ◽  
Feigned Cognitive Impairment

Forced-choice performance validity tests are routinely used for the detection of feigned cognitive impairment. The drift diffusion model deconstructs performance into distinct cognitive processes using accuracy and response time measures. It thereby offers a unique approach for gaining insight into examinees’ speed-accuracy trade-offs and the cognitive processes that underlie their performance. The current study is the first to perform such analyses using a well-established forced-choice performance validity test. To achieve this aim, archival data of healthy participants, either simulating cognitive impairment in the Word Memory Test or performing it to the best of their ability, were analyzed using the EZ-diffusion model ( N = 198). The groups differed in the three model parameters, with drift rate emerging as the best predictor of group membership. These findings provide initial evidence for the usefulness of the drift diffusion model in clarifying the cognitive processes underlying feigned cognitive impairment and encourage further research.

scholarly journals Motor response vigour and fixations reflect subjective preferences during intertemporal choice

2021 ◽  
Author(s):  
Elke Smith ◽  
Jan Peters
Keyword(s):  
Decision Making ◽  
Diffusion Model ◽  
Motor Response ◽  
Intertemporal Choice ◽  
Magnitude Effect ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Value Differences ◽  
Subjective Value ◽  
Subjective Utility

Value-based decision-making is of central interest in cognitive neuroscience and psychology, as well as in the context of neuropsychiatric disorders characterised by decision-making impairments. Studies examining (neuro-)computational mechanisms underlying choice behaviour typically focus on participants' decisions. However, there is increasing evidence that option valuation might also be reflected in motor response vigour and eye movements, implicit measures of subjective utility. To examine motor response vigour and visual fixation correlates of option valuation in intertemporal choice, we set up a task where the participants selected an option by pressing a grip force transducer, simultaneously tracking fixation shifts between options. As outlined in our preregistration (https://osf.io/k6jct), we used hierarchical Bayesian parameter estimation to model the choices assuming hyperbolic discounting, compared variants of the softmax and drift diffusion model, and assessed the relationship between response vigour and the estimated model parameters. The behavioural data were best explained by a drift diffusion model specifying a non-linear scaling of the drift rate by the subjective value differences. Replicating previous findings (Green et al., 1997; Wagner et al., 2020a), we found a magnitude effect for temporal discounting, such that higher rewards were discounted less. This magnitude effect was further reflected in response vigour, such that stronger forces were exerted in the high vs. the low magnitude condition. Bayesian hierarchical linear regression further revealed higher grip forces, faster response times and a lower number of fixation shifts for trials with higher subjective value differences. Our data suggest that subjective utility or implicit valuation is reflected in response vigour during intertemporal choice. Taking into account response vigour might thus provide deeper insight into decision-making, reward valuation and maladaptive changes in these processes, e.g. in the context of neuropsychiatric disorders.

scholarly journals Simultaneous Hierarchical Bayesian Parameter Estimation for Reinforcement Learning and Drift Diffusion Models: a Tutorial and Links to Neural Data

2020 ◽  
Vol 3 (4) ◽  
pp. 458-471 ◽  
Author(s):  
Mads L. Pedersen ◽  
Michael J. Frank
Keyword(s):  
Parameter Estimation ◽  
Reinforcement Learning ◽  
Diffusion Model ◽  
Simultaneous Estimation ◽  
Model Parameters ◽  
Hierarchical Bayesian ◽  
Drift Diffusion Model ◽  
Drift Diffusion ◽  
Bayesian Parameter Estimation ◽  
The Impact

AbstractCognitive models have been instrumental for generating insights into the brain processes underlying learning and decision making. In reinforcement learning it has recently been shown that not only choice proportions but also their latency distributions can be well captured when the choice function is replaced with a sequential sampling model such as the drift diffusion model. Hierarchical Bayesian parameter estimation further enhances the identifiability of distinct learning and choice parameters. One caveat is that these models can be time-consuming to build, sample from, and validate, especially when models include links between neural activations and model parameters. Here we describe a novel extension to the widely used hierarchical drift diffusion model (HDDM) toolbox, which facilitates flexible construction, estimation, and evaluation of the reinforcement learning drift diffusion model (RLDDM) using hierarchical Bayesian methods. We describe the types of experiments most applicable to the model and provide a tutorial to illustrate how to perform quantitative data analysis and model evaluation. Parameter recovery confirmed that the method can reliably estimate parameters with varying numbers of synthetic subjects and trials. We also show that the simultaneous estimation of learning and choice parameters can improve the sensitivity to detect brain–behavioral relationships, including the impact of learned values and fronto-basal ganglia activity patterns on dynamic decision parameters.

Sign in / Sign up

Export Citation Format

Share Document