Impulsivity as Bayesian inference under dopaminergic control

Bayesian models successfully account for several of dopamine (DA)’s effects on contextual calibration in interval timing and reward estimation. In these models, DA controls the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice tasks, in which agents must choose between small rewards delivered soon and large rewards delivered later. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases ‘impulsivity.’ However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect—promoting selection of the smaller/sooner option (more impulsivity). Second, in both cases, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later option. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts a correlation between behavioral phenotypes: Animals that are better timers will also appear less impulsive.Significance StatementDoes dopamine make animals more or less impulsive? Though impulsivity features prominently in several dopamine-related conditions, how dopamine actually influences impulsivity has remained unclear. In intertemporal choice tasks (ITCs), wherein animals must choose between small rewards delivered soon and large rewards delivered later, administering dopamine makes animals more willing to wait for larger/later rewards in some conditions (consistent with lower impulsivity), but less willing in others. We hypothesize that dopamine does not necessarily influence impulsivity at all, but rather gates the influence of contextual information during decision making. We show that this account explains an array of curious findings in ITCs, including the seemingly conflicting results above. Our work encourages a reexamination of ITCs as a method for assessing impulsivity.