Rational inattention and tonic dopamine

Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA—the average reward theory and the Bayesian theory in which DA controls precision—have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of ‘rational inattention,’ which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock—thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.

Download Full-text

Rational Inattention and Tonic Dopamine

10.1101/2020.10.04.325175 ◽

2020 ◽

Author(s):

John G. Mikhael ◽

Lucy Lai ◽

Samuel J. Gershman

Keyword(s):

Reinforcement Learning ◽

Intertemporal Choice ◽

Interval Timing ◽

Internal Clock ◽

Bayesian Theory ◽

Average Reward ◽

Rational Inattention ◽

The Subject ◽

Temporal Stimuli ◽

Experimental Findings

AbstractSlow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA—the average reward theory and the Bayesian theory in which DA controls precision—have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of ‘rational inattention,’ which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock—thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.Author SummaryThe roles of tonic dopamine (DA) have been the subject of much speculation, partly due to the variety of processes it has been implicated in. For instance, tonic DA modulates how we learn new information, but also affects how previously learned information is used. DA affects the speed of our internal timing mechanism, but also modulates the degree to which our temporal estimates are influenced by context. DA improves performance in some tasks, but seems only to affect confidence in others. Are there common principles that govern the role of DA across these domains? In this work, we introduce the concept of ‘rational inattention,’ originally borrowed from economics, to the DA literature. We show how the rational inattention account of DA unites two influential theories that are seemingly at odds: the average reward theory and the Bayesian theory of tonic DA. We then show how this framework reconciles the diverse roles of DA, which cannot be addressed by either theory alone.

Download Full-text

Time perception and patience: individual differences in interval timing precision predict choice impulsivity in European starlings, Sturnus vulgaris

Animal Cognition ◽

10.1007/s10071-020-01456-2 ◽

2021 ◽

Author(s):

Clare Andrews ◽

Jonathon Dunn ◽

Daniel Nettle ◽

Melissa Bateson

Keyword(s):

Individual Differences ◽

Numerical Model ◽

Intertemporal Choice ◽

Interval Timing ◽

Sturnus Vulgaris ◽

Cognitive Mechanisms ◽

Time Intervals ◽

European Starlings ◽

Temporal Reproduction ◽

Limited Variation

AbstractImpulsivity, in the sense of the extent rewards are devalued as the time until their realization increases, is linked to various negative outcomes in humans, yet understanding of the cognitive mechanisms underlying it is limited. Variation in the imprecision of interval timing is a possible contributor to variation in impulsivity. We use a numerical model to generate predictions concerning the effect of timing imprecision on impulsivity. We distinguish between fixed imprecision (the imprecision that applies even when timing the very shortest time intervals) and proportional imprecision (the rate at which imprecision increases as the interval becomes longer). The model predicts that impulsivity should increase with increasing fixed imprecision, but decrease with increasing proportional imprecision. We present data from a cohort of European starlings (Sturnus vulgaris, n = 28) in which impulsivity had previously been measured through an intertemporal choice paradigm. We tested interval timing imprecision in the same individuals using a tri-peak temporal reproduction procedure. We found repeatable individual differences in both fixed and proportional imprecision. As predicted, birds with greater proportional imprecision in interval timing made fewer impulsive choices, whilst those with greater fixed imprecision tended to make more. Contradictory observations in the literature regarding the direction of association between timing imprecision and impulsivity might be clarified by distinguishing between fixed and proportional components of imprecision.

Download Full-text

Quantifying the contribution of individual variation in timing to delay-discounting

Scientific Reports ◽

10.1038/s41598-021-97496-w ◽

2021 ◽

Vol 11 (1) ◽

Cited By ~ 1

Author(s):

Evgeniya Lukinova ◽

Jeffrey C. Erlich

Keyword(s):

Individual Differences ◽

Time Perception ◽

Delay Discounting ◽

Intertemporal Choice ◽

Internal Clock ◽

Self Control ◽

Important Relationship ◽

Psychology And Economics ◽

Sense Of Time ◽

Choice Tasks

AbstractDelay-discounting studies in neuroscience, psychology, and economics have been mostly focused on concepts of self-control, reward evaluation, and discounting. Another important relationship to consider is the link between intertemporal choice and time perception. We presented 50 college students with timing tasks on the range of seconds to minutes and intertemporal-choice tasks on both the time-scale of seconds and of days. We hypothesized that individual differences in time perception would influence decisions about short experienced delays but not long delays. While we found some evidence that individual differences in internal clock speed account for some unexplained variance between choices across time-horizons, overall our findings suggest a nominal contribution of the altered sense of time in intertemporal choice.

Download Full-text

Association of the altered sense of time in the seconds-to-minute range with intertemporal choices across time-horizons.

10.31234/osf.io/ut5ms ◽

2020 ◽

Author(s):

Evgeniya Lukinova ◽

Jeffrey C Erlich

Keyword(s):

Individual Differences ◽

Time Perception ◽

Intertemporal Choice ◽

Internal Clock ◽

Self Control ◽

Time Horizons ◽

Important Relationship ◽

Psychology And Economics ◽

Sense Of Time ◽

Choice Tasks

Delay-discounting studies in neuroscience, psychology, and economics have been mostly focused on concepts of self-control, reward evaluation, and discounting. Another important relationship to consider is the link between intertemporal choice and time perception. We presented 50 college students with timing exercises on the range of seconds to minutes and intertemporal-choice tasks on both the time-scale of seconds and of days. We hypothesized that individual differences in time perception would influence decisions about short experienced delays but not long delays. While we found some evidence that individual differences in internal clock speed account for some unexplained variance between choices across time-horizons, overall our findings suggest a nominal contribution of the altered sense of time in intertemporal choice.

Download Full-text

Temporal Discounting Across Adulthood: A Systematic Review and Meta-analysis

10.31234/osf.io/7ysxa ◽

2020 ◽

Author(s):

Kendra Leigh Seaman ◽

Sade J Abiodun ◽

Zöe Fenn ◽

Gregory Russell Samanez-Larkin ◽

Rui Mata

Keyword(s):

Age Differences ◽

Intertemporal Choice ◽

Meta Analysis ◽

Empirical Work ◽

Adult Life ◽

Temporal Discounting ◽

Total N ◽

Adult Age ◽

Adult Age Differences ◽

Choice Tasks

A number of developmental theories have been proposed that make differential predictions about the links between age and temporal discounting; that is, the valuation of rewards at different points in time. Most empirical studies examining adult age differences in temporal discounting have relied on economic intertemporal choice tasks, which pit choosing a smaller, sooner monetary reward against choosing a larger, later one. Although initial studies using these tasks suggested older adults discount less than younger adults, follow-up studies provided heterogeneous, and thus inconclusive, results. Using an open science approach, we test the replicability of adult age differences in temporal discounting by conducting a preregistered systematic literature search and meta-analysis of adult age differences in intertemporal choice tasks. Across 37 cross-sectional studies (Total N = 104,736), we found no reliable relation between age and temporal discounting (r = -0.081, 95% CI [-0.185, 0.025]). We also found little evidence of publication bias or p-hacking. Exploratory analyses of moderators found no effect of experimental design (e.g., extreme-group vs. continuous age), incentives (hypothetical vs. rewards), amount of delay (e.g., days, weeks, months, or years), or quantification of discounting behavior (e.g., proportion of immediate choices vs. parameters from computational modeling). Additional analyses of 12 participant-level data sets found little support for a nonlinear relation between age and temporal discounting across adulthood. Overall, the results suggest that adult age is not reliably associated with individual differences in temporal discounting. We provide recommendations for future empirical work on temporal discounting across the adult life span.

Download Full-text