Risk-Promoting Effects of Reward-Paired Cues in Human Sign- and Goal-Trackers

The incentive sensitization theory of addiction proposes that through repeated associations with addictive rewards, addiction-related stimuli acquire a disproportionately powerful motivational pull on behaviour. Animal research suggests trait-like individual variation in the degree of incentive salience attribution to reward-predictive cues, defined phenotypically as sign-tracking (high) and goal-tracking (low incentive salience attribution). While these phenotypes have been linked to addiction features in rodents, their translational validity has been little studied. Here, we examined whether sign- and goal-tracking in healthy human volunteers modulates the effects of reward-paired cues on cost-benefit decision making. Sign-tracking was measured in a Pavlovian conditioning paradigm as the amount of eye gaze fixation on the reward-predictive cue versus the location of impending reward delivery. In Study 1 (Cherkasova et al, 2018), participants were randomly assigned to perform a two-choice lottery task in which rewards were either accompanied (cued, n=63) or unaccompanied (uncued, n=68) by money images and casino jingles. In Study 2, participants (n=58) performed cued and uncued versions of the task in a within-subjects design. Across both studies, cues promoted riskier choice, and both studies yielded evidence of goal-tracking being associated with greater risk-promoting effects of cues. These findings are at odds with the notion of sign-trackers being preferentially susceptible to the influence of reward cues on behavior and point to the role of mechanisms besides incentive salience in mediating such influences.

Directed motor actions and choice signalling drive cortical acetylcholine dynamics.

Numerous cognitive functions including attention, learning, and plasticity are influenced by the dynamic patterns of acetylcholine release across the brain. How acetylcholine mediates these functions in cortex remains unclear, as the spatiotemporal relationship between cortical acetylcholine and behavioral events has not been precisely measured across task learning. To dissect this relationship, we quantified motor behavior and sub-second acetylcholine dynamics in primary somatosensory cortex during acquisition and performance of a tactile-guided object localization task. We found that acetylcholine dynamics were spatially homogenous and directly attributable to whisker motion and licking, rather than sensory cues or reward delivery. As task performance improved across training, acetylcholine release to the first lick in a trial became dramatically and specifically potentiated, paralleling the emergence of a choice-signalling basis for this motor action. These results show that acetylcholine dynamics in sensory cortex are driven by directed motor actions to gather information and act upon it.

Managing the economic challenges in the treatment of heart failure

Background Treatment of heart failure is complex and inherently challenging. Patients traverse multiple practice settings as inpatients and outpatients, often resulting in fragmented care. The Center for Medicare and Medicaid Services is implementing payment programs that reward delivery of high-quality, cost-effective care, and one of the newer programs, the Bundled Payment for Care Improvement Advanced program, attempts to improve the coordination of care across practices for a hospitalization episode and post-acute care. The quality and cost of care contribute to its value, but value may be defined in different ways by different entities. Conclusions The rapidly changing world of digital health may contribute to or detract from the quality and cost of care. Health systems, payers, and patients are all grappling with these issues, which were reviewed at a symposium at the Heart Failure Society of America conference in Philadelphia, Pennsylvania on September 14, 2019. This article constitutes the proceedings from that symposium.

A low-cost open-source 5-choice operant box system optimized for electrophysiology and optophysiology in mice

Operant boxes enable the application of complex behavioural paradigms to support circuit neuroscience and drug discovery research. However, commercial operant box systems are expensive and often not optimised for combining behaviour with neurophysiology. Here we introduce a fully open-source Python-based operant-box system in a 5-choice design (pyOS-5) that enables assessment of multiple cognitive and affective functions. It is optimized for fast turn-over between animals, and for testing of tethered mice for simultaneous physiological recordings or optogenetic manipulation. For reward delivery, we developed peristaltic and syringe pumps based on a stepper motor and 3D-printed parts. Tasks are specified using a Python-based syntax implemented on custom-designed printed circuit boards that are commercially available at low cost. We developed an open-source graphical user interface (GUI) and task definition scripts to conduct assays assessing operant learning, attention, impulsivity, working memory, or cognitive flexibility, alleviating the need for programming skills of the end user. All behavioural events are recorded with millisecond resolution, and TTL-outputs and -inputs allow straightforward integration with physiological recordings and closed-loop manipulations. This combination of features realizes a cost-effective, nose-poke-based operant box system that allows reliable circuit-neuroscience experiments investigating correlates of cognition and emotion in large cohorts of subjects.

Dopamine mediates the bidirectional update of interval timing

The role of dopamine as a reward prediction error signal in reinforcement learning tasks has been well-established over the past decades. Recent work has shown that the reward prediction error interpretation can also account for the effects of dopamine on interval timing by controlling the speed of subjective time. According to this theory, the timing of the dopamine signal relative to reward delivery dictates whether subjective time speeds up or slows down: Early DA signals speed up subjective time and late signals slow it down. To test this bidirectional prediction, we reanalyzed measurements of dopaminergic neurons in the substantia nigra pars compacta of mice performing a self-timed movement task. Using the slope of ramping dopamine activity as a read-out of subjective time speed, we found that trial-by-trial changes in the slope could be predicted from the timing of dopamine activity on the previous trial. This result provides a key piece of evidence supporting a unified computational theory of reinforcement learning and interval timing.

Reward timing matters in motor learning

Reward can improve motor learning and the consolidation of motor memories. Identifying the features of reward feedback that are critical for motor learning is a necessary step for successful integration into rehabilitation programs. One central feature of reward feedback that may affect motor learning is its timing, that is, the delay after which reward is delivered following movement execution. In fact, research on associative learning has shown that short and long reward delays (e.g., 1 and 6 s following action execution) activate preferentially the striatum and the hippocampus, respectively, which both contribute with varying degrees to motor learning. Given the distinct functional role of these two areas, we hypothesized that reward timing could modulate how people learn and consolidate a new motor skill. In sixty healthy participants, we found that delaying reward delivery by a few seconds influenced motor learning dynamics. Indeed, training with a short reward delay (i.e., 1 s) induced slow, yet continuous gains in performance, while a long reward delay (i.e., 6 s) led to initially high learning rates that were followed by an early plateau in the learning curve and a lower endpoint performance. Moreover, participants who successfully learned the skill with a short reward delay displayed overnight consolidation, while those who trained with a long reward delay exhibited an impairment in the consolidation of the motor memory. Overall, our data show that reward timing affects motor learning, potentially by modulating the engagement of different learning processes, a finding that could be exploited in future rehabilitation programs.

Astrocytes in the external globus pallidus coordinate flexibility of action strategy

The external globus pallidus (GPe) is an integrative hub and gateway for behavioral flexibility in reward-related behaviors. However, it remains unknown whether enriched astrocytes in the GPe guide behavioral flexibility. Here, we trained mice to exhibit goal-directed and habitual reward-seeking behaviors using the behavior tasks with effort- and time-based reward delivery, respectively. Then, we examined the temporal dynamics of GPe astrocytes during goal-directed and habitual learning. Overall, GPe astrocytes were substantially silenced during habitual learning compared to goal-directed learning. In the timescale of action events, GPe astrocyte activities were increased immediately after termination of reward-taking behavior before the following action. However, during habitual learning, the increase of astrocyte activity was not evident. Moreover, support vector machine (SVM) analysis demonstrated that GPe astrocytes dynamics predicted whether mice perform goal-directed or habitual behaviors. Interestingly, chemogenetic activation of GPe astrocytes, which dampened GPe neuronal firings and habitual behaviors, exhibting goal-directed behaviors. Strikingly, brief and repeated attentional stimulations recapitulated the effect of chemogenetic activation of GPe in intervening the habitual reward-seeking behaviors with increased GPe astrocyte activities. Our findings reveal a novel insight that increasing GPe astrocytic activities attenuates habitual behavior and improves behavioral flexibility, which may provide a potential therapeutic target for decision-making-related disorders, such as obsessive-compulsive disorder and addiction.

Nicotine Enhances Goal-Tracking in Ethanol and Food Pavlovian Conditioned Approach Paradigms

RationaleNicotine promotes alcohol intake through pharmacological and behavioral interactions. As an example of the latter, nicotine can facilitate approach toward food- and alcohol-associated stimuli (“sign-tracking”) in lever-Pavlovian conditioned approach (PavCA) paradigms. However, we recently reported that nicotine can also enhance approach toward locations of reward delivery (“goal-tracking”) triggered by ethanol-predictive stimuli when the location of ethanol delivery is non-static (i.e., a retractable sipper bottle).ObjectiveTo determine whether the non-static nature of the reward location could have biased the development of goal-tracking in our previous study (Loney et al., 2019); we assessed the effect of nicotine in a lever-PavCA paradigm wherein the location of ethanol delivery was static (i.e., a stationary liquid receptacle). Then, to determine whether nicotine’s enhancement of goal-tracking is unique to ethanol-predictive stimuli, we assessed the effect of systemic nicotine on approach triggered by food-predictive stimuli in a lever-PavCA paradigm.MethodsLong–Evans rats were used in two PavCA experiments wherein a lever predicted the receipt of ethanol (15% vol/vol; experiment 1) or food (experiment 2) into a stationary receptacle. Prior to testing, rats were administered nicotine (0.4 mg/kg subcutaneously) or saline systemically.ResultsIn both experiments, nicotine increased measures of goal-tracking, but not sign-tracking.ConclusionNicotine can facilitate approach to reward locations without facilitating approach to reward-predictive stimuli. As such, conceptualization of the mechanisms by which nicotine affects behavior must be expanded to explain an enhancement of goal-tracking by nicotine.