Reversing anterior insular cortex neuronal hypoexcitability attenuates compulsive behavior in juvenile rats

Development of self-regulatory competencies during adolescence is partially dependent on normative brain maturation. Here we report that juvenile rats as compared to adults exhibit impulsive and compulsive-like behavioral traits, the latter being associated with lower expression of mRNA levels of the immediate early gene zif268 in the anterior insula (AI). This observation suggests that deficits in AI function in juvenile rats could explain their immature pattern of interoceptive cue integration in rational decision-making and compulsive phenotype. In support of this, here we report hypoexcitability of juvenile layer-V pyramidal neurons in the AI, concomitant with reduced glutamatergic synaptic input to these cells. Chemogenetic activation of the AI attenuated the compulsive trait suggesting that delayed maturation of the AI results in suboptimal integration of sensory and cognitive information in adolescents and this contributes to inflexible behaviors in specific conditions of reward availability.

Choice history effects in mice and humans improve reward harvesting efficiency

Choice history effects describe how future choices depend on the history of past choices. In experimental tasks this is typically framed as a bias because it often diminishes the experienced reward rates. However, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of earning a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences on reward availability over time. To circumvent this, we use a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By examining the behavior of optimal agents in the VI task we discover that choice history effects observed in animals serve to maximize reward harvesting efficiency. We further distil the function of choice history effects by manipulating first- and second-order statistics of the environment. We find that choice history effects primarily reflect the growth rate of the reward probability of the unchosen option, whereas reward history effects primarily reflect environmental volatility. Based on observed choice history effects in animals, we develop a reinforcement learning model that explicitly incorporates choice history over multiple time scales into the decision process, and we assess its predictive adequacy in accounting for the associated behavior. We show that this new variant, known as the double trace model, has a higher performance in predicting choice data, and shows near optimal reward harvesting efficiency in simulated environments. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias.

The dissociable effects of reward on sequential motor behaviour

Reward has consistently been shown to enhance motor performance however its beneficial effects appear to be largely unspecific. While reward has been shown to invigorate performance, it also enhances learning and/or retention. Therefore, a mechanistic account of the effects of reward on motor behaviour is lacking. Here we tested the hypothesis that these distinct reward-based improvements are driven by dissociable reward types: explicit reward (i.e. money) and performance feedback (i.e. points). Experiment 1 showed that explicit reward instantaneously improved movement times (MT) using a novel sequential reaching task. In contrast, performance-based feedback led to learning-related improvements. Importantly, pairing both maximised MT performance gains and accelerated movement fusion. Fusion describes an optimisation process during which neighbouring sequential movements blend together to form singular actions. Results from experiment 2 served as a replication and showed that fusion led to enhanced performance speed whilst also improving movement efficiency through increased smoothness. Finally, experiment 3 showed that these improvements in performance persist for 24 hours even without reward availability. This highlights the dissociable impact of explicit reward and performance feedback, with their combination maximising performance gains and leading to stable improvements in the speed and efficiency of sequential actions.

Specialized coding patterns among dorsomedial prefrontal neuronal ensembles predict conditioned reward seeking

Non-overlapping cell populations within dorsomedial prefrontal cortex (dmPFC), defined by gene expression or projection target, control dissociable aspects of reward seeking through unique activity patterns. However, even within these defined cell populations considerable cell-to-cell variability is found, suggesting that greater resolution is needed to understand information processing in dmPFC. Here we use two-photon calcium imaging in awake, behaving mice to monitor the activity of dmPFC excitatory neurons throughout Pavlovian reward conditioning. We characterize five unique neuronal ensembles that each encode specialized information related to a sucrose reward, reward-predictive cues, and behavioral responses to those cues. The ensembles differentially emerge across daily training sessions - and stabilize after learning - in a manner that improves the predictive validity of dmPFC activity dynamics for deciphering variables related to behavioral conditioning. Our results characterize the complex dmPFC neuronal ensemble dynamics that stably predict reward availability and initiation of conditioned reward seeking following cue-reward learning.

Unveiling the neural correlates of habit in the dorsal striatum

We have recently reported sustained inhibition in the dorsomedial striatum (DMS) and sustained excitation in the dorsolateral striatum (DLS) during execution of a lever press sequence in a discrete-trials task promoting habit. This sustained dorsostriatal activity was present early on, and did not clearly change in step with improved performance over ten training sessions. Early onset of sequence-related neural activity could have resulted from rapid habitual learning promoted by presentation of lever cues, predicting reward availability and delivery. To test this hypothesis, we compared DLS and DMS spiking activity in the discrete trials habit-promoting task with two task variants that promote goal-directed behavior. Comparison of the three tasks revealed that mean neuronal spiking activity was generally sustained across the lever press sequence in the task promoting habit and characterized by overall excitation in DLS and inhibition in DMS relative to baseline. In contrast, mean activity differences in DLS and DMS were much less prominent, and most changes occurred transiently around individual lever presses, in the tasks promoting goal-directed behavior. These results indicate that sequence delineation cues, such as the lever cues in these studies, promote habitual behavior and that this habitual behavior is encoded in the striatum by cue-triggered sustained DLS excitation and DMS inhibition that likely reflects cue-elicited behavioral chunking.

Continuity within somatosensory cortical map shapes the integration of optogenetic input

The topographic organization of sensory cortices is a prominent feature, but its functional role remains unclear. Particularly, how activity is integrated within a cortical area depending on its topography is unknown. Here, we trained mice expressing channelrhodopsin in cortical excitatory neurons to track a bar photostimulation that rotated smoothly over the primary somatosensory cortex (S1). When photostimulation was aimed at vS1, the area which contains a contiguous representation of the whisker array at the periphery, mice could learn to discriminate angular positions of the bar to obtain a reward. In contrast, they could not learn the task when the photostimulation was aimed at the representation of the trunk and legs in S1, where neighboring zones represent distant peripheral body parts, introducing discontinuities. Mice demonstrated anticipation of reward availability, specifically when cortical topography enabled to predict future sensory activation. These results are particularly helpful for designing efficient cortical sensory neuroprostheses.

Specialized coding patterns among dorsomedial prefrontal neuronal ensembles during conditioned reward seeking.

Non-overlapping cell populations within dorsomedial prefrontal cortex (dmPFC), defined by gene expression or projection target, control dissociable aspects of reward seeking through unique activity patterns. However, even within these defined cell populations considerable cell-to-cell variability is found, suggesting that greater resolution is needed to understand information processing in dmPFC. Here we use two-photon calcium imaging in awake, behaving mice to monitor the activity of dmPFC excitatory neurons throughout Pavlovian sucrose conditioning. We characterize five unique neuronal ensembles that each encode specialized information related to a reward, reward-predictive cues, and behavioral responses to reward-predictive cues. The ensembles differentially emerge across learning, and stabilize after learning, in a manner that improves the predictive validity of dmPFC activity dynamics for deciphering variables related to behavioral conditioning. Our results characterize the complex dmPFC neuronal ensemble dynamics that relay learning-dependent signals for prediction of reward availability and initiation of conditioned reward seeking.

A normative account of choice history effects in mice and humans

Choice history effects describe how future choices depend on the history of past choices. Choice history effects are typically framed as a bias rather than an adaptive phenomenon because the phenomenon generally degrades reward rates in experimental tasks. How-ever, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of obtaining a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences in regard to reward availability over time. To circumvent this, we used a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By manipulating first- and second-order statistics of the environment, we dissociated choice history, reward history, and reaction times. We found that choice history effects reflect the growth rate of the reward probability of the unchosen option, reward history effects reflect environmental volatility, and reaction time reflects overall reward rate. By testing in mice and humans, we show that the same choice history effects can be generalized across species and that these effects are similar to those observed in optimal agents. Furthermore, we develop a new reinforcement learning model that explicitly incorporates choice history over multiple timescales into the decision process, and we examine its predictive adequacy in accounting for the associated behavioral data. We show that this new variant, known as the double trace model, has a higher predictive adequacy of choice data, in addition to better reward harvesting efficiency in simulated environments. Finally, we show that the choice history effects emerge in optimal models of foraging in habitats with diminishing returns, thus linking this phenomenon to a wider class of optimality models in behavioral ecology. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias.

Noisy communities and signal detection: why do foragers visit rewardless flowers?

Floral communities present complex and shifting resource landscapes for flower-foraging animals. Strong similarities among the floral displays of different plant species, paired with high variability in reward distributions across time and space, can weaken correlations between floral signals and reward status. As a result, it should be difficult for foragers to discriminate between rewarding and rewardless flowers. Building on signal detection theory in behavioural ecology, we use hypothetical probability density functions to examine graphically how plant signals pose challenges to forager decision-making. We argue that foraging costs associated with incorrect acceptance of rewardless flowers and incorrect rejection of rewarding ones interact with community-level reward availability to determine the extent to which rewardless and rewarding species should overlap in flowering time. We discuss the evolutionary consequences of these phenomena from both the forager and the plant perspectives. This article is part of the theme issue ‘Signal detection theory in recognition systems: from evolving models to experimental tests’.