Activity patterns of serotonin neurons underlying cognitive flexibility

Serotonin is implicated in mood and affective disorders. However, growing evidence suggests that a core endogenous role is to promote flexible adaptation to changes in the causal structure of the environment, through behavioral inhibition and enhanced plasticity. We used long-term photometric recordings in mice to study a population of dorsal raphe serotonin neurons, whose activity we could link to normal reversal learning using pharmacogenetics. We found that these neurons are activated by both positive and negative prediction errors, and thus report signals similar to those proposed to promote learning in conditions of uncertainty. Furthermore, by comparing the cue responses of serotonin and dopamine neurons, we found differences in learning rates that could explain the importance of serotonin in inhibiting perseverative responding. Our findings show how the activity patterns of serotonin neurons support a role in cognitive flexibility, and suggest a revised model of dopamine–serotonin opponency with potential clinical implications.

Download Full-text

Firing patterns of serotonin neurons underlying cognitive flexibility

10.1101/059758 ◽

2016 ◽

Cited By ~ 3

Author(s):

Sara Matias ◽

Eran Lottem ◽

Guillaume P. Dugué ◽

Zachary F. Mainen

Keyword(s):

Cognitive Flexibility ◽

Causal Structure ◽

Dopamine Neurons ◽

Prediction Errors ◽

Firing Patterns ◽

Learning Rates ◽

Serotonin Neurons ◽

Endogenous Role ◽

Flexible Adaptation

Serotonin is implicated in mood and affective disorders1,2 but growing evidence suggests that its core endogenous role may be to promote flexible adaptation to changes in the causal structure of the environment3–8. This stems from two functions of endogenous serotonin activation: inhibiting learned responses that are not currently adaptive9,10 and driving plasticity to reconfigure them1113. These mirror dual functions of dopamine in invigorating reward-related responses and promoting plasticity that reinforces new ones16,17. However, while dopamine neurons are known to be activated by reward prediction errors18,19, consistent with theories of reinforcement learning, the reported firing patterns of serotonin neurons21–23 do not accord with any existing theories1,24,25. Here, we used long-term photometric recordings in mice to study a genetically-defined population of dorsal raphe serotonin neurons whose activity we could link to normal reversal learning. We found that these neurons are activated by both positive and negative prediction errors, thus reporting the kind of surprise signal proposed to promote learning in conditions of uncertainty26,27. Furthermore, by comparing cue responses of serotonin and dopamine neurons we found differences in learning rates that could explain the importance of serotonin in inhibiting perseverative responding. Together, these findings show how the firing patterns of serotonin neurons support a role in cognitive flexibility and suggest a revised model of dopamine-serotonin opponency with potential clinical implications.

Download Full-text

A nonlinear relationship between prediction errors and learning rates in human reinforcement learning

10.1101/751222 ◽

2019 ◽

Author(s):

Erdem Pulcu

Keyword(s):

Reinforcement Learning ◽

Nonlinear Relationship ◽

Prediction Errors ◽

Learning Rates ◽

The Face ◽

In The Wild ◽

Actual Outcome ◽

Update Rules ◽

Different Sources

AbstractWe are living in a dynamic world in which stochastic relationships between cues and outcome events create different sources of uncertainty1 (e.g. the fact that not all grey clouds bring rain). Living in an uncertain world continuously probes learning systems in the brain, guiding agents to make better decisions. This is a type of value-based decision-making which is very important for survival in the wild and long-term evolutionary fitness. Consequently, reinforcement learning (RL) models describing cognitive/computational processes underlying learning-based adaptations have been pivotal in behavioural2,3 and neural sciences4–6, as well as machine learning7,8. This paper demonstrates the suitability of novel update rules for RL, based on a nonlinear relationship between prediction errors (i.e. difference between the agent’s expectation and the actual outcome) and learning rates (i.e. a coefficient with which agents update their beliefs about the environment), that can account for learning-based adaptations in the face of environmental uncertainty. These models illustrate how learners can flexibly adapt to dynamically changing environments.

Download Full-text

Rare Rewards Amplify Dopamine Learning Responses

10.1101/851709 ◽

2019 ◽

Author(s):

Kathryn M. Rothenhoefer ◽

Tao Hong ◽

Aydin Alikaya ◽

William R. Stauffer

Keyword(s):

Probability Distributions ◽

Reaction Times ◽

Neural Mechanism ◽

Dopamine Neurons ◽

Point Estimate ◽

Prediction Errors ◽

Average Value ◽

Learning Rates ◽

Behavioral Choices ◽

Predicted Values

AbstractDopamine neurons drive learning by coding reward prediction errors (RPEs), which are formalized as subtractions of predicted values from reward values. Subtractions accommodate point estimate predictions of value, such as the average value. However, point estimate predictions fail to capture many features of choice and learning behaviors. For instance, reaction times and learning rates consistently reflect higher moments of probability distributions. Here, we demonstrate that dopamine RPE responses code probability distributions. We presented monkeys with rewards that were drawn from the tails of normal and uniform reward size distributions to generate rare and common RPEs, respectively. Behavioral choices and pupil diameter measurements indicated that monkeys learned faster and registered greater arousal from rare RPEs, compared to common RPEs of identical magnitudes. Dopamine neuron recordings indicated that rare rewards amplified RPE responses. These results demonstrate that dopamine responses reflect probability distributions and suggest a neural mechanism for the amplified learning and enhanced arousal associated with rare events.

Download Full-text