Computational mechanisms of distributed value representations and mixed learning strategies

Learning appropriate representations of the reward environment is challenging in the real world where there are many options, each with multiple attributes or features. Despite existence of alternative solutions for this challenge, neural mechanisms underlying emergence and adoption of value representations and learning strategies remain unknown. To address this, we measure learning and choice during a multi-dimensional probabilistic learning task in humans and trained recurrent neural networks (RNNs) to capture our experimental observations. We find that human participants estimate stimulus-outcome associations by learning and combining estimates of reward probabilities associated with the informative feature followed by those of informative conjunctions. Through analyzing representations, connectivity, and lesioning of the RNNs, we demonstrate this mixed learning strategy relies on a distributed neural code and opponency between excitatory and inhibitory neurons through value-dependent disinhibition. Together, our results suggest computational and neural mechanisms underlying emergence of complex learning strategies in naturalistic settings.

Pupil Dilation and Response Slowing Distinguish Deliberate Explorative Choices in the Probabilistic Learning Task

This study examined whether pupil size and response time would distinguish directed exploration from random exploration and exploitation. Eighty-nine participants performed the two-choice probabilistic learning task while their pupil size and response time were continuously recorded. Using LMM analysis, we estimated differences in the pupil size and response time between the advantageous and disadvantageous choices as a function of learning success, i.e., whether or not a participant has learned the probabilistic contingency between choices and their outcomes. We proposed that before a true value of each choice became known to a decision-maker, both advantageous and disadvantageous choices represented a random exploration of the two options with an equally uncertain outcome, whereas the same choices after learning manifested exploitation and direct exploration strategies, respectively. We found that disadvantageous choices were associated with increases both in response time and pupil size, but only after the participants had learned the choice-reward contingencies. For the pupil size, this effect was strongly amplified for those disadvantageous choices that immediately followed gains as compared to losses in the preceding choice. Pupil size modulations were evident during the behavioral choice rather than during the pretrial baseline. These findings suggest that occasional disadvantageous choices, which violate the acquired internal utility model, represent directed exploration. This exploratory strategy shifts choice priorities in favor of information seeking and its autonomic and behavioral concomitants are mainly driven by the conflict between the behavioral plan of the intended exploratory choice and its strong alternative, which has already proven to be more rewarding.

Stereotypes Disrupt Probabilistic Category Learning

We ask whether task-irrelevant associations of a social nature, such as stereotypes, may be “sticky” and disrupt probabilistic learning and updating more than non-social task-irrelevant associations. Across three experiments, participants learned the probabilistic outcomes of different combinations of cards based on feedback in either a social (i.e. forecasting crime) or nonsocial (i.e. forecasting weather) learning context. During learning, participants were presented with either task-irrelevant social (i.e. Black or White faces) or non-social (i.e. Darker or Lighter clouds) stimuli that were congruent or incongruent with the social and nonsocial learning context. We found that learning was impaired in the social compared to nonsocial learning context, despite the fact that both sets of stimuli were unrelated to the stimulus/outcome pairings that participants were asked to learn (Studies 1-2). Moreover, we found that learning decrements extended beyond the mere presence of distracting human faces (Study 2), and that they extended to both positively and negatively-valenced stereotypes (Study 3). Next, we tested competing hypotheses regarding whether learning decrements were due to “first order” stereotype application or inhibition at the trial-level, or due to “second order” cognitive load disruptions that accumulate across trials (Meta-Analysis 1). We found evidence for second- versus first-order disruptions, such that participants who were more internally motivated to respond without prejudice learned less accurately. This finding suggests that participants who are more likely to self-monitor their responses in contexts where they may appear prejudiced perform significantly worse due to taxed cognitive functioning. We discuss the implications for the influence of stereotypes on probabilistic category learning.

Amphetamine alters the Reward Positivity in humans and mice

The development of pro-cognitive therapeutics for psychiatric disorders has been beset with difficulties. This is in part due to the absence of pharmacologically-sensitive cognitive biomarkers common to humans and rodents. Here, we describe a cross-species translational measure of reward processing that is sensitive to the dopamine agonist, d-amphetamine. Motivated by human electroencephalographic (EEG) findings, we recently reported that frontal midline delta-band power is also an electrophysiological biomarker of reward surprise in mice. Here, we determined the impact on this reward-related EEG response from humans (n=23) and mice (n=28) performing a probabilistic learning task under parametric doses of d-amphetamine (human: placebo, 10 mg, 20 mg; mice: placebo, 0.1 mg/kg, 0.3 mg.kg, 1.0 mg/kg). In humans, d-amphetamine boosted the Reward Positivity event-related potential (ERP) component as well as the spectral delta-band representation of this signal. In mice, only the Reward Positivity ERP component was significantly boosted by d-amphetamine. In sum, the present results confirm the role of dopamine in the generation of the Reward Positivity, and support the first pharmacologically valid biomarker of reward sensitivity across species.