Pramipexole Enhances Reward Learning by Preserving Value Estimates

Background: Dopamine D2-like receptor agonists show promise as treatments for depression. They are thought to act by altering how individuals learn from rewarding experiences. However, the nature of these reward learning alterations, and the mechanisms by which they are produced is not clear. Reinforcement learning accounts describe three distinct processes that may produce similar changes in reward learning behaviour; increased reward sensitivity, increased inverse decision temperature and decreased value decay. As these processes produce equivalent effects on behaviour, arbitrating between them requires measurement of how expectations and prediction errors are altered. In the present study, we characterised the behavioural effects of a sustained 2-week course of the D2/3/4 receptor agonist pramipexole on reward learning and used fMRI measures of expectation and prediction error to assess which of these three mechanistic processes were responsible for the behavioural effects. Methods: 40 healthy volunteers (Age: 18-43, 50% female) were randomly allocated to receive either two weeks of pramipexole (titrated to 1mg/day) or placebo in a double-blind, between subject design. Participants completed a probabilistic instrumental learning task, in which stimuli were associated with either rewards or losses, before the pharmacological intervention and twice between days 12-15 of the intervention (once with and once without fMRI). Both asymptotic choice accuracy, and a reinforcement learning model, were used to assess reward learning. Results: Behaviourally, pramipexole specifically increased choice accuracy in the reward condition, with no effect in the loss condition. Pramipexole increased the BOLD response in the orbital frontal cortex during the expectation of win trials but decreased the BOLD response to reward prediction errors in the ventromedial prefrontal cortex. This pattern of results indicates that pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning. Conclusions: The D2-like receptor agonist pramipexole enhances reward learning by preserving learned values. This is a plausible candidate mechanism for pramipexoles observed anti-depressant effect.

Post-error slowing reflects the joint impact of adaptive and maladaptive processes during decision making

Errors and their consequences are typically studied by investigating changes in decision speed and accuracy in trials that follow an error, commonly referred to as "post-error adjustments". Many studies have reported that subjects slow down following an error, a phenomenon called "post-error slowing" (PES). However, the functional significance of PES is still a matter of debate as it is not always adaptive. That is, it is not always associated with a gain in performance and can even occur with a decline in accuracy. Here, we hypothesized that the nature of PES is influenced by one's speed-accuracy tradeoff policy, which determines the overall level of choice accuracy in the task at hand. To test this hypothesis, we investigated post-error adjustments in subjects performing the same task while they were required to either emphasize speed (low accuracy) or cautiousness (high accuracy) in two distinct contexts (hasty and cautious contexts, respectively) experienced on separate days. Accordingly, our data indicate that post-error adjustments varied according to the context in which subjects performed the task, with PES being solely significant in the hasty context. In addition, we only observed a gain in performance after errors in a specific trial type, suggesting that post-error adjustments depend on a complex combination of processes that affect the speed of ensuing actions as well as the degree to which such PES comes with a gain in performance.

Neuronal origins of biases in economic choices under sequential offers

Economic choices are characterized by a variety of biases. Understanding their origins is a long-term goal for neuroeconomics, but progress on this front has been limited. Here we examined choice biases observed when two goods are offered sequentially. In the experiments, rhesus monkeys chose between different juices offered simultaneously or in sequence. Choices under sequential offers were less accurate (higher variability). They were also biased in favor of the second offer (order bias) and in favor of the preferred juice (preference bias). Analysis of neuronal activity recorded in orbitofrontal cortex revealed that these phenomena emerged at different computational stages. The lower choice accuracy reflected weaker offer value signals (valuation stage), the order bias emerged during value comparison (decision stage), and the preference bias emerged late in the trial (post-comparison). Our approach, leveraging recent notions on the neural mechanisms of economic decisions, may shed light on other aspects of choice behavior.

Approach direction and accuracy, but not response times, show spatial-numerical association in chicks

Chicks trained to identify a target item in a sagittally-oriented series of identical items show a higher accuracy for the target on the left, rather than that on the right, at test when the series was rotated by 90°. Such bias seems to be due to a right hemispheric dominance in visuospatial tasks. Up to now, the bias was highlighted by looking at accuracy, the measure mostly used in non-human studies to detect spatial numerical association, SNA. In the present study, processing by each hemisphere was assessed by scoring three variables: accuracy, response times and direction of approach. Domestic chicks were tested under monocular vision conditions, as in the avian brain input to each eye is mostly processed by the contralateral hemisphere. Four-day-old chicks learnt to peck at the 4th element in a sagittal series of 10 identical elements. At test, when facing a series oriented fronto-parallel, birds confined their responses to the visible hemifield, with high accuracy for the 4th element. The first element in the series was also highly selected, suggesting an anchoring strategy to start the proto-counting at one end of the series. In the left monocular condition, chicks approached the series starting from the left, and in the right monocular condition, they started from the right. Both hemispheres appear to exploit the same strategy, scanning the series from the most lateral element in the clear hemifield. Remarkably, there was no effect in the response times: equal latency was scored for correct or incorrect and for left vs. right responses. Overall, these data indicate that the measures implying a direction of choice, accuracy and direction of approach, and not velocity, i.e., response times, can highlight SNA in this paradigm. We discuss the relevance of the selected measures to unveil SNA.

Optogenetic suppression of the medial septum impairs working memory maintenance

Spatial working memory (SWM) is the ability to encode, maintain, and retrieve spatial information over a temporal gap, and relies on a network of structures including the medial septum (MS), which provides critical input to the hippocampus. Although the role of the MS in SWM is well-established, up until recently, we have been unable to use temporally precise circuit manipulation techniques to examine the specific role of the MS in SWM, particularly to distinguish between encoding, maintenance, and retrieval. Here, we test the hypothesis that the MS supports the maintenance of spatial information over a temporal gap using precisely timed optogenetic suppression delivered during specific portions of three different tasks, two of which rely on SWM and one that does not. In experiment 1, we found that MS optogenetic suppression impaired choice accuracy of a SWM dependent conditional discrimination task. Moreover, this deficit was only observed when MS suppression was delivered during the cue-sampling, but not the cue-retrieval, portion of the trial. There was also no deficit when MS neurons were optogenetically suppressed as rats performed a SWM-independent variant of the task. In experiment 2, we tested whether MS suppression affected choice accuracy on a delayed nonmatch to position (DNMP) task when suppression was limited to the sample, delay, and choice phases of the task. We found that MS suppression delivery during the delay phase of the DNMP task, but not during the sample or choice phases, impaired choice accuracy. Our results collectively suggest that the MS plays an important role in SWM by maintaining task-relevant information over a temporal delay.

Computations of confidence are modulated by mentalizing ability

Do people have privileged and direct access to their own minds, or do we infer our own thoughts and feelings indirectly, as we would infer the mental states of others? In this study we shed light on this question by examining how mentalizing ability—the set of processes involved in understanding other people’s thoughts and feelings—relates to metacognitive efficiency—the ability to reflect on one’s own performance. In a general population sample (N = 477) we showed that mentalizing ability and self-reported socio-communicative skills are positively correlated with perceptual metacognitive efficiency, even after controlling for choice accuracy. By modelling the trial-by-trial formation of confidence we showed that mentalizing ability predicted the association between response times and confidence, suggesting those with better mentalizing ability were more sensitive to inferential cues to self-performance. In a second study we showed that both mentalizing and metacognitive efficiency were lower in autistic participants (N = 40) when compared with age, gender, IQ, and education-matched non-autistic participants. Together, our results suggest that the ability to understand other people’s minds predicts self-directed metacognition.

Control over sampling boosts numerical evidence processing in human decisions from experience

When acquiring information about choice alternatives, decision makers may have varying levels of control over which and how much information they sample before making a choice. How does control over sampling affect the quality of experience-based decisions? Here, combining variants of a numerical sampling task with neural recordings, we show that control over when to stop sampling can enhance (i) behavioral choice accuracy, (ii) the build-up of parietal decision signals, and (iii) the encoding of numerical sample information in multivariate electroencephalogram (EEG) patterns. None of these effects were observed when participants could only control which alternatives to sample, but not when to stop sampling. Furthermore, levels of control had no effect on early sensory signals or on the extent to which sample information leaked from memory. The results indicate that freedom to stop sampling can amplify decisional evidence processing from the outset of information acquisition and lead to more accurate choices.

Global visual confidence

Visual confidence is the observers’ estimate of their precision in one single perceptual decision. Ultimately, however, observers often need to judge their confidence over a task in general rather than merely on one single decision. Here, we measured the global confidence acquired across multiple perceptual decisions. Participants performed a dual task on two series of oriented stimuli. The perceptual task was an orientation-discrimination judgment. The metacognitive task was a global confidence judgment: observers chose the series for which they felt they had performed better in the perceptual task. We found that choice accuracy in global confidence judgments improved as the number of items in the series increased, regardless of whether the global confidence judgment was made before (prospective) or after (retrospective) the perceptual decisions. This result is evidence that global confidence judgment was based on an integration of confidence information across multiple perceptual decisions rather than on a single one. Furthermore, we found a tendency for global confidence choices to be influenced by response times, and more so for recent perceptual decisions than earlier ones in the series of stimuli. Using model comparison, we found that global confidence is well described as a combination of noisy estimates of sensory evidence and position-weighted response-time evidence. In summary, humans can integrate information across multiple decisions to estimate global confidence, but this integration is not optimal, in particular because of biases in the use of response-time information.

The cognition/metacognition tradeoff

Integration-to-boundary is an optimal decision algorithm that takes samples of evidence until the posterior reaches a decision boundary, resulting in the fastest decisions for a target accuracy. For example, integration-to-boundary achieves faster mean-RT compared with taking a fixed number of samples that result in the same choice-accuracy. Here we show that this advantage comes at a cost in metacognitive accuracy. We show that integration-to-boundary results in less variability in evidence-integration, and is less predictive of choice accuracy. We test this in two experiments, in which all participants carried out two sessions that manipulated the response-mode protocol: free-response (evidence terminated by the subject response) vs interrogation (fixed number of evidence samples, which is the same as in the free-response session). In both sessions the participants observe a sequence of evidence frames (2/sec) and they first enter a choice and then a confidence response. As predicted, the latter protocol enhances metacognitive accuracy.

Global visual confidence

Visual confidence is the observers’ estimate of their precision in one single perceptual decision. Ultimately, however, observers often need to judge their confidence over a task in general rather than merely on one single decision. Here, we measured the global confidence acquired across multiple perceptual decisions. Participants performed a dual task on two series of oriented stimuli. The perceptual task was an orientation-discrimination judgment. The metacognitive task was a global confidence judgment: observers chose the series for which they felt they had performed better in the perceptual task. We found that choice accuracy in global confidence judgments improved as the number of items in the series increased, regardless of whether the global confidence judgment was made before (prospective) or after (retrospective) the perceptual decisions. This result is evidence that global confidence judgment was based on an integration of confidence information across multiple perceptual decisions rather than on a single one. Furthermore, we found a tendency for global confidence choices to be influenced by response times, and more so for recent perceptual decisions than earlier ones in the series of stimuli. Using model comparison, we found that global confidence is well described as a combination of noisy estimates of sensory evidence and position-weighted response-time evidence. In summary, humans can integrate information across multiple decisions to estimate global confidence, but this integration is not optimal, in particular because of biases in the use of response-time information.