Single-trial modeling separates multiple overlapping prediction errors during reward processing in human EEG

Learning signals during reinforcement learning and cognitive control rely on valenced reward prediction errors (RPEs) and non-valenced salience prediction errors (PEs) driven by surprise magnitude. A core debate in reward learning focuses on whether valenced and non-valenced PEs can be isolated in the human electroencephalogram (EEG). We combine behavioral modeling and single-trial EEG regression to disentangle sequential PEs in an interval timing task dissociating outcome valence, magnitude, and probability. Multiple regression across temporal, spatial, and frequency dimensions characterized a spatio-tempo-spectral cascade from early valenced RPE value to non-valenced RPE magnitude, followed by outcome probability indexed by a late frontal positivity. Separating negative and positive outcomes revealed the valenced RPE value effect is an artifact of overlap between two non-valenced RPE magnitude responses: frontal theta feedback-related negativity on losses and posterior delta reward positivity on wins. These results reconcile longstanding debates on the sequence of components representing reward and salience PEs in the human EEG.

The Negative Contingency Illusion: A Cognitive Bias Leading to Misjudgement of Protection

Humans possess a highly adaptive ability to draw inferences about the world by recognizing meaningful links between stimuli and events: making contingency judgements. We describe a systematic bias in contingency judgements that we label the negative contingency illusion in which individuals falsely judge a cue to be protective against an outcome. We demonstrate that the illusion arises when outcome probability is low and occurs when there is no actual relationship between cue and outcome and even when there is a modest positive relationship between cue and outcome. Such misjudgements may lead individuals to superstitious beliefs and could have major public health implications if they lead to the belief in and promotion of treatments that are ineffective or deleterious to the prevention and treatment of illness.

A time-varying diffusion model analysis of value-based decision-making: effects of adult age and probability information saliency

Empirical evidence has shown that visually enhancing the saliency of reward probability information can ease the cognitive demands of value comparisons and improve value-based decision-making in old age. In the present study, we used a time-varying DDM that includes starting time parameters (henceforth starting time DDM, stDDM) to better understand how increasing the saliency of probability may affect the dynamics of value-based decision-making. We enhanced the saliency of reward probability by using a color-coding scheme as a decision-aid in a mixed lottery choice task, with which the decisions of younger and older adults were assessed. Older adults’ evidence accumulation processes were less sensitive to information about outcome probability and magnitude than those of younger adults. The decision-aid enhanced the weighting of probability and magnitude information in both age groups, as well as the starting time advantage for probability information relative to magnitude information. Older adults who had a lower baseline value sensitivity, as reflected in the parameters from non-aid trials, benefited more from increasing information saliency in improving decisions. Furthermore, in older adults, this aid-induced effect was related to individual’s spontaneous eye-blink rate, a potential proxy of dopamine functioning. Taken together, analyzing the behavioral data using the stDDM revealed new evidence for adult age differences during value-based decisions: not do only older adults weigh the outcome probability and magnitude less than younger adults, they also do not process information about probability sooner than magnitude. Visually enhancing the saliency of probability information can benefit older decision makers in shifting their decision dynamics to be more similar to younger adults.

Temporal Fluctuation of Mood in Gaming Task Modulates Feedback Negativity: EEG Study With Virtual Reality

Feedback outcomes are generally classified into positive and negative feedback. People often predict a feedback outcome with information that is based on both objective facts and uncertain subjective information, such as a mood. For example, if an action leads to good results consecutively, people performing the action overestimate the behavioral result of the next action. In electroencephalogram measurements, negative feedback evokes negative potential, called feedback negativity, and positive feedback evokes positive potential, called reward positivity. The present study investigated the relationship between the degree of the mood caused by the feedback outcome and the error-related brain potentials. We measured the electroencephalogram activity while the participants played a virtual reality shooting game. The experimental task was to shoot down a cannonball flying toward the player using a handgun. The task difficulty was determined from the size and curve of the flying cannonball. These gaming parameters affected the outcome probability of shooting the target in the game. We also implemented configurations in the game, such as the player’s life points and play times. These configurations affected the outcome magnitude of shooting the target in the game. Moreover, we used the temporal accuracy of shooting in the game as the parameter of the mood. We investigated the relationship between these experimental features and the event-related potentials using the single-trial-based linear mixed-effects model analysis. The feedback negativity was observed at an error trial, and its amplitude was modulated with the outcome probability and the mood. Conversely, reward positivity was observed at hit trials, but its amplitude was modulated with the outcome probability and outcome magnitude. This result suggests that feedback negativity is enhanced according to not only the feedback probability but also the mood that was changed depending on the temporal gaming outcome.

The effect of uncertainty and outcome probability on non-instrumental information seeking

People’s desire to seek or avoid information is not only influenced by the possible outcomes of an event, but the probability of those particular outcomes occurring. There are competing explanations however as to how and why people’s desire for non-instrumental information is affected by factors including expected value, probability of outcome, and a unique formulation of outcome uncertainty. Over two experiments, we find that people’s preference for non-instrumental information is positively correlated with probability when the outcome is positive (i.e., winning money) and negatively correlated when the outcome is negative (i.e., losing money). Furthermore, at the aggregate level, we find the probability of an outcome to be a better predictor of information preference than the expected value of the event or its outcome uncertainty.

Rational After All: Changes in Probability Matching Behaviour Across Time in Humans and Monkeys

Probability matching—where subjects given probabilistic in-put respond in a way that is proportional to those input probabilities—has long been thought to be characteristic of primate performance in probability learning tasks in a variety of contexts, from decision making to the learning of linguistic variation in humans. However, such behaviour is puzzling because it is not optimal in a decision theoretic sense; the optimal strategy is to always select the alternative with the highest positive-outcome probability, known as maximising(in decision making) or regularising (in linguistic tasks). While the tendency to probability match seems to depend somewhat on the participants and the task (i.e., infants are less likely to probability match than adults, monkeys probability matchless than humans, and probability matching is less likely in linguistic tasks), existing studies suffer from a range of deficiencies which make it difficult to robustly assess these differences. In this project we present a series of experiments which systematically test the development of probability matching behaviour over time in simple decision making tasks, across species (humans and Guinea baboons), task complexity, and task domain (linguistic vs non-linguistic).

Emotion in animal contests

Emotions encompass cognitive and behavioural responses to reward and punishment. Using contests as a case-study, we propose that short-term emotions underpin animals' assessments, decision-making and behaviour. Equating contest assessments to emotional ‘appraisals', we describe how contestants appraise more than resource value and outcome probability. These appraisals elicit the cognition, drive and neurophysiology that governs aggressive behaviour. We discuss how recent contest outcomes induce long-term moods, which impact subsequent contest behaviour. Finally, we distinguish between integral (objectively relevant) and incidental (objectively irrelevant) emotions and moods (affective states). Unlike existing ecological models, our approach predicts that incidental events influence contest dynamics, and that contests become incidental influences themselves, potentially causing maladaptive decision-making. As affective states cross contexts, a more holistic ethology (incorporating emotions and moods) would illuminate animal cognition and behaviour.

Multiple sequential prediction errors during reward processing in the human brain

SUMMARYRecent developments in reinforcement learning, cognitive control, and systems neuroscience highlight the complimentary roles in learning of valenced reward prediction errors (RPEs) and non-valenced salience prediction errors (PEs) driven by the magnitude of surprise. A core debate in reward learning focuses on whether valenced and non-valenced PEs can be isolated in the human electroencephalogram (EEG). Here, we combine behavioral modeling and single-trial EEG regression revealing a sequence of valenced and non-valenced PEs in an interval timing task dissociating outcome valence, magnitude, and probability. Multiple regression across temporal, spatial, and frequency dimensions revealed a spatio-tempo-spectral cascade from valenced RPE value represented by the feedback related negativity event-related potential (ERP) followed by non-valenced RPE magnitude and outcome probability effects indexed by subsequent P300 and late frontal positivity ERPs. The results show that learning is supported by a sequence of multiple PEs evident in the human EEG.