scholarly journals Effects of affective arousal on choice behavior, reward prediction errors, and feedback-related negativities in human reward-based decision making

2015 ◽  
Vol 6 ◽  
Author(s):  
Hong-Hsiang Liu ◽  
Ming H. Hsieh ◽  
Yung-Fong Hsu ◽  
Wen-Sung Lai
Keyword(s):  
Decision Making ◽  
Choice Behavior ◽  
Prediction Errors ◽  
Affective Arousal ◽  
Reward Prediction

scholarly journals The contribution of striatal pseudo-reward prediction errors to value-based decision-making

2017 ◽  
Author(s):  
Ernest Mas-Herrero ◽  
Guillaume Sescousse ◽  
Roshan Cools ◽  
Josep Marco-Pallarés
Keyword(s):  
Decision Making ◽  
Choice Behavior ◽  
Prediction Errors ◽  
Life Outcomes ◽  
Stimulus Response ◽  
Hierarchical Reinforcement Learning ◽  
Reward Prediction ◽  
Fmri Study ◽  
Reward Contingencies ◽  
The Brain

AbstractMost studies that have investigated the brain mechanisms underlying learning have focused on the ability to learn simple stimulus-response associations. However, in everyday life, outcomes are often obtained through complex behavioral patterns involving a series of actions. In such scenarios, parallel learning systems are important to reduce the complexity of the learning problem, as proposed in the framework of hierarchical reinforcement learning (HRL). One of the key features of HRL is the computation of pseudo-reward prediction errors (PRPEs) which allow the reinforcement of actions that led to a sub-goal before the final goal itself is achieved. Here we wanted to test the hypothesis that, despite not carrying any rewarding value per se, pseudo-rewards might generate a bias in choice behavior when reward contingencies are not well-known or uncertain. Second, we also hypothesized that this bias might be related to the strength of PRPE striatal representations. In order to test these ideas, we developed a novel decision-making paradigm to assess reward prediction errors (RPEs) and PRPEs in two studies (fMRI study: n = 20; behavioural study: n = 19). Our results show that overall participants developed a preference for the most pseudo-rewarding option throughout the task, even though it did not lead to more monetary rewards. fMRI analyses revealed that this preference was predicted by individual differences in the relative striatal sensitivity to PRPEs vs RPEs. Together, our results indicate that pseudo-rewards generate learning signals in the striatum and subsequently bias choice behavior despite their lack of association with actual reward.

Emotion prediction errors guide socially adaptive behavior

2021 ◽  
Author(s):  
Joseph Heffner ◽  
Jae-Young Son ◽  
Oriel FeldmanHall
Keyword(s):  
Decision Making ◽  
Real Time ◽  
Adaptive Behavior ◽  
Emotional Response ◽  
New Method ◽  
Prediction Errors ◽  
Emotional Experiences ◽  
Past Work ◽  
Reward Prediction ◽  
Expected Outcomes

People make decisions based on deviations from expected outcomes, known as prediction errors. Past work has focused on reward prediction errors, largely ignoring violations of expected emotional experiences—emotion prediction errors. We leverage a new method to measure real-time fluctuations in emotion as people decide to punish or forgive others. Across four studies (N=1,016), we reveal that emotion and reward prediction errors have distinguishable contributions to choice, such that emotion prediction errors exert the strongest impact during decision-making. We additionally find that a choice to punish or forgive can be decoded in less than a second from an evolving emotional response, suggesting emotions swiftly influence choice. Finally, individuals reporting significant levels of depression exhibit selective impairments in using emotion—but not reward—prediction errors. Evidence for emotion prediction errors potently guiding social behaviors challenge standard decision-making models that have focused solely on reward.

scholarly journals Reward activity in ventral pallidum tracks satiety-sensitive preference and drives choice behavior

2020 ◽  
Vol 6 (45) ◽  
pp. eabc9321
Author(s):  
David J. Ottenheimer ◽  
Karen Wang ◽  
Xiao Tong ◽  
Kurt M. Fraser ◽  
Jocelyn M. Richard ◽  
...  
Keyword(s):  
Decision Making ◽  
Nervous System ◽  
Prediction Error ◽  
Choice Behavior ◽  
Physiological State ◽  
Ventral Pallidum ◽  
Optogenetic Stimulation ◽  
Reward Prediction ◽  
Behavioral Preference ◽  
Stimulation Of

A key function of the nervous system is producing adaptive behavior across changing conditions, like physiological state. Although states like thirst and hunger are known to impact decision-making, the neurobiology of this phenomenon has been studied minimally. Here, we tracked evolving preference for sucrose and water as rats proceeded from a thirsty to sated state. As rats shifted from water choices to sucrose choices across the session, the activity of a majority of neurons in the ventral pallidum, a region crucial for reward-related behaviors, closely matched the evolving behavioral preference. The timing of this signal followed the pattern of a reward prediction error, occurring at the cue or the reward depending on when reward identity was revealed. Additionally, optogenetic stimulation of ventral pallidum neurons at the time of reward was able to reverse behavioral preference. Our results suggest that ventral pallidum neurons guide reward-related decisions across changing physiological states.

scholarly journals Positive reward prediction errors during decision-making strengthen memory encoding

2019 ◽  
Vol 3 (7) ◽  
pp. 719-732 ◽  
Author(s):  
Anthony I. Jang ◽  
Matthew R. Nassar ◽  
Daniel G. Dillon ◽  
Michael J. Frank
Keyword(s):  
Decision Making ◽  
Prediction Errors ◽  
Memory Encoding ◽  
Reward Prediction

scholarly journals The contribution of striatal pseudo-reward prediction errors to value-based decision-making

NeuroImage ◽  
2019 ◽  
Vol 193 ◽  
pp. 67-74 ◽  
Author(s):  
Ernest Mas-Herrero ◽  
Guillaume Sescousse ◽  
Roshan Cools ◽  
Josep Marco-Pallarés
Keyword(s):  
Decision Making ◽  
Prediction Errors ◽  
Reward Prediction

scholarly journals Increased fronto-striatal reward prediction errors moderate decision making in obsessive–compulsive disorder

2017 ◽  
Vol 47 (7) ◽  
pp. 1246-1258 ◽  
Author(s):  
T. U. Hauser ◽  
R. Iannaccone ◽  
R. J. Dolan ◽  
J. Ball ◽  
J. Hättenschwiler ◽  
...  
Keyword(s):  
Decision Making ◽  
Obsessive Compulsive Disorder ◽  
Late Onset ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Learning Mechanisms ◽  
Obsessive Compulsive ◽  
Compulsive Disorder ◽  
Reward Prediction ◽  
Reinforcement Learning Models

BackgroundObsessive–compulsive disorder (OCD) has been linked to functional abnormalities in fronto-striatal networks as well as impairments in decision making and learning. Little is known about the neurocognitive mechanisms causing these decision-making and learning deficits in OCD, and how they relate to dysfunction in fronto-striatal networks.MethodWe investigated neural mechanisms of decision making in OCD patients, including early and late onset of disorder, in terms of reward prediction errors (RPEs) using functional magnetic resonance imaging. RPEs index a mismatch between expected and received outcomes, encoded by the dopaminergic system, and are known to drive learning and decision making in humans and animals. We used reinforcement learning models and RPE signals to infer the learning mechanisms and to compare behavioural parameters and neural RPE responses of the OCD patients with those of healthy matched controls.ResultsPatients with OCD showed significantly increased RPE responses in the anterior cingulate cortex (ACC) and the putamen compared with controls. OCD patients also had a significantly lower perseveration parameter than controls.ConclusionsEnhanced RPE signals in the ACC and putamen extend previous findings of fronto-striatal deficits in OCD. These abnormally strong RPEs suggest a hyper-responsive learning network in patients with OCD, which might explain their indecisiveness and intolerance of uncertainty.

scholarly journals A neuronal mechanism underlying decision-making deficits during hyperdopaminergic states

2017 ◽  
Author(s):  
Jeroen P.H. Verharen ◽  
Johannes W. de Jong ◽  
Theresia J.M. Roelofs ◽  
Christiaan F.M. Huffels ◽  
Ruud van Zessen ◽  
...  
Keyword(s):  
Decision Making ◽  
Substance Abuse ◽  
Prefrontal Cortex ◽  
Nucleus Accumbens ◽  
Multidisciplinary Approach ◽  
Prediction Errors ◽  
Risky Decision ◽  
Reward Prediction ◽  
Decision Making Processes ◽  
Mesencephalic Dopamine

AbstractHyperdopaminergic states in mental disorders are associated with disruptive deficits in decision-making. However, the precise contribution of topographically distinct mesencephalic dopamine pathways to decision-making processes remains elusive. Here we show, using a multidisciplinary approach, how hyperactivity of ascending projections from the ventral tegmental area (VTA) contributes to faulty decision-making in rats. Activation of the VTA-nucleus accumbens pathway leads to insensitivity to loss and punishment due to impaired processing of negative reward prediction errors. In contrast, activation of the VTA-prefrontal cortex pathway promotes risky decision-making without affecting the ability to choose the economically most beneficial option. Together, these findings show how malfunction of ascending VTA projections affects value-based decision-making, providing a mechanistic understanding of the reckless behaviors seen in substance abuse, mania, and after dopamine replacement therapy in Parkinson’s disease.

scholarly journals Pupil responses as indicators of value-based decision-making

2018 ◽  
Author(s):  
Joanne C. Van Slooten ◽  
Sara Jahfari ◽  
Tomas Knapen ◽  
Jan Theeuwes
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Computational Model ◽  
Cognitive Processes ◽  
Learning Task ◽  
Brain Regions ◽  
Computational Approach ◽  
Prediction Errors ◽  
Reward Prediction ◽  
Exciting Possibility

AbstractPupil responses have been used to track cognitive processes during decision-making. Studies have shown that in these cases the pupil reflects the joint activation of many cortical and subcortical brain regions, also those traditionally implicated in value-based learning. However, how the pupil tracks value-based decisions and reinforcement learning is unknown. We combined a reinforcement learning task with a computational model to study pupil responses during value-based decisions, and decision evaluations. We found that the pupil closely tracks reinforcement learning both across trials and participants. Prior to choice, the pupil dilated as a function of trial-by-trial fluctuations in value beliefs. After feedback, early dilation scaled with value uncertainty, whereas later constriction scaled with reward prediction errors. Our computational approach systematically implicates the pupil in value-based decisions, and the subsequent processing of violated value beliefs, ttese dissociable influences provide an exciting possibility to non-invasively study ongoing reinforcement learning in the pupil.

scholarly journals Using Primary Reinforcement to Enhance Translatability of a Human Affect and Decision-Making Judgment Bias Task

2021 ◽  
pp. 1-13
Author(s):  
Vikki Neville ◽  
Peter Dayan ◽  
Iain D. Gilchrist ◽  
Elizabeth S. Paul ◽  
Michael Mendl
Keyword(s):  
Decision Making ◽  
Affective State ◽  
Lapse Rate ◽  
Prediction Errors ◽  
Affective Valence ◽  
Behavioral Measures ◽  
Judgment Bias ◽  
Affective Arousal ◽  
Nonhuman Animals ◽  
Positive Valence

Abstract Good translatability of behavioral measures of affect (emotion) between human and nonhuman animals is core to comparative studies. The judgment bias (JB) task, which measures “optimistic” and “pessimistic” decision-making under ambiguity as indicators of positive and negative affective valence, has been used in both human and nonhuman animals. However, one key disparity between human and nonhuman studies is that the former typically use secondary reinforcers (e.g., money) whereas the latter typically use primary reinforcers (e.g., food). To address this deficiency and shed further light on JB as a measure of affect, we developed a novel version of a JB task for humans using primary reinforcers. Data on decision-making and reported affective state during the JB task were analyzed using computational modeling. Overall, participants grasped the task well, and as anticipated, their reported affective valence correlated with trial-by-trial variation in offered volume of juice. In addition, previous findings from monetary versions of the task were replicated: More positive prediction errors were associated with more positive affective valence, a higher lapse rate was associated with lower affective arousal, and affective arousal decreased as a function of number of trials completed. There was no evidence that more positive valence was associated with greater “optimism,” but instead, there was evidence that affective valence influenced the participants' decision stochasticity, whereas affective arousal tended to influence their propensity for errors. This novel version of the JB task provides a useful tool for investigation of the links between primary reward and punisher experience, affect, and decision-making, especially from a comparative perspective.

Sign in / Sign up

Export Citation Format

Share Document