scholarly journals The neural encoding of information prediction errors during non-instrumental information seeking

2017 ◽  
Author(s):  
Maja Brydevall ◽  
Daniel Bennett ◽  
Carsten Murawski ◽  
Stefan Bode
Keyword(s):  
Decision Making ◽  
Information Seeking ◽  
Prediction Error ◽  
Reward Processing ◽  
Event Related Potential ◽  
Recent Theory ◽  
Prediction Errors ◽  
Neural Encoding ◽  
New Information ◽  
Human Participants

ABSTRACTIn a dynamic world, accurate beliefs about the environment are vital for survival, and individuals should therefore regularly seek out new information with which to update their beliefs. This aspect of behaviour is not well captured by standard theories of decision making, and the neural mechanisms of information seeking remain unclear. One recent theory posits that valuation of information results from representation of informative stimuli within canonical neural reward-processing circuits, even if that information lacks instrumental use. We investigated this question by recording EEG from twenty-three human participants performing a non-instrumental information-seeking task. In this task, participants could pay a monetary cost to receive advance information about the likelihood of receiving reward in a lottery at the end of each trial. Behavioural results showed that participants were willing to incur considerable monetary costs to acquire early but non-instrumental information. Analysis of the event-related potential elicited by informative cues revealed that the feedback-related negativity independently encoded both an information prediction error and a reward prediction error. These findings are consistent with the hypothesis that information seeking results from processing of information within neural reward circuits, and suggests that information may represent a distinct dimension of valuation in decision making under uncertainty.

scholarly journals Novelty is not Surprise: Exploration and learning in human sequential decision-making

2020 ◽  
Author(s):  
He A. Xu ◽  
Alireza Modirshanechi ◽  
Marco P. Lehmann ◽  
Wulfram Gerstner ◽  
Michael H. Herzog
Keyword(s):  
Decision Making ◽  
Prediction Error ◽  
Event Related Potential ◽  
Sequential Decision Making ◽  
Sequential Decision ◽  
Model Free ◽  
Reward Prediction ◽  
Rate Of Learning ◽  
Human Participants ◽  
Exploration Exploitation

AbstractDrivers of reinforcement learning (RL), beyond reward, are controversially debated. Novelty and surprise are often used equivocally in this debate. Here, using a deep sequential decision-making paradigm, we show that reward, novelty, and surprise play different roles in human RL. Surprise controls the rate of learning, whereas novelty and the novelty prediction error (NPE) drive exploration. Exploitation is dominated by model-free (habitual) action choices. A theory that takes these separate effects into account predicts on average 73 percent of the action choices of human participants after the first encounter of a reward and allows us to dissociate surprise and novelty in the EEG signal. While the event-related potential (ERP) at around 300ms is positively correlated with surprise, novelty, NPE, reward, and the reward prediction error, the ERP response to novelty and NPE starts earlier than that to surprise.

scholarly journals Neural encoding of perceived patch value during competitive and hazardous virtual foraging

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Brian Silston ◽  
Toby Wise ◽  
Song Qi ◽  
Xin Sui ◽  
Peter Dayan ◽  
...  
Keyword(s):  
Decision Making ◽  
Social Foraging ◽  
Survival Strategy ◽  
Neural Encoding ◽  
Individual Level ◽  
Hazardous Environments ◽  
Predatory Attack ◽  
Multidimensional Information ◽  
Human Participants ◽  
Foraging Task

AbstractNatural observations suggest that in safe environments, organisms avoid competition to maximize gain, while in hazardous environments the most effective survival strategy is to congregate with competition to reduce the likelihood of predatory attack. We probed the extent to which survival decisions in humans follow these patterns, and examined the factors that determined individual-level decision-making. In a virtual foraging task containing changing levels of competition in safe and hazardous patches with virtual predators, we demonstrate that human participants inversely select competition avoidant and risk diluting strategies depending on perceived patch value (PPV), a computation dependent on reward, threat, and competition. We formulate a mathematically grounded quantification of PPV in social foraging environments and show using multivariate fMRI analyses that PPV is encoded by mid-cingulate cortex (MCC) and ventromedial prefrontal cortices (vMPFC), regions that integrate action and value signals. Together, these results suggest humans utilize and integrate multidimensional information to adaptively select patches highest in PPV, and that MCC and vMPFC play a role in adapting to both competitive and predatory threats in a virtual foraging setting.

scholarly journals Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

2015 ◽  
Vol 113 (1) ◽  
pp. 200-205 ◽  
Author(s):  
Kenneth T. Kishida ◽  
Ignacio Saez ◽  
Terry Lohrenz ◽  
Mark R. Witcher ◽  
Adrian W. Laxton ◽  
...  
Keyword(s):  
Parkinson’S Disease ◽  
Decision Making ◽  
Parkinson's Disease ◽  
Animal Models ◽  
Large Body ◽  
Reward Processing ◽  
Dopamine Neurons ◽  
Model Organisms ◽  
Mammalian Brain ◽  
Prediction Errors

In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson’s disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson’s disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.

scholarly journals The value of what’s to come: neural mechanisms coupling prediction error and reward anticipation

2019 ◽  
Author(s):  
Kiyohito Iigaya ◽  
Tobias U. Hauser ◽  
Zeb Kurth-Nelson ◽  
John P. O’Doherty ◽  
Peter Dayan ◽  
...  
Keyword(s):  
Decision Making ◽  
Information Seeking ◽  
Brain Activity ◽  
Well Being ◽  
Brain Regions ◽  
Functional Coupling ◽  
Hippocampal Activity ◽  
Neural Underpinnings ◽  
To Come ◽  
Human Participants

Having something to look forward to is a keystone of well-being. Anticipation of a future reward, like an upcoming vacation, can often be more gratifying than the very experience itself. Theories of anticipation have described how it induces behaviors ranging from beneficial information-seeking through to harmful addiction. However, it remains unclear how neural systems compute an attractive value from anticipation, instead of from the reward itself. To address this gap, we administered a decision-making task to human participants that allowed us to analyze brain activity during receipt of information predictive of future pleasant outcomes. Using a computational model of anticipatory value that captures participants’ decisions, we show that an anticipatory value signal is orchestrated by influences from three brain regions. Ventromedial prefrontal cortex (vmPFC) tracks the value of anticipation; dopaminergic midbrain responds to information that enhances anticipation, while sustained hippocampal activity provides a functional coupling between these regions. This coordinating function of the hippocampus is consistent with its known role in episodic future thinking. Our findings shed new light on the neural underpinnings of anticipation’s influence over decision-making, while also unifying a range of phenomena associated with risk and time-delay preference.

scholarly journals Testing the Information–Seeking Theory of Openness/Intellect

2020 ◽  
pp. per.2271
Author(s):  
Hayley K. Jach ◽  
Luke D. Smillie
Keyword(s):  
Information Seeking ◽  
Value Of Information ◽  
European Association ◽  
General Purpose ◽  
Recent Theory ◽  
Personality Psychology ◽  
New Information ◽  
Uncertainty Intolerance ◽  
Reward Value ◽  
Domain Level

Why are open people open? A recent theory suggests that openness/intellect reflects sensitivity to the reward value of information, but so far, this has undergone few direct tests. To assess preferences for information, we constructed a novel task, adapted from information–seeking paradigms within decision science, in which participants could choose to see information related to a guessing game they had just completed. Across two studies (one exploratory, n = 151; one confirmatory, n = 301), openness/intellect did not predict information seeking. Our results thus do not support a straightforward version of the theory, whereby open individuals display a general–purpose sensitivity to any sort of new information. However, trait curiosity (arguably a facet of openness/intellect) predicted information seeking in both studies, and uncertainty intolerance (inversely related to openness/intellect) predicted information seeking in Study 2. Thus, it is possible that the domain–level null association masks two divergent information–seeking pathways: one approach motivated (curiosity) and one avoidance motivated (uncertainty intolerance). It remains to be seen whether these conflicting motivations can be isolated and if doing so reveals any association between information–seeking and the broader openness/intellect domain. © 2020 European Association of Personality Psychology

scholarly journals Dopamine, Prediction Error and Beyond

2020 ◽  
pp. 107385842090759
Author(s):  
Kelly M. J. Diederen ◽  
Paul C. Fletcher
Keyword(s):  
Prediction Error ◽  
Large Body ◽  
Theoretical Models ◽  
Optimal Choice ◽  
Reward Processing ◽  
Prediction Errors ◽  
The Difference ◽  
Past Experiences ◽  
Mental Pathology

A large body of work has linked dopaminergic signaling to learning and reward processing. It stresses the role of dopamine in reward prediction error signaling, a key neural signal that allows us to learn from past experiences, and that facilitates optimal choice behavior. Latterly, it has become clear that dopamine does not merely code prediction error size but also signals the difference between the expected value of rewards, and the value of rewards actually received, which is obtained through the integration of reward attributes such as the type, amount, probability and delay. More recent work has posited a role of dopamine in learning beyond rewards. These theories suggest that dopamine codes absolute or unsigned prediction errors, playing a key role in how the brain models associative regularities within its environment, while incorporating critical information about the reliability of those regularities. Work is emerging supporting this perspective and, it has inspired theoretical models of how certain forms of mental pathology may emerge in relation to dopamine function. Such pathology is frequently related to disturbed inferences leading to altered internal models of the environment. Thus, it is critical to understand the role of dopamine in error-related learning and inference.

scholarly journals Event-related potentials reflect prediction errors and pop-out during comprehension of degraded speech

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Leah Banellis ◽  
Rodika Sokoliuk ◽  
Conor J Wild ◽  
Howard Bowman ◽  
Damian Cruse
Keyword(s):  
Prediction Error ◽  
Event Related Potentials ◽  
Event Related Potential ◽  
Prediction Errors ◽  
Top Down ◽  
Degraded Speech ◽  
Related Potentials ◽  
Task Goals ◽  
Post Hoc

Abstract Comprehension of degraded speech requires higher-order expectations informed by prior knowledge. Accurate top-down expectations of incoming degraded speech cause a subjective semantic ‘pop-out’ or conscious breakthrough experience. Indeed, the same stimulus can be perceived as meaningless when no expectations are made in advance. We investigated the event-related potential (ERP) correlates of these top-down expectations, their error signals and the subjective pop-out experience in healthy participants. We manipulated expectations in a word-pair priming degraded (noise-vocoded) speech task and investigated the role of top-down expectation with a between-groups attention manipulation. Consistent with the role of expectations in comprehension, repetition priming significantly enhanced perceptual intelligibility of the noise-vocoded degraded targets for attentive participants. An early ERP was larger for mismatched (i.e. unexpected) targets than matched targets, indicative of an initial error signal not reliant on top-down expectations. Subsequently, a P3a-like ERP was larger to matched targets than mismatched targets only for attending participants—i.e. a pop-out effect—while a later ERP was larger for mismatched targets and did not significantly interact with attention. Rather than relying on complex post hoc interactions between prediction error and precision to explain this apredictive pattern, we consider our data to be consistent with prediction error minimization accounts for early stages of processing followed by Global Neuronal Workspace-like breakthrough and processing in service of task goals.

scholarly journals Neural Signatures of Prediction Errors in a Decision-Making Task Are Modulated by Action Execution Failures

2018 ◽  
Author(s):  
Samuel D. McDougle ◽  
Peter A. Butcher ◽  
Darius Parvin ◽  
Fasial Mushtaq ◽  
Yael Niv ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Learning Task ◽  
Prediction Errors ◽  
Action Execution ◽  
Model Driven ◽  
Assignment Of Credit ◽  
Level Information ◽  
High Level

AbstractDecisions must be implemented through actions, and actions are prone to error. As such, when an expected outcome is not obtained, an individual should not only be sensitive to whether the choice itself was suboptimal, but also whether the action required to indicate that choice was executed successfully. The intelligent assignment of credit to action execution versus action selection has clear ecological utility for the learner. To explore this scenario, we used a modified version of a classic reinforcement learning task in which feedback indicated if negative prediction errors were, or were not, associated with execution errors. Using fMRI, we asked if prediction error computations in the human striatum, a key substrate in reinforcement learning and decision making, are modulated when a failure in action execution results in the negative outcome. Participants were more tolerant of non-rewarded outcomes when these resulted from execution errors versus when execution was successful but the reward was withheld. Consistent with this behavior, a model-driven analysis of neural activity revealed an attenuation of the signal associated with negative reward prediction error in the striatum following execution failures. These results converge with other lines of evidence suggesting that prediction errors in the mesostriatal dopamine system integrate high-level information during the evaluation of instantaneous reward outcomes.

scholarly journals Prediction errors disrupt hippocampal representations and update episodic memories

2021 ◽  
Vol 118 (51) ◽  
pp. e2117625118
Author(s):  
Alyssa H. Sinclair ◽  
Grace M. Manalili ◽  
Iva K. Brunec ◽  
R. Alison Adcock ◽  
Morgan D. Barense
Keyword(s):  
Prediction Error ◽  
Attention And Memory ◽  
Prediction Errors ◽  
Memory Updating ◽  
Learning From Errors ◽  
Activation Patterns ◽  
New Information ◽  
Episodic Memories ◽  
Mri Study ◽  
The Relationship

The brain supports adaptive behavior by generating predictions, learning from errors, and updating memories to incorporate new information. Prediction error, or surprise, triggers learning when reality contradicts expectations. Prior studies have shown that the hippocampus signals prediction errors, but the hypothesized link to memory updating has not been demonstrated. In a human functional MRI study, we elicited mnemonic prediction errors by interrupting familiar narrative videos immediately before the expected endings. We found that prediction errors reversed the relationship between univariate hippocampal activation and memory: greater hippocampal activation predicted memory preservation after expected endings, but memory updating after surprising endings. In contrast to previous studies, we show that univariate activation was insufficient for understanding hippocampal prediction error signals. We explain this surprising finding by tracking both the evolution of hippocampal activation patterns and the connectivity between the hippocampus and neuromodulatory regions. We found that hippocampal activation patterns stabilized as each narrative episode unfolded, suggesting sustained episodic representations. Prediction errors disrupted these sustained representations and the degree of disruption predicted memory updating. The relationship between hippocampal activation and subsequent memory depended on concurrent basal forebrain activation, supporting the idea that cholinergic modulation regulates attention and memory. We conclude that prediction errors create conditions that favor memory updating, prompting the hippocampus to abandon ongoing predictions and make memories malleable.

scholarly journals Aberrant reward prediction error during Pavlovian appetitive learning in alexithymia

2019 ◽  
Vol 14 (10) ◽  
pp. 1119-1129
Author(s):  
Francesca Starita ◽  
Mattia Pietrelli ◽  
Caterina Bertini ◽  
Giuseppe di Pellegrino
Keyword(s):  
Prediction Error ◽  
Event Related Potential ◽  
Extensive Literature ◽  
Prediction Errors ◽  
Appetitive Conditioning ◽  
Emotional Stimuli ◽  
Internal Representations ◽  
Reward Prediction ◽  
Underlying Mechanisms ◽  
Emotional Value

Abstract Extensive literature shows that alexithymia, a subclinical trait defined by difficulties in identifying and describing feelings, is characterized by multifaceted impairments in processing emotional stimuli. Nevertheless, its underlying mechanisms remain elusive. Here, we hypothesize that alexithymia may be characterized by an alteration in learning the emotional value of encountered stimuli and test this by assessing differences between individuals with low (LA) and high (HA) levels of alexithymia in the computation of reward prediction errors (RPEs) during Pavlovian appetitive conditioning. As a marker of RPE, the amplitude of the feedback-related negativity (FRN) event-related potential was assessed while participants were presented with two conditioned stimuli (CS) associated with expected or unexpected feedback, indicating delivery of reward or no-reward. No-reward (vs reward) feedback elicited the FRN both in LA and HA. However, unexpected (vs expected) feedback enhanced the FRN in LA but not in HA, indicating impaired computation of RPE in HA. Thus, although HA show preserved sensitivity to rewards, they cannot use this response to update the value of CS that predict them. This impairment may hinder the construction of internal representations of emotional stimuli, leaving individuals with alexithymia unable to effectively recognize, respond and regulate their response to emotional stimuli.

Sign in / Sign up

Export Citation Format

Share Document