scholarly journals Monetary feedback modulates performance and electrophysiological indices of belief updating in reward learning

2017 ◽  
Author(s):  
Daniel Bennett ◽  
Karen Sasmita ◽  
Ryan T. Maloney ◽  
Carsten Murawski ◽  
Stefan Bode
Keyword(s):  
Bayesian Inference ◽  
Cognitive Strategies ◽  
Learning Task ◽  
Feedback Condition ◽  
Event Related Potential ◽  
Reward Learning ◽  
Instructive Feedback ◽  
Motivational State ◽  
Belief Updating ◽  
Learning Rates

AbstractBelief updating entails the incorporation of new information about the environment into internal models of the world. Bayesian inference is the statistically optimal strategy for performing belief updating in the presence of uncertainty. An important open question is whether the use of cognitive strategies that implement Bayesian inference is dependent upon motivational state and, if so, how this is reflected in electrophysiological signatures of belief updating in the brain. Here we recorded the electroencephalogram of participants performing a simple reward learning task with both monetary and non-monetary instructive feedback conditions. Our aim was to distinguish the influence of the rewarding properties of feedback on belief updating from the information content of the feedback itself. A Bayesian updating model allowed us to quantify different aspects of belief updating across trials, including the size of belief updates and the uncertainty of beliefs. Faster learning rates were observed in the monetary feedback condition compared to the instructive feedback condition, while belief updates were generally larger, and belief uncertainty smaller, with monetary compared to instructive feedback. Larger amplitudes in the monetary feedback condition were found for three event-related potential components: the P3a, the feedback-related negativity (FRN) and the late positive potential (LPP). These findings suggest that motivational state influences inference strategies in reward learning, and this is reflected in the electrophysiological correlates of belief updating.

scholarly journals Autistic traits are related to worse performance in a volatile reward learning task despite adaptive learning rates

Autism ◽  
2020 ◽  
pp. 136236132096223 ◽  
Author(s):  
Judith Goris ◽  
Massimo Silvetti ◽  
Tom Verguts ◽  
Jan R Wiersema ◽  
Marcel Brass ◽  
...  
Keyword(s):  
Adaptive Learning ◽  
Autistic Traits ◽  
Learning Task ◽  
Low Noise ◽  
Reward Learning ◽  
Optimal Decision ◽  
Noise Condition ◽  
Impaired Performance ◽  
Learning Rates ◽  
Reward Contingencies

Recent theories propose that autism is characterized by an impairment in determining when to learn and when not. We investigated this by estimating learning rate in environments varying in volatility and uncertainty. Specifically, we correlated autistic traits (in 163 neurotypical participants) with modelled learning behaviour during probabilistic reward learning under the following three conditions: a Stationary Low Noise condition with stable reward contingencies, a Volatile condition with changing reward contingencies and a Stationary High Noise condition where reward probabilities for all options were 60%, resulting in an uncertain, noisy environment. Consistent with earlier findings, we found less optimal decision-making in the Volatile condition for participants with more autistic traits. However, we observed no correlations between underlying adjustments in learning rates and autistic traits, suggesting no impairment in updating learning rates in response to volatile versus noisy environments. Exploratory analyses indicated that impaired performance in the Volatile condition in participants with more autistic traits, was specific to trials with reward contingencies opposite to those initially learned, suggesting a primacy bias. We conclude that performance in volatile environments is lower in participants with more autistic traits, but this cannot be unambiguously attributed to difficulties with adjusting learning rates. Lay abstract Recent theories propose that autism is characterized by an impairment in determining when to learn and when not. Here, we investigated this hypothesis by estimating learning rates (i.e. the speed with which one learns) in three different environments that differed in rule stability and uncertainty. We found that neurotypical participants with more autistic traits performed worse in a volatile environment (with unstable rules), as they chose less often for the most rewarding option. Exploratory analyses indicated that performance was specifically worse when reward rules were opposite to those initially learned for participants with more autistic traits. However, there were no differences in the adjustment of learning rates between participants with more versus less autistic traits. Together, these results suggest that performance in volatile environments is lower in participants with more autistic traits, but that this performance difference cannot be unambiguously explained by an impairment in adjusting learning rates.

Knowledge Acquired from Learning: New Evidence of Hierarchical Conceptualization

1994 ◽  
Vol 79 (2) ◽  
pp. 975-993 ◽  
Author(s):  
Alberto Montare
Keyword(s):  
Sex Differences ◽  
Knowledge Acquisition ◽  
Reversal Learning ◽  
Concept Formation ◽  
Learning Task ◽  
Learning Rates ◽  
Level 3 ◽  
Level 1 ◽  
New Evidence ◽  
Level 2

Following successful inductive acquisition of procedural cognition of a discrimination-reversal learning task, 50 female and 50 male undergraduates articulated declarative cognizance of knowledge acquired from learning. Tests of four hypotheses showed that (1) increasingly higher levels of declarative cognizance were associated with faster learning rates, (2) six new cases of cognition-without-cognizance were observed, (3) students presumably using secondary signalization learned faster than those presumably using primary signalization, and (4) no sex differences in learning rates or declarative cognizance were observed. The notion that explicit levels of declarative cognizance may represent implicit hierarchical conceptualization comprised of four systems of knowledge acquisition led to the conclusions that primary signalization may account for inductive senscept formation at Level 1 and for inductive percept formation at Level 2, whereas emergent secondary signalization may account for inductive precept formation at Level 3 and for inductive concept formation at Level 4.

scholarly journals Smoking as a Common Modulator of Sensory Gating and Reward Learning in Individuals with Psychotic Disorders

2021 ◽  
Vol 11 (12) ◽  
pp. 1581
Author(s):  
Alexis E. Whitton ◽  
Kathryn E. Lewandowski ◽  
Mei-Hua Hall
Keyword(s):  
Bipolar Disorder ◽  
Prediction Error ◽  
Psychotic Disorders ◽  
Predictive Coding ◽  
Sensory Gating ◽  
Event Related Potential ◽  
Reward Learning ◽  
Smoking History ◽  
Impaired Performance ◽  
Psychotic Bipolar Disorder

Motivational and perceptual disturbances co-occur in psychosis and have been linked to aberrations in reward learning and sensory gating, respectively. Although traditionally studied independently, when viewed through a predictive coding framework, these processes can both be linked to dysfunction in striatal dopaminergic prediction error signaling. This study examined whether reward learning and sensory gating are correlated in individuals with psychotic disorders, and whether nicotine—a psychostimulant that amplifies phasic striatal dopamine firing—is a common modulator of these two processes. We recruited 183 patients with psychotic disorders (79 schizophrenia, 104 psychotic bipolar disorder) and 129 controls and assessed reward learning (behavioral probabilistic reward task), sensory gating (P50 event-related potential), and smoking history. Reward learning and sensory gating were correlated across the sample. Smoking influenced reward learning and sensory gating in both patient groups; however, the effects were in opposite directions. Specifically, smoking was associated with improved performance in individuals with schizophrenia but impaired performance in individuals with psychotic bipolar disorder. These findings suggest that reward learning and sensory gating are linked and modulated by smoking. However, disorder-specific associations with smoking suggest that nicotine may expose pathophysiological differences in the architecture and function of prediction error circuitry in these overlapping yet distinct psychotic disorders.

scholarly journals Biased belief in the Bayesian brain: A deeper look at the evidence

2019 ◽  
Author(s):  
Ben M Tappin ◽  
Stephen Gadsby
Keyword(s):  
Bayesian Inference ◽  
Empirical Evidence ◽  
Belief Formation ◽  
Hierarchical Bayesian Models ◽  
Common Occurrence ◽  
Substantial Evidence ◽  
Belief Updating ◽  
Bayesian Brain ◽  
Key Points ◽  
Study Designs

A recent critique of hierarchical Bayesian models of delusion argues that, contrary to a key assumption of these models, belief formation in the healthy (i.e., neurotypical) mind is manifestly non-Bayesian. Here we provide a deeper examination of the empirical evidence underlying this critique. We argue that this evidence does not convincingly refute the assumption that belief formation in the neurotypical mind approximates Bayesian inference. Our argument rests on two key points. First, evidence that purports to reveal the most damning violation of Bayesian updating in human belief formation is counterweighted by substantial evidence that indicates such violations are the rare exception—not a common occurrence. Second, the remaining evidence does not demonstrate convincing violations of Bayesian inference in human belief updating; primarily because this evidence derives from study designs that produce results that are not obviously inconsistent with Bayesian principles.

scholarly journals Consolidation of reward memory during sleep does not require dopaminergic activation

2019 ◽  
Author(s):  
M Alizadeh Asfestani ◽  
V Brechtmann ◽  
J Santiago ◽  
J Born ◽  
GB Feld
Keyword(s):  
Learning Task ◽  
Reward Learning ◽  
Nocturnal Sleep ◽  
Double Blind ◽  
Dopaminergic Neurotransmission ◽  
Long Duration ◽  
Differential Retention ◽  
Better Than ◽  
Dopaminergic Activation

AbstractSleep enhances memories, especially, if they are related to future rewards. Although dopamine has been shown to be a key determinant during reward learning, the role of dopaminergic neurotransmission for amplifying reward-related memories during sleep remains unclear. In the present study, we scrutinize the idea that dopamine is needed for the preferential consolidation of rewarded information. We blocked dopaminergic neurotransmission, thereby aiming to wipe out preferential sleep-dependent consolidation of high over low rewarded memories during sleep. Following a double-blind, balanced, crossover design 20 young healthy men received the dopamine d2-like receptor blocker Sulpiride (800 mg) or placebo, after learning a Motivated Learning Task. The task required participants to memorize 80 highly and 80 lowly rewarded pictures. Half of them were presented for a short (750 ms) and a long duration (1500 ms), respectively, which enabled to dissociate effects of reward on sleep-associated consolidation from those of mere encoding depth. Retrieval was tested after a retention interval of 20 h that included 8 h of nocturnal sleep. As expected, at retrieval, highly rewarded memories were remembered better than lowly rewarded memories, under placebo. However, there was no evidence for an effect of blocking dopaminergic neurotransmission with Sulpiride during sleep on this differential retention of rewarded information. This result indicates that dopaminergic activation is not required for the preferential consolidation of reward-associated memory. Rather it appears that dopaminergic activation only tags such memories at encoding for intensified reprocessing during sleep.

scholarly journals Consolidation of Reward Memory during Sleep Does Not Require Dopaminergic Activation

2020 ◽  
Vol 32 (9) ◽  
pp. 1688-1703 ◽  
Author(s):  
Marjan Alizadeh Asfestani ◽  
Valentin Brechtmann ◽  
João Santiago ◽  
Andreas Peter ◽  
Jan Born ◽  
...  
Keyword(s):  
Learning Task ◽  
Reward Learning ◽  
Nocturnal Sleep ◽  
Double Blind ◽  
Dopaminergic Neurotransmission ◽  
Msec Duration ◽  
Differential Retention ◽  
Better Than ◽  
Dopaminergic Activation

Sleep enhances memories, especially if they are related to future rewards. Although dopamine has been shown to be a key determinant during reward learning, the role of dopaminergic neurotransmission for amplifying reward-related memories during sleep remains unclear. In this study, we scrutinize the idea that dopamine is needed for the preferential consolidation of rewarded information. We impaired dopaminergic neurotransmission, thereby aiming to wipe out preferential sleep-dependent consolidation of high- over low-rewarded memories during sleep. Following a double-blind, balanced, crossover design, 17 young healthy men received the dopamine d2-like receptor blocker sulpiride (800 mg) or placebo, after learning a motivated learning task. The task required participants to memorize 80 highly and 80 lowly rewarded pictures. Half of them were presented for a short (750 msec) and a long (1500 msec) duration, respectively, which permitted dissociation of the effects of reward on sleep-associated consolidation from those of mere encoding depth. Retrieval was tested after a retention interval of approximately 22 hr that included 8 hr of nocturnal sleep. As expected, at retrieval, highly rewarded memories were remembered better than lowly rewarded memories, under placebo. However, there was no evidence for an effect of reducing dopaminergic neurotransmission with sulpiride during sleep on this differential retention of rewarded information. This result indicates that dopaminergic activation likely is not required for the preferential consolidation of reward-associated memory. Rather, it appears that dopaminergic activation only tags such memories at encoding for intensified reprocessing during sleep.

scholarly journals Neural signatures of reward and sensory error feedback processing in motor learning

2019 ◽  
Vol 121 (4) ◽  
pp. 1561-1574 ◽  
Author(s):  
Dimitrios J. Palidis ◽  
Joshua G. A. Cashaback ◽  
Paul L. Gribble
Keyword(s):  
Visual Feedback ◽  
Motor Adaptation ◽  
Learning Task ◽  
Reward Learning ◽  
P300 Amplitude ◽  
Error Feedback ◽  
Learning Mechanisms ◽  
Visuomotor Rotation ◽  
Feedback Processing ◽  
Motor Commands

At least two distinct processes have been identified by which motor commands are adapted according to movement-related feedback: reward-based learning and sensory error-based learning. In sensory error-based learning, mappings between sensory targets and motor commands are recalibrated according to sensory error feedback. In reward-based learning, motor commands are associated with subjective value, such that successful actions are reinforced. We designed two tasks to isolate reward- and sensory error-based motor adaptation, and we used electroencephalography in humans to identify and dissociate the neural correlates of reward and sensory error feedback processing. We designed a visuomotor rotation task to isolate sensory error-based learning that was induced by altered visual feedback of hand position. In a reward learning task, we isolated reward-based learning induced by binary reward feedback that was decoupled from the visual target. A fronto-central event-related potential called the feedback-related negativity (FRN) was elicited specifically by binary reward feedback but not sensory error feedback. A more posterior component called the P300 was evoked by feedback in both tasks. In the visuomotor rotation task, P300 amplitude was increased by sensory error induced by perturbed visual feedback and was correlated with learning rate. In the reward learning task, P300 amplitude was increased by reward relative to nonreward and by surprise regardless of feedback valence. We propose that during motor adaptation the FRN specifically reflects a reward-based learning signal whereas the P300 reflects feedback processing that is related to adaptation more generally. NEW & NOTEWORTHY We studied the event-related potentials evoked by feedback stimuli during motor adaptation tasks that isolate reward- and sensory error-based learning mechanisms. We found that the feedback-related negativity was specifically elicited by binary reward feedback, whereas the P300 was observed in both tasks. These results reveal neural processes associated with different learning mechanisms and elucidate which classes of errors, from a computational standpoint, elicit the feedback-related negativity and P300.

scholarly journals Affective bias as a rational response to the statistics of rewards and punishments

eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
Erdem Pulcu ◽  
Michael Browning
Keyword(s):  
Information Content ◽  
Learning Task ◽  
Negative Outcomes ◽  
Computational Framework ◽  
Positive Events ◽  
Treatment Development ◽  
Learning Rates ◽  
Novel Approach ◽  
Affective Bias ◽  
Human Participants

Affective bias, the tendency to differentially prioritise the processing of negative relative to positive events, is commonly observed in clinical and non-clinical populations. However, why such biases develop is not known. Using a computational framework, we investigated whether affective biases may reflect individuals’ estimates of the information content of negative relative to positive events. During a reinforcement learning task, the information content of positive and negative outcomes was manipulated independently by varying the volatility of their occurrence. Human participants altered the learning rates used for the outcomes selectively, preferentially learning from the most informative. This behaviour was associated with activity of the central norepinephrine system, estimated using pupilometry, for loss outcomes. Humans maintain independent estimates of the information content of distinct positive and negative outcomes which may bias their processing of affective events. Normalising affective biases using computationally inspired interventions may represent a novel approach to treatment development.

scholarly journals Deficits in reinforcement learning but no link to apathy in patients with schizophrenia

2017 ◽  
Vol 7 (1) ◽  
Author(s):  
Matthias N. Hartmann-Riemer ◽  
Steffen Aschenbrenner ◽  
Magdalena Bossert ◽  
Celina Westermann ◽  
Erich Seifritz ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Negative Symptoms ◽  
Learning Task ◽  
Reward Learning ◽  
Positive Outcomes ◽  
Learning Deficits ◽  
Loss Avoidance ◽  
Healthy Control ◽  
Precise Representation ◽  
The Stability

Abstract Negative symptoms in schizophrenia have been linked to selective reinforcement learning deficits in the context of gains combined with intact loss-avoidance learning. Fundamental mechanisms of reinforcement learning and choice are prediction error signaling and the precise representation of reward value for future decisions. It is unclear which of these mechanisms contribute to the impairments in learning from positive outcomes observed in schizophrenia. A recent study suggested that patients with severe apathy symptoms show deficits in the representation of expected value. Considering the fundamental relevance for the understanding of these symptoms, we aimed to assess the stability of these findings across studies. Sixty-four patients with schizophrenia and 19 healthy control participants performed a probabilistic reward learning task. They had to associate stimuli with gain or loss-avoidance. In a transfer phase participants indicated valuation of the previously learned stimuli by choosing among them. Patients demonstrated an overall impairment in learning compared to healthy controls. No effects of apathy symptoms on task indices were observed. However, patients with schizophrenia learned better in the context of loss-avoidance than in the context of gain. Earlier findings were thus partially replicated. Further studies are needed to clarify the mechanistic link between negative symptoms and reinforcement learning.

scholarly journals Learning “How to Learn”: Super Declarative Motor Learning Is Impaired in Parkinson’s Disease

2017 ◽  
Vol 2017 ◽  
pp. 1-8 ◽  
Author(s):  
Lucio Marinelli ◽  
Carlo Trompetto ◽  
Stefania Canneva ◽  
Laura Mori ◽  
Flavio Nobili ◽  
...  
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Performance Optimization ◽  
Learning Task ◽  
Daily Activities ◽  
Learning Capacity ◽  
Learning Rates ◽  
New Information ◽  
Drug Naïve ◽  
Better Than

Learning new information is crucial in daily activities and occurs continuously during a subject’s lifetime. Retention of learned material is required for later recall and reuse, although learning capacity is limited and interference between consecutively learned information may occur. Learning processes are impaired in Parkinson’s disease (PD); however, little is known about the processes related to retention and interference. The aim of this study is to investigate the retention and anterograde interference using a declarative sequence learning task in drug-naive patients in the disease’s early stages. Eleven patients with PD and eleven age-matched controls learned a visuomotor sequence, SEQ1, during Day1; the following day, retention of SEQ1 was assessed and, immediately after, a new sequence of comparable complexity, SEQ2, was learned. The comparison of the learning rates of SEQ1 on Day1 and SEQ2 on Day2 assessed the anterograde interference of SEQ1 on SEQ2. We found that SEQ1 performance improved in both patients and controls on Day2. Surprisingly, controls learned SEQ2 better than SEQ1, suggesting the absence of anterograde interference and the occurrence of learning optimization, a process that we defined as “learning how to learn.” Patients with PD lacked such improvement, suggesting defective performance optimization processes.

Sign in / Sign up

Export Citation Format

Share Document