scholarly journals Beta oscillations following performance feedback predict subsequent recall of task-relevant information

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Azadeh HajiHosseini ◽  
Cendri A. Hutcherson ◽  
Clay B. Holroyd
Keyword(s):  
Reinforcement Learning ◽  
Performance Feedback ◽  
Relevant Information ◽  
Learning Task ◽  
Reward Processing ◽  
Recall Condition ◽  
Error Feedback ◽  
Reward Delivery ◽  
Beta Power ◽  
Subsequent Recall

Abstract Reward delivery in reinforcement learning tasks elicits increased beta power in the human EEG over frontal areas of the scalp but it is unclear whether these 20–30 Hz oscillations directly facilitate reward learning. We previously proposed that frontal beta is not specific to reward processing but rather reflects the role of prefrontal cortex in maintaining and transferring task-related information to other brain areas. To test this proposal, we had subjects perform a reinforcement learning task followed by a memory recall task in which subjects were asked to recall stimuli associated either with reward feedback (Reward Recall condition) or error feedback (Error Recall condition). We trained a classifier on post-feedback beta power in the Reward Recall condition to discriminate trials associated with reward feedback from those associated with error feedback and then tested the classifier on post-feedback beta power in the Error Recall condition. Crucially, the model classified error-related beta in the Error Recall condition as reward-related. The model also predicted stimulus recall from post-feedback beta power irrespective of feedback valence and task condition. These results indicate that post-feedback beta power is not specific to reward processing but rather reflects a more general task-related process.

scholarly journals How the level of reward awareness changes the computational and electrophysiological signatures of reinforcement learning

2018 ◽  
Author(s):  
C.M.C. Correa ◽  
S. Noorman ◽  
J. Jiang ◽  
S. Palminteri ◽  
M.X Cohen ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Present Report ◽  
Subjective Evaluation ◽  
Computational Modelling ◽  
Learning Task ◽  
Reward Processing ◽  
Neural Activities ◽  
Electrophysiological Recordings ◽  
Neural Computations

AbstractThe extent to which subjective awareness influences reward processing, and thereby affects future decisions is currently largely unknown. In the present report, we investigated this question in a reinforcement-learning framework, combining perceptual masking, computational modeling and electroencephalographic recordings (human male and female participants). Our results indicate that degrading the visibility of the reward decreased -without completely obliterating- the ability of participants to learn from outcomes, but concurrently increased their tendency to repeat previous choices. We dissociated electrophysiological signatures evoked by the reward-based learning processes from those elicited by the reward-independent repetition of previous choices and showed that these neural activities were significantly modulated by reward visibility. Overall, this report sheds new light on the neural computations underlying reward-based learning and decision-making and highlights that awareness is beneficial for the trial-by-trial adjustment of decision-making strategies.Significance statementThe notion of reward is strongly associated with subjective evaluation, related to conscious processes such as “pleasure”, “liking” and “wanting”. Here we show that degrading reward visibility in a reinforcement learning task decreases -without completely obliterating- the ability of participants to learn from outcomes, but concurrently increases subjects tendency to repeat previous choices. Electrophysiological recordings, in combination with computational modelling, show that neural activities were significantly modulated by reward visibility. Overall, we dissociate different neural computations underlying reward-based learning and decision-making, which highlights a beneficial role of reward awareness in adjusting decision-making strategies.

scholarly journals Gaze-contingent reinforcement learning reveals incentive value of social signals in young children and adults

2017 ◽  
Vol 284 (1850) ◽  
pp. 20162747 ◽  
Author(s):  
Angélina Vernetti ◽  
Tim J. Smith ◽  
Atsushi Senju
Keyword(s):  
Reinforcement Learning ◽  
Visual Cues ◽  
Real Life ◽  
Learning Task ◽  
Social Cues ◽  
Social Signals ◽  
Social Orienting ◽  
Incentive Value ◽  
Reward Delivery ◽  
New Findings

While numerous studies have demonstrated that infants and adults preferentially orient to social stimuli, it remains unclear as to what drives such preferential orienting. It has been suggested that the learned association between social cues and subsequent reward delivery might shape such social orienting. Using a novel, spontaneous indication of reinforcement learning (with the use of a gaze contingent reward-learning task), we investigated whether children and adults' orienting towards social and non-social visual cues can be elicited by the association between participants' visual attention and a rewarding outcome. Critically, we assessed whether the engaging nature of the social cues influences the process of reinforcement learning. Both children and adults learned to orient more often to the visual cues associated with reward delivery, demonstrating that cue–reward association reinforced visual orienting. More importantly, when the reward-predictive cue was social and engaging, both children and adults learned the cue–reward association faster and more efficiently than when the reward-predictive cue was social but non-engaging. These new findings indicate that social engaging cues have a positive incentive value. This could possibly be because they usually coincide with positive outcomes in real life, which could partly drive the development of social orienting.

scholarly journals Comparison of conventional and rapid-acting antidepressants in a rodent probabilistic reversal learning task

2020 ◽  
Vol 4 ◽  
pp. 239821282090717 ◽  
Author(s):  
Matthew P. Wilkinson ◽  
John P. Grogan ◽  
Jack R. Mellor ◽  
Emma S. J. Robinson
Keyword(s):  
Reinforcement Learning ◽  
Reversal Learning ◽  
Antidepressant Treatment ◽  
Learning Task ◽  
Reward Processing ◽  
Reward Learning ◽  
Feedback Sensitivity ◽  
Rule Changes ◽  
Reinforcement Learning Model ◽  
Probabilistic Reversal Learning

Deficits in reward processing are a central feature of major depressive disorder with patients exhibiting decreased reward learning and altered feedback sensitivity in probabilistic reversal learning tasks. Methods to quantify probabilistic learning in both rodents and humans have been developed, providing translational paradigms for depression research. We have utilised a probabilistic reversal learning task to investigate potential differences between conventional and rapid-acting antidepressants on reward learning and feedback sensitivity. We trained 12 rats in a touchscreen probabilistic reversal learning task before investigating the effect of acute administration of citalopram, venlafaxine, reboxetine, ketamine or scopolamine. Data were also analysed using a Q-learning reinforcement learning model to understand the effects of antidepressant treatment on underlying reward processing parameters. Citalopram administration decreased trials taken to learn the first rule and increased win-stay probability. Reboxetine decreased win-stay behaviour while also decreasing the number of rule changes animals performed in a session. Venlafaxine had no effect. Ketamine and scopolamine both decreased win-stay probability, number of rule changes performed and motivation in the task. Insights from the reinforcement learning model suggested that reboxetine led animals to choose a less optimal strategy, while ketamine decreased the model-free learning rate. These results suggest that reward learning and feedback sensitivity are not differentially modulated by conventional and rapid-acting antidepressant treatment in the probabilistic reversal learning task.

scholarly journals EEG correlates of physical effort and reward processing during reinforcement learning

2020 ◽  
Vol 124 (2) ◽  
pp. 610-622
Author(s):  
Dimitrios J. Palidis ◽  
Paul L. Gribble
Keyword(s):  
Reinforcement Learning ◽  
Task Performance ◽  
Performance Feedback ◽  
Reward Processing ◽  
Neural Responses ◽  
Physical Effort ◽  
Motivational Effect ◽  
Subsequent Task

Choices probabilistically determined the physical effort requirements for a subsequent task, and reward depended on task performance. Feedback revealing whether choices resulted in easy or hard effort did not elicit reinforcement learning signals. However, the neural responses to reinforcement were modulated by preceding effort. Thus, effort itself was not treated as loss or punishment, but it affected the responses to subsequent reinforcement outcomes. This may explain how effort can enhance the motivational effect of reward.

Neuroevolution approach for reinforcement learning task applied to control system of a cart pole system

2019 ◽  
Author(s):  
Marco Boaretto ◽  
Gabriel Chaves Becchi ◽  
Luiza Scapinello Aquino ◽  
Aderson Cleber Pifer ◽  
Helon Vicente Hultmann Ayala ◽  
...  
Keyword(s):  
Control System ◽  
Reinforcement Learning ◽  
Learning Task ◽  
Pole System

Model-Based and Model-Free Social Cognition

2019 ◽  
Author(s):  
Leor M Hackel ◽  
Jeffrey Jordan Berg ◽  
Björn Lindström ◽  
David Amodio
Keyword(s):  
Reinforcement Learning ◽  
Social Cognition ◽  
Learning Strategies ◽  
Memory Systems ◽  
Learning Task ◽  
Financial Advisors ◽  
Model Based ◽  
Model Free ◽  
Systems Model ◽  
Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

scholarly journals Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Batel Yifrah ◽  
Ayelet Ramaty ◽  
Genela Morris ◽  
Avi Mendelsohn
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Declarative Memory ◽  
Contextual Information ◽  
Memory Performance ◽  
Relevant Information ◽  
Subjective Memory ◽  
Types Of Information ◽  
Reinforcement Learning Models ◽  
Implicit And Explicit

AbstractDecision making can be shaped both by trial-and-error experiences and by memory of unique contextual information. Moreover, these types of information can be acquired either by means of active experience or by observing others behave in similar situations. The interactions between reinforcement learning parameters that inform decision updating and memory formation of declarative information in experienced and observational learning settings are, however, unknown. In the current study, participants took part in a probabilistic decision-making task involving situations that either yielded similar outcomes to those of an observed player or opposed them. By fitting alternative reinforcement learning models to each subject, we discerned participants who learned similarly from experience and observation from those who assigned different weights to learning signals from these two sources. Participants who assigned different weights to their own experience versus those of others displayed enhanced memory performance as well as subjective memory strength for episodes involving significant reward prospects. Conversely, memory performance of participants who did not prioritize their own experience over others did not seem to be influenced by reinforcement learning parameters. These findings demonstrate that interactions between implicit and explicit learning systems depend on the means by which individuals weigh relevant information conveyed via experience and observation.

scholarly journals The rational use of causal inference to guide reinforcement learning strengthens with age

2019 ◽  
Author(s):  
Alexandra O. Cohen ◽  
Kate Nussenbaum ◽  
Hayley Dorfman ◽  
Samuel J. Gershman ◽  
Catherine A. Hartley
Keyword(s):  
Reinforcement Learning ◽  
Causal Structure ◽  
Learning Task ◽  
Negative Events ◽  
Shape Learning ◽  
Adolescents And Adults ◽  
Bayesian Reinforcement Learning ◽  
External Causes ◽  
Reinforcement Learning Models ◽  
Best Fit

Beliefs about the controllability of positive or negative events in the environment can shape learning throughout the lifespan. Previous research has shown that adults’ learning is modulated by beliefs about the causal structure of the environment such that they will update their value estimates to a lesser extent when the outcomes can be attributed to hidden causes. The present study examined whether external causes similarly influenced outcome attributions and learning across development. Ninety participants, ages 7 to 25 years, completed a reinforcement learning task in which they chose between two options with fixed reward probabilities. Choices were made in three distinct environments in which different hidden agents occasionally intervened to generate positive, negative, or random outcomes. Participants’ beliefs about hidden-agent intervention aligned well with the true probabilities of positive, negative, or random outcome manipulation in each of the three environments. Computational modeling of the learning data revealed that while the choices made by both adults (ages 18 - 25) and adolescents (ages 13 - 17) were best fit by Bayesian reinforcement learning models that incorporate beliefs about hidden agent intervention, those of children (ages 7 - 12) were best fit by a one learning rate model that updates value estimates based on choice outcomes alone. Together, these results suggest that while children demonstrate explicit awareness of the causal structure of the task environment they do not implicitly use beliefs about the causal structure of the environment to guide reinforcement learning in the same manner as adolescents and adults.

Sign in / Sign up

Export Citation Format

Share Document