scholarly journals Electrophysiological biomarkers of behavioral dimensions from cross-species paradigms

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
James F. Cavanagh ◽  
David Gregg ◽  
Gregory A. Light ◽  
Sarah L. Olguin ◽  
Richard F. Sharp ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Cognitive Control ◽  
A Priori ◽  
Progressive Ratio ◽  
Learning Task ◽  
Clinical Settings ◽  
Continuous Performance Task ◽  
Electrophysiological Signals ◽  
Behavioral Constructs ◽  
Band Activity

AbstractThere has been a fundamental failure to translate preclinically supported research into clinically efficacious treatments for psychiatric disorders. One of the greatest impediments toward improving this species gap has been the difficulty of identifying translatable neurophysiological signals that are related to specific behavioral constructs. Here, we present evidence from three paradigms that were completed by humans and mice using analogous procedures, with each task eliciting candidate a priori defined electrophysiological signals underlying effortful motivation, reinforcement learning, and cognitive control. The effortful motivation was assessed using a progressive ratio breakpoint task, yielding a similar decrease in alpha-band activity over time in both species. Reinforcement learning was assessed via feedback in a probabilistic learning task with delta power significantly modulated by reward surprise in both species. Additionally, cognitive control was assessed in the five-choice continuous performance task, yielding response-locked theta power seen across species, and modulated by difficulty in humans. Together, these successes, and also the teachings from these failures, provide a roadmap towards the use of electrophysiology as a method for translating findings from the preclinical assays to the clinical settings.

scholarly journals Memory-reliant Post-error Slowing Is Associated with Successful Learning and Fronto-occipital Activity

2016 ◽  
Vol 28 (10) ◽  
pp. 1539-1552 ◽  
Author(s):  
Björn C. Schiffler ◽  
Rita Almeida ◽  
Mathias Granqvist ◽  
Sara L. Bengtsson
Keyword(s):  
Reinforcement Learning ◽  
Cognitive Control ◽  
Negative Feedback ◽  
Test Performance ◽  
Cognitive Task ◽  
Inferior Frontal Gyrus ◽  
Response Speed ◽  
Occipital Cortex ◽  
Learning Task ◽  
Learning Theories

Negative feedback after an action in a cognitive task can lead to devaluing that action on future trials as well as to more cautious responding when encountering that same choice again. These phenomena have been explored in the past by reinforcement learning theories and cognitive control accounts, respectively. Yet, how cognitive control interacts with value updating to give rise to adequate adaptations under uncertainty is less clear. In this fMRI study, we investigated cognitive control-based behavioral adjustments during a probabilistic reinforcement learning task and studied their influence on performance in a later test phase in which the learned value of items is tested. We provide support for the idea that functionally relevant and memory-reliant behavioral adjustments in the form of post-error slowing during reinforcement learning are associated with test performance. Adjusting response speed after negative feedback was correlated with BOLD activity in right inferior frontal gyrus and bilateral middle occipital cortex during the event of receiving the feedback. Bilateral middle occipital cortex activity overlapped partly with activity reflecting feedback deviance from expectations as measured by unsigned prediction error. These results suggest that cognitive control and feature processing cortical regions interact to implement feedback-congruent adaptations beneficial to learning.

scholarly journals Trading off the cost of conflict against expected rewards

2018 ◽  
Author(s):  
Nura Sidarus ◽  
Stefano Palminteri ◽  
Valérian Chambon
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Cognitive Control ◽  
Reversal Learning ◽  
Computational Models ◽  
Learning Task ◽  
Learning Rates ◽  
Different Types ◽  
Irrelevant Distractors ◽  
The Cost

AbstractValue-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Yet, it remains unclear whether conflict is also perceived as a cost in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of being in conflict with an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that conflict was avoided when evidence for either action alternative was weak, demonstrating that the cost of conflict was traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one’s actions and external distractors. Our results show that the subjective cost of conflict factors into value-based decision-making, and highlights that different types of conflict may have different effects on learning about action outcomes.

scholarly journals Intermittent Absence of Control during Reinforcement Learning Interferes with Pavlovian Bias in Action Selection

2020 ◽  
Vol 32 (4) ◽  
pp. 646-663 ◽  
Author(s):  
Gábor Csifcsák ◽  
Eirik Melsæter ◽  
Matthias Mittner
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Cognitive Control ◽  
Learned Helplessness ◽  
Well Being ◽  
Learning Task ◽  
Healthy Adults ◽  
Task Demands ◽  
Behavioral Manipulation ◽  
Different Response

The ability to control the occurrence of rewarding and punishing events is crucial for our well-being. Two ways to optimize performance are to follow heuristics like Pavlovian biases to approach reward and avoid loss or to rely more on slowly accumulated stimulus–action associations. Although reduced control over outcomes has been linked to suboptimal decision-making in clinical conditions associated with learned helplessness, it is unclear how uncontrollability of the environment is related to the arbitration between different response strategies. This study directly tested whether a behavioral manipulation designed to induce learned helplessness in healthy adults (intermittent loss of control over feedback in a reinforcement learning task; “yoking”) would modulate the magnitude of Pavlovian bias and the neurophysiological signature of cognitive control (frontal midline theta power) in healthy adults. Using statistical analysis and computational modeling of behavioral data and electroencephalographic signals, we found stronger Pavlovian influences and alterations in frontal theta activity in the yoked group. However, these effects were not accompanied by reduced performance in experimental blocks with regained control, indicating that our behavioral manipulation was not potent enough for inducing helplessness and impaired coping ability with task demands. We conclude that the level of contingency between instrumental choices and rewards/punishments modulates Pavlovian bias during value-based decision-making, probably via interfering with the implementation of cognitive control. These findings might have implications for understanding the mechanisms underlying helplessness in various psychiatric conditions.

scholarly journals Intermittent absence of control during reinforcement learning interferes with Pavlovian bias in action selection

2019 ◽  
Author(s):  
Gábor Csifcsák ◽  
Eirik Melsæter ◽  
Matthias Mittner
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Cognitive Control ◽  
Learned Helplessness ◽  
Well Being ◽  
Learning Task ◽  
Healthy Adults ◽  
Behavioral Manipulation ◽  
Frontal Midline Theta ◽  
Different Response

The ability to control the occurrence of rewarding and punishing events is crucial for our well-being. Two ways to optimize performance are to follow heuristics like Pavlovian biases to approach reward and avoid loss, or to rely more on slowly accumulated stimulus-action associations. Although reduced control over outcomes has been linked to suboptimal decision-making in clinical conditions associated with learned helplessness, it is unclear how uncontrollability of the environment is related to the arbitration between different response strategies.This study directly tested whether a behavioral manipulation designed to induce learned helplessness in healthy adults (intermittent loss of control over feedback in a reinforcement learning task; “yoking”) would modulate the magnitude of Pavlovian bias and the neurophysiological signature of cognitive control (frontal midline theta power) in healthy adults. Using statistical analysis and computational modeling of behavioral data and electroencephalographic signals, we found stronger Pavlovian influences and alterations in frontal theta activity in the yoked group. However, these effects were not accompanied by reduced performance in experimental blocks with regained control, indicating that our behavioral manipulation was not potent enough for inducing helplessness and impaired coping ability with task demands.We conclude that the level of contingency between instrumental choices and rewards/punishments modulates Pavlovian bias during value-based decision-making, probably via interfering with the implementation of cognitive control. These findings might have implications for understanding the mechanisms underlying helplessness in various psychiatric conditions.

Neuroevolution approach for reinforcement learning task applied to control system of a cart pole system

2019 ◽  
Author(s):  
Marco Boaretto ◽  
Gabriel Chaves Becchi ◽  
Luiza Scapinello Aquino ◽  
Aderson Cleber Pifer ◽  
Helon Vicente Hultmann Ayala ◽  
...  
Keyword(s):  
Control System ◽  
Reinforcement Learning ◽  
Learning Task ◽  
Pole System

Model-Based and Model-Free Social Cognition

2019 ◽  
Author(s):  
Leor M Hackel ◽  
Jeffrey Jordan Berg ◽  
Björn Lindström ◽  
David Amodio
Keyword(s):  
Reinforcement Learning ◽  
Social Cognition ◽  
Learning Strategies ◽  
Memory Systems ◽  
Learning Task ◽  
Financial Advisors ◽  
Model Based ◽  
Model Free ◽  
Systems Model ◽  
Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

Genetic Variation of Serotonin Function and Cognitive Control

2007 ◽  
Vol 19 (12) ◽  
pp. 1923-1931 ◽  
Author(s):  
Alexander Strobel ◽  
Gesine Dreisbach ◽  
Johannes Müller ◽  
Thomas Goschke ◽  
Burkhard Brocke ◽  
...  
Keyword(s):  
Genetic Variation ◽  
Cognitive Control ◽  
Personality Traits ◽  
Anxiety And Depression ◽  
Continuous Performance ◽  
Continuous Performance Task ◽  
Performance Task ◽  
The Impact ◽  
Related Personality

Although it is widely accepted that serotonin plays a pivotal role in the modulation of anxiety- and depression-related personality traits as well as in the pathogenesis of anxiety disorders and depression, the role of serotonin in cognition is less clear. In the present study, we investigated the involvement of serotonin in cognitive behaviors by examining the impact of genetic variation in key regulators of serotonergic neurotransmission on behavioral measures in a cognitive control task. Eighty-five healthy participants performed a cued continuous performance task (the AX Continuous Performance Task [AXCPT]) and were genotyped for polymorphisms in the transcriptional control regions of the tryptophan hydroxylase 2 gene (TPH2 G-703T; rs4570625) and the serotonin transporter gene (5-HTTLPR). The core result was that individuals lacking the rare TPH2 T allele were not faster than T allele carriers, but committed fewer errors and were less variable in responding. These findings parallel those of a recent study where an enhancement of executive control in individuals without the rare TPH2 T/T genotype was observed. Together with recent evidence that individuals without the T allele exhibit higher scores in anxiety- and depression-related personality traits, our results underscore the role of the TPH2 G-703T polymorphism in the modulation of behavior and raise the intriguing possibility that genetic variants associated with higher negative emotionality may have beneficial effects on some cognitive functions.

Sign in / Sign up

Export Citation Format

Share Document