A new computational account of cognitive control over reinforcement-based decision-making: Modeling of a probabilistic learning task

2015 ◽  
Vol 71 ◽  
pp. 112-123 ◽  
Author(s):  
Sareh Zendehrouh
Keyword(s):  
Decision Making ◽  
Cognitive Control ◽  
Learning Task ◽  
Probabilistic Learning

scholarly journals Trading off the cost of conflict against expected rewards

2018 ◽  
Author(s):  
Nura Sidarus ◽  
Stefano Palminteri ◽  
Valérian Chambon
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Cognitive Control ◽  
Reversal Learning ◽  
Computational Models ◽  
Learning Task ◽  
Learning Rates ◽  
Different Types ◽  
Irrelevant Distractors ◽  
The Cost

AbstractValue-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Yet, it remains unclear whether conflict is also perceived as a cost in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of being in conflict with an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that conflict was avoided when evidence for either action alternative was weak, demonstrating that the cost of conflict was traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one’s actions and external distractors. Our results show that the subjective cost of conflict factors into value-based decision-making, and highlights that different types of conflict may have different effects on learning about action outcomes.

scholarly journals Intermittent Absence of Control during Reinforcement Learning Interferes with Pavlovian Bias in Action Selection

2020 ◽  
Vol 32 (4) ◽  
pp. 646-663 ◽  
Author(s):  
Gábor Csifcsák ◽  
Eirik Melsæter ◽  
Matthias Mittner
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Cognitive Control ◽  
Learned Helplessness ◽  
Well Being ◽  
Learning Task ◽  
Healthy Adults ◽  
Task Demands ◽  
Behavioral Manipulation ◽  
Different Response

The ability to control the occurrence of rewarding and punishing events is crucial for our well-being. Two ways to optimize performance are to follow heuristics like Pavlovian biases to approach reward and avoid loss or to rely more on slowly accumulated stimulus–action associations. Although reduced control over outcomes has been linked to suboptimal decision-making in clinical conditions associated with learned helplessness, it is unclear how uncontrollability of the environment is related to the arbitration between different response strategies. This study directly tested whether a behavioral manipulation designed to induce learned helplessness in healthy adults (intermittent loss of control over feedback in a reinforcement learning task; “yoking”) would modulate the magnitude of Pavlovian bias and the neurophysiological signature of cognitive control (frontal midline theta power) in healthy adults. Using statistical analysis and computational modeling of behavioral data and electroencephalographic signals, we found stronger Pavlovian influences and alterations in frontal theta activity in the yoked group. However, these effects were not accompanied by reduced performance in experimental blocks with regained control, indicating that our behavioral manipulation was not potent enough for inducing helplessness and impaired coping ability with task demands. We conclude that the level of contingency between instrumental choices and rewards/punishments modulates Pavlovian bias during value-based decision-making, probably via interfering with the implementation of cognitive control. These findings might have implications for understanding the mechanisms underlying helplessness in various psychiatric conditions.

scholarly journals Intermittent absence of control during reinforcement learning interferes with Pavlovian bias in action selection

2019 ◽  
Author(s):  
Gábor Csifcsák ◽  
Eirik Melsæter ◽  
Matthias Mittner
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Cognitive Control ◽  
Learned Helplessness ◽  
Well Being ◽  
Learning Task ◽  
Healthy Adults ◽  
Behavioral Manipulation ◽  
Frontal Midline Theta ◽  
Different Response

The ability to control the occurrence of rewarding and punishing events is crucial for our well-being. Two ways to optimize performance are to follow heuristics like Pavlovian biases to approach reward and avoid loss, or to rely more on slowly accumulated stimulus-action associations. Although reduced control over outcomes has been linked to suboptimal decision-making in clinical conditions associated with learned helplessness, it is unclear how uncontrollability of the environment is related to the arbitration between different response strategies.This study directly tested whether a behavioral manipulation designed to induce learned helplessness in healthy adults (intermittent loss of control over feedback in a reinforcement learning task; “yoking”) would modulate the magnitude of Pavlovian bias and the neurophysiological signature of cognitive control (frontal midline theta power) in healthy adults. Using statistical analysis and computational modeling of behavioral data and electroencephalographic signals, we found stronger Pavlovian influences and alterations in frontal theta activity in the yoked group. However, these effects were not accompanied by reduced performance in experimental blocks with regained control, indicating that our behavioral manipulation was not potent enough for inducing helplessness and impaired coping ability with task demands.We conclude that the level of contingency between instrumental choices and rewards/punishments modulates Pavlovian bias during value-based decision-making, probably via interfering with the implementation of cognitive control. These findings might have implications for understanding the mechanisms underlying helplessness in various psychiatric conditions.

A Bird in the Hand Isn’t Good for Long

2018 ◽  
Vol 65 (1) ◽  
pp. 23-31 ◽  
Author(s):  
Stefan Scherbaum ◽  
Simon Frisch ◽  
Maja Dshemuchadse
Keyword(s):  
Decision Making ◽  
Cognitive Control ◽  
The Other ◽  
Decision Task ◽  
Short Term ◽  
Additional Time ◽  
Control Processes ◽  
Folk Wisdom ◽  
Study Participants ◽  
Insight Into

Abstract. Folk wisdom tells us that additional time to make a decision helps us to refrain from the first impulse to take the bird in the hand. However, the question why the time to decide plays an important role is still unanswered. Here we distinguish two explanations, one based on a bias in value accumulation that has to be overcome with time, the other based on cognitive control processes that need time to set in. In an intertemporal decision task, we use mouse tracking to study participants’ responses to options’ values and delays which were presented sequentially. We find that the information about options’ delays does indeed lead to an immediate bias that is controlled afterwards, matching the prediction of control processes needed to counter initial impulses. Hence, by using a dynamic measure, we provide insight into the processes underlying short-term oriented choices in intertemporal decision making.

Age-Related Differences in Motivational Integration and Cognitive Control

2019 ◽  
Author(s):  
Debbie Marianne Yee ◽  
Sarah L Adams ◽  
Asad Beck ◽  
Todd Samuel Braver
Keyword(s):  
Older Adults ◽  
Decision Making ◽  
Cognitive Control ◽  
Task Performance ◽  
Cognitive Task ◽  
Monetary Reward ◽  
Potential Candidate ◽  
Younger Adults ◽  
Age Related ◽  
Cognitive Task Performance

Motivational incentives play an influential role in value-based decision-making and cognitive control. A compelling hypothesis in the literature suggests that the brain integrates the motivational value of diverse incentives (e.g., motivational integration) into a common currency value signal that influences decision-making and behavior. To investigate whether motivational integration processes change during healthy aging, we tested older (N=44) and younger (N=54) adults in an innovative incentive integration task paradigm that establishes dissociable and additive effects of liquid (e.g., juice, neutral, saltwater) and monetary incentives on cognitive task performance. The results reveal that motivational incentives improve cognitive task performance in both older and younger adults, providing novel evidence demonstrating that age-related cognitive control deficits can be ameliorated with sufficient incentive motivation. Additional analyses revealed clear age-related differences in motivational integration. Younger adult task performance was modulated by both monetary and liquid incentives, whereas monetary reward effects were more gradual in older adults and more strongly impacted by trial-by-trial performance feedback. A surprising discovery was that older adults shifted attention from liquid valence toward monetary reward throughout task performance, but younger adults shifted attention from monetary reward toward integrating both monetary reward and liquid valence by the end of the task, suggesting differential strategic utilization of incentives. Together these data suggest that older adults may have impairments in incentive integration, and employ different motivational strategies to improve cognitive task performance. The findings suggest potential candidate neural mechanisms that may serve as the locus of age-related change, providing targets for future cognitive neuroscience investigations.

scholarly journals Cognitive Control of Working Memory: A Model-Based Approach

2021 ◽  
Vol 11 (6) ◽  
pp. 721
Author(s):  
Russell J. Boag ◽  
Niek Stevenson ◽  
Roel van Dooren ◽  
Anne C. Trutti ◽  
Zsuzsika Sjoerds ◽  
...  
Keyword(s):  
Decision Making ◽  
Working Memory ◽  
Cognitive Control ◽  
Relevant Information ◽  
Decision Time ◽  
Control Processes ◽  
Prepotent Response ◽  
New Information ◽  
Empirical Performance ◽  
Flow Of Information

Working memory (WM)-based decision making depends on a number of cognitive control processes that control the flow of information into and out of WM and ensure that only relevant information is held active in WM’s limited-capacity store. Although necessary for successful decision making, recent work has shown that these control processes impose performance costs on both the speed and accuracy of WM-based decisions. Using the reference-back task as a benchmark measure of WM control, we conducted evidence accumulation modeling to test several competing explanations for six benchmark empirical performance costs. Costs were driven by a combination of processes, running outside of the decision stage (longer non-decision time) and showing the inhibition of the prepotent response (lower drift rates) in trials requiring WM control. Individuals also set more cautious response thresholds when expecting to update WM with new information versus maintain existing information. We discuss the promise of this approach for understanding cognitive control in WM-based decision making.

Sign in / Sign up

Export Citation Format

Share Document