scholarly journals S146. ASSOCIATION OF IMPAIRED MODEL-FREE DECISION-MAKING WITH ABERRANT STRIATAL DOPAMINE, BRAIN ACTIVATION, AND COGNITIVE DIFFICULTIES IN PATIENTS WITH SCHIZOPHRENIA DURING PSYCHOTIC REMISSION

2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S91-S92
Author(s):  
Felix Brandl ◽  
Mihai Avram ◽  
Jorge Cabello ◽  
Mona Mustafa ◽  
Claudia Leucht ◽  
...  
Keyword(s):  
Decision Making ◽  
Cognitive Impairments ◽  
Dorsal Striatum ◽  
Striatal Dopamine ◽  
Dopamine Synthesis ◽  
Cognitive Difficulties ◽  
Model Based ◽  
Model Free ◽  
Trail Making ◽  
Task Activation

Abstract Background Human decision-making ranges between the extremes of automatic and fast model-free behavior (i.e., relying only on previous outcomes) and more flexible, but computationally demanding model-based behavior (i.e., implementing cognitive models). Model-based/model-free decision-making can be investigated using sequential decision tasks and has been shown to be associated with presynaptic striatal dopamine synthesis. During phases of psychotic remission in schizophrenia, dopamine synthesis in the dorsal striatum is reduced. We hypothesized that particularly model-free decision-making is impaired in schizophrenia during psychotic remission and is associated with (i) abnormal dopamine synthesis in dorsal striatum, (ii) aberrant task-activation in dorsal striatum, and (iii) cognitive difficulties in patients (e.g., reduced speed). Methods 26 patients with chronic schizophrenia, currently in psychotic remission, and 22 healthy controls (matched by age and gender) were enrolled in the study. Model-based/model-free decision-making was evaluated with a two-stage Markov decision task, followed by computational modeling of subjects’ learning behavior. Presynaptic dopamine synthesis was assessed by 18F-DOPA positron emission tomography and subsequent graphical Patlak analysis. Task-activation was measured by functional magnetic resonance imaging. Cognitive impairments were quantified by Trail-Making-Test A (among others). Associations between decision-making parameters, dopamine synthesis, task-activation, and cognitive impairments were tested by correlation analyses. Results Patients with schizophrenia showed selectively impaired model-free decision-making. 18F-DOPA uptake (i.e., presynaptic dopamine synthesis capacity) in the dorsal striatum was decreased in patients. Impaired model-free decision-making in patients correlated with (i) decreased dopamine synthesis in dorsal striatum, (ii) abnormal task-activation in dorsal striatum, and (iii) lower speed in Trail-Making-Test A. Discussion Results demonstrate an association of reduced dorsal striatal dopamine synthesis and brain activity with impaired model-free decision-making in schizophrenia, which potentially contributes to cognitive difficulties.

scholarly journals S145. CORTICO-THALAMIC DYSCONNECTIVITY LINKS WITH ABERRANT STRIATAL DOPAMINE IN SCHIZOPHRENIA A SIMULTANEOUS 18F-DOPA-PET/RESTING-STATE FMRI STUDY

2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S91-S91
Author(s):  
Mihai Avram ◽  
Felix Brandl ◽  
Franziska Knolle ◽  
Jorge Cabello ◽  
Claudia Leucht ◽  
...  
Keyword(s):  
Resting State ◽  
Dorsal Striatum ◽  
Correlation Coefficients ◽  
Striatal Dopamine ◽  
Dopamine Synthesis ◽  
Salience Network ◽  
Thalamic Nuclei ◽  
Cortical Networks ◽  
Sensorimotor Network ◽  
System Changes

Abstract Background In schizophrenia, among the most consistent brain changes are both aberrant dopamine function in the dorsal striatum and aberrant intrinsic functional connectivity (iFC) between distinct cortical networks and thalamic nuclei; however, it is unknown whether these changes are pathophysiologically related. Such a relationship is expected because cortico-thalamic-connectivity is modulated by striatal dopamine within topographically distinct, parallel but interacting cortico-basal-ganglia-thalamic circuits. We hypothesized: (1) Within-circuits, aberrant striatal dopamine contributes to aberrant cortico-thalamic-iFC, specifically, associative-striatum dopamine contributes to salience-network-thalamic-iFC, and sensorimotor-striatum dopamine to auditory-sensorimotor-network-thalamic-iFC. (2) Due to between-circuits interactions following an anterior-to-posterior gradient, salience-network-centered-system changes contribute to auditory-sensorimotor-network-centered-system changes. Methods To test these hypotheses, 19 patients with schizophrenia during symptomatic remission of positive symptoms and 19 age- and sex-comparable controls underwent simultaneous fluorodihydroxyphenyl-L-alanine positron emission tomography (18F-DOPA-PET) and resting-state functional magnetic resonance imaging (rs-fMRI). The influx constant kicer based on 18F-DOPA-PET was used to measure dopamine synthesis capacity (DSC), indicating striatal dopamine function; correlation coefficients between rs-fMRI time-series of cortical networks and thalamic regions-of-interest were used to measure iFC. Results In the salience-network(SAL)-centered-system, patients had reduced associative-striatum-DSC, which correlated positively with SAL-mediodorsal-thalamus-iFC and mediated the reduction of SAL-thalamic-iFC in patients. In the auditory-sensorimotor-network(ASM)-centered-system, patients had reduced sensorimotor-striatum-DSC, which correlated positively with ASM-ventrolateral-thalamus-iFC, but did not mediate increased ASM-thalamic-iFC in patients. Finally, aberrant DSC and iFC of the SAL-centered-system mediated corresponding changes in the ASM-centered-system. Discussion Results demonstrate that cortico-thalamic-dysconnectivity links with aberrant striatal dopamine in schizophrenia - in a topographically distinct way, with an anterior-to-posterior gradient, and primary changes in the SAL-centered system.

scholarly journals Arbitration between Action Strategies in Obsessive-Compulsive Disorder

2015 ◽  
Vol 22 (2) ◽  
pp. 188-198 ◽  
Author(s):  
Patricia Gruner ◽  
Alan Anticevic ◽  
Daeyeol Lee ◽  
Christopher Pittenger
Keyword(s):  
Decision Making ◽  
Obsessive Compulsive Disorder ◽  
Behavioral Outcomes ◽  
Cognitive Remediation ◽  
Obsessive Compulsive ◽  
Compulsive Disorder ◽  
Model Based ◽  
Model Free ◽  
Action Strategies ◽  
Evaluative Processes

Decision making in a complex world, characterized both by predictable regularities and by frequent departures from the norm, requires dynamic switching between rapid habit-like, automatic processes and slower, more flexible evaluative processes. These strategies, formalized as “model-free” and “model-based” reinforcement learning algorithms, respectively, can lead to divergent behavioral outcomes, requiring a mechanism to arbitrate between them in a context-appropriate manner. Recent data suggest that individuals with obsessive-compulsive disorder (OCD) rely excessively on inflexible habit-like decision making during reinforcement-driven learning. We propose that inflexible reliance on habit in OCD may reflect a functional weakness in the mechanism for context-appropriate dynamic arbitration between model-free and model-based decision making. Support for this hypothesis derives from emerging functional imaging findings. A deficit in arbitration in OCD may help reconcile evidence for excessive reliance on habit in rewarded learning tasks with an older literature suggesting inappropriate recruitment of circuitry associated with model-based decision making in unreinforced procedural learning. The hypothesized deficit and corresponding circuitry may be a particularly fruitful target for interventions, including cognitive remediation.

scholarly journals Higher Meta-cognitive Ability Predicts Less Reliance on Over Confident Habitual Learning System

2019 ◽  
Author(s):  
Sara Ershadmanesh ◽  
Mostafa Miandari ◽  
Abdol-hossein Vahabie ◽  
Majid Nili Ahmadabadi
Keyword(s):  
Decision Making ◽  
Cognitive Ability ◽  
Learning Strategies ◽  
Learning Style ◽  
Learning Systems ◽  
Learning System ◽  
Variable Environment ◽  
Free System ◽  
Model Based ◽  
Model Free

AbstractMany studies on human and animals have provided evidence for the contribution of goal-directed and habitual valuation systems in learning and decision-making. These two systems can be modeled using model-based (MB) and model-free (MF) algorithms in Reinforcement Learning (RL) framework. Here, we study the link between the contribution of these two learning systems to behavior and meta-cognitive capabilities. Using computational modeling we showed that in a highly variable environment, where both learning strategies have chance level performances, model-free learning predicts higher confidence in decisions compared to model-based strategy. Our experimental results showed that the subjects’ meta-cognitive ability is negatively correlated with the contribution of model-free system to their behavior while having no correlation with the contribution of model-based system. Over-confidence of the model-free system justifies this counter-intuitive result. This is a new explanation for individual difference in learning style.

scholarly journals Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making

2015 ◽  
Vol 112 (5) ◽  
pp. 1595-1600 ◽  
Author(s):  
Lorenz Deserno ◽  
Quentin J. M. Huys ◽  
Rebecca Boehme ◽  
Ralph Buchert ◽  
Hans-Jochen Heinze ◽  
...  
Keyword(s):  
Ventral Striatum ◽  
Behavioral Control ◽  
Striatal Dopamine ◽  
Quantitative Relation ◽  
Sequential Decision ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Neuropsychiatric Diseases ◽  
Neural Signatures

Dual system theories suggest that behavioral control is parsed between a deliberative “model-based” and a more reflexive “model-free” system. A balance of control exerted by these systems is thought to be related to dopamine neurotransmission. However, in the absence of direct measures of human dopamine, it remains unknown whether this reflects a quantitative relation with dopamine either in the striatum or other brain areas. Using a sequential decision task performed during functional magnetic resonance imaging, combined with striatal measures of dopamine using [18F]DOPA positron emission tomography, we show that higher presynaptic ventral striatal dopamine levels were associated with a behavioral bias toward more model-based control. Higher presynaptic dopamine in ventral striatum was associated with greater coding of model-based signatures in lateral prefrontal cortex and diminished coding of model-free prediction errors in ventral striatum. Thus, interindividual variability in ventral striatal presynaptic dopamine reflects a balance in the behavioral expression and the neural signatures of model-free and model-based control. Our data provide a novel perspective on how alterations in presynaptic dopamine levels might be accompanied by a disruption of behavioral control as observed in aging or neuropsychiatric diseases such as schizophrenia and addiction.

scholarly journals Hunger improves reinforcement-driven but not planned action

2021 ◽  
Author(s):  
Maaike M.H. van Swieten ◽  
Rafal Bogacz ◽  
Sanjay G. Manohar
Keyword(s):  
Decision Making ◽  
Sequential Decision ◽  
Context Sensitive ◽  
Decision Systems ◽  
Model Based ◽  
Model Free ◽  
Prevalence Of Obesity ◽  
Model Free Control ◽  
Decision Mode

AbstractHuman decisions can be reflexive or planned, being governed respectively by model-free and model-based learning systems. These two systems might differ in their responsiveness to our needs. Hunger drives us to specifically seek food rewards, but here we ask whether it might have more general effects on these two decision systems. On one hand, the model-based system is often considered flexible and context-sensitive, and might therefore be modulated by metabolic needs. On the other hand, the model-free system’s primitive reinforcement mechanisms may have closer ties to biological drives. Here, we tested participants on a well-established two-stage sequential decision-making task that dissociates the contribution of model-based and model-free control. Hunger enhanced overall performance by increasing model-free control, without affecting model-based control. These results demonstrate a generalised effect of hunger on decision-making that enhances reliance on primitive reinforcement learning, which in some situations translates into adaptive benefits.Significance statementThe prevalence of obesity and eating disorder is steadily increasing. To counteract problems related to eating, people need to make rational decisions. However, appetite may switch us to a different decision mode, making it harder to achieve long-term goals. Here we show that planned and reinforcement-driven actions are differentially sensitive to hunger. Hunger specifically affected reinforcement-driven actions, and did not affect the planning of actions. Our data shows that people behave differently when they are hungry. We also provide a computational model of how the behavioural changes might arise.

scholarly journals Model-based decision making and model-free learning

2020 ◽  
Vol 30 (15) ◽  
pp. R860-R865 ◽  
Author(s):  
Nicole Drummond ◽  
Yael Niv
Keyword(s):  
Decision Making ◽  
Model Based ◽  
Model Free

scholarly journals Model based planners reflect on their model-free propensities

2021 ◽  
Vol 17 (1) ◽  
pp. e1008552
Author(s):  
Rani Moran ◽  
Mehdi Keramati ◽  
Raymond J. Dolan
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Drug Abuse ◽  
Learning Theory ◽  
Planning Model ◽  
Present Evidence ◽  
Model Based ◽  
Model Free ◽  
Short And Long Term

Dual-reinforcement learning theory proposes behaviour is under the tutelage of a retrospective, value-caching, model-free (MF) system and a prospective-planning, model-based (MB), system. This architecture raises a question as to the degree to which, when devising a plan, a MB controller takes account of influences from its MF counterpart. We present evidence that such a sophisticated self-reflective MB planner incorporates an anticipation of the influences its own MF-proclivities exerts on the execution of its planned future actions. Using a novel bandit task, wherein subjects were periodically allowed to design their environment, we show that reward-assignments were constructed in a manner consistent with a MB system taking account of its MF propensities. Thus, in the task participants assigned higher rewards to bandits that were momentarily associated with stronger MF tendencies. Our findings have implications for a range of decision making domains that includes drug abuse, pre-commitment, and the tension between short and long-term decision horizons in economics.

scholarly journals Metacontrol of decision-making strategies in human aging

2019 ◽  
Author(s):  
Florian Bolenz ◽  
Wouter Kool ◽  
Andrea M.F. Reiter ◽  
Ben Eppinger
Keyword(s):  
Older Adults ◽  
Decision Making ◽  
Cognitive Abilities ◽  
Reduced Model ◽  
Task Structure ◽  
Costs And Benefits ◽  
Sequential Decision ◽  
Model Based Control ◽  
Model Based ◽  
Model Free

When making decisions, humans employ different strategies which are commonly formalized as model-free and model-based reinforcement learning. While previous research has reported reduced model-based control with aging, it remains unclear whether this is due to limited cognitive capacities or a reduced willingness to engage in an effortful strategy. Moreover, it is not clear how aging affects the metacontrol of decision making, i.e. the dynamic adaptation of decision-making strategies to varying situational demands. To this end, we tested younger and older adults in a sequential decision-making task that dissociates model-free and model-based control. In contrast to previous research, in this study we applied a task in which model-based control led to higher payoffs in terms of monetary reward. Moreover, we manipulated the costs and benefits associated with model-based control by varying reward magnitude as well as the stability of the task structure. Compared to younger adults, older adults showed reduced reliance on model-based decision making and less adaptation of decision-making strategies to varying costs and benefits of model-based control. Our findings suggest that aging affects the dynamic metacontrol of decision-making strategies and that reduced model-based control in older adults is due to limited cognitive abilities to represent the structure of the task.

scholarly journals Metacontrol of decision-making strategies in human aging

eLife ◽  
2019 ◽  
Vol 8 ◽  
Author(s):  
Florian Bolenz ◽  
Wouter Kool ◽  
Andrea MF Reiter ◽  
Ben Eppinger
Keyword(s):  
Older Adults ◽  
Decision Making ◽  
Cognitive Abilities ◽  
Cross Sectional Study ◽  
Reduced Model ◽  
Task Structure ◽  
Sequential Decision ◽  
Cross Sectional ◽  
Model Based ◽  
Model Free

Humans employ different strategies when making decisions. Previous research has reported reduced reliance on model-based strategies with aging, but it remains unclear whether this is due to cognitive or motivational factors. Moreover, it is not clear how aging affects the metacontrol of decision making, that is the dynamic adaptation of decision-making strategies to varying situational demands. In this cross-sectional study, we tested younger and older adults in a sequential decision-making task that dissociates model-free and model-based strategies. In contrast to previous research, model-based strategies led to higher payoffs. Moreover, we manipulated the costs and benefits of model-based strategies by varying reward magnitude and the stability of the task structure. Compared to younger adults, older adults showed reduced model-based decision making and less adaptation of decision-making strategies. Our findings suggest that aging affects the metacontrol of decision-making strategies and that reduced model-based strategies in older adults are due to limited cognitive abilities.

Sign in / Sign up

Export Citation Format

Share Document