scholarly journals Model-free decision making is prioritized when learning to avoid harming others

2020 ◽  
Vol 117 (44) ◽  
pp. 27719-27730 ◽  
Author(s):  
Patricia L. Lockwood ◽  
Miriam C. Klein-Flügge ◽  
Ayat Abdurahman ◽  
Molly J. Crockett

Moral behavior requires learning how our actions help or harm others. Theoretical accounts of learning propose a key division between “model-free” algorithms that cache outcome values in actions and “model-based” algorithms that map actions to outcomes. Here, we tested the engagement of these mechanisms and their neural basis as participants learned to avoid painful electric shocks for themselves and a stranger. We found that model-free decision making was prioritized when learning to avoid harming others compared to oneself. Model-free prediction errors for others relative to self were tracked in the thalamus/caudate. At the time of choice, neural activity consistent with model-free moral learning was observed in subgenual anterior cingulate cortex (sgACC), and switching after harming others was associated with stronger connectivity between sgACC and dorsolateral prefrontal cortex. Finally, model-free moral learning varied with individual differences in moral judgment. Our findings suggest moral learning favors efficiency over flexibility and is underpinned by specific neural mechanisms.

2019 ◽  
Author(s):  
Patricia L. Lockwood ◽  
Miriam Klein-Flügge ◽  
Ayat Abdurahman ◽  
Molly J. Crockett

AbstractMoral behaviour requires learning how our actions help or harm others. Theoretical accounts of learning propose a key division between ‘model-free’ algorithms that efficiently cache outcome values in actions and ‘model-based’ algorithms that prospectively map actions to outcomes, a distinction that may be critical for moral learning. Here, we tested the engagement of these learning mechanisms and their neural basis as participants learned to avoid painful electric shocks for themselves and a stranger. We found that model-free learning was prioritized when avoiding harm to others compared to oneself. Model-free prediction errors for others relative to self were tracked in the thalamus/caudate at the time of the outcome. At the time of choice, a signature of model-free moral learning was associated with responses in subgenual anterior cingulate cortex (sgACC), and resisting this model-free influence was predicted by stronger connectivity between sgACC and dorsolateral prefrontal cortex. Finally, multiple behavioural and neural correlates of model-free moral learning varied with individual differences in moral judgment. Our findings suggest moral learning favours efficiency over flexibility and is underpinned by specific neural mechanisms.


2018 ◽  
Author(s):  
Xiaoxue Gao ◽  
Hongbo Yu ◽  
Ignacio Saez ◽  
Philip R. Blue ◽  
Lusha Zhu ◽  
...  

AbstractHumans are capable of integrating social contextual information into decision-making processes to adjust their attitudes towards inequity. This context-dependency emerges both when individual is better off (i.e. advantageous inequity) and worse off (i.e. disadvantageous inequity) than others. It is not clear however, whether the context-dependent processing of advantageous and disadvantageous inequity rely on dissociable or shared neural mechanisms. Here, by combining an interpersonal interactive game that gave rise to interpersonal guilt and different versions of the dictator games that enabled us to characterize individual weights on aversion to advantageous and disadvantageous inequity, we investigated the neural mechanisms underlying the two forms of inequity aversion in the interpersonal guilt context. In each round, participants played a dot-estimation task with an anonymous co-player. The co-players received pain stimulation with 50% probability when anyone responded incorrectly. At the end of each round, participants completed a dictator game, which determined payoffs of him/herself and the co-player. Both computational model-based and model-free analyses demonstrated that when inflicting pain upon co-players (i.e., the guilt context), participants cared more about advantageous inequity and became less sensitive to disadvantageous inequity, compared with other social contexts. The contextual effects on two forms of inequity aversion are uncorrelated with each other at the behavioral level. Neuroimaging results revealed that the context-dependent representation of inequity aversion exhibited a spatial gradient in activity within the insula, with anterior parts predominantly involved in the aversion to advantageous inequity and posterior parts predominantly involved in the aversion to disadvantageous inequity. The dissociable mechanisms underlying the two forms of inequity aversion are further supported by the involvement of right dorsolateral prefrontal cortex and dorsomedial prefrontal cortex in advantageous inequity processing, and the involvement of right amygdala and dorsal anterior cingulate cortex in disadvantageous inequity processing. These results extended our understanding of decision-making processes involving inequity and the social functions of inequity aversion.


2018 ◽  
Vol 115 (22) ◽  
pp. E5233-E5242 ◽  
Author(s):  
Amanda R. Arulpragasam ◽  
Jessica A. Cooper ◽  
Makiah R. Nuutinen ◽  
Michael T. Treadway

We are presented with choices each day about how to invest our effort to achieve our goals. Critically, these decisions must frequently be made under conditions of incomplete information, where either the effort required or possible reward to be gained is uncertain. Such choices therefore require the development of potential value estimates to guide effortful goal-directed behavior. To date, however, the neural mechanisms for this expectation process are unknown. Here, we used computational fMRI during an effort-based decision-making task where trial-wise information about effort costs and reward magnitudes was presented separately over time, thereby allowing us to model distinct effort/reward computations as choice-relevant information unfolded. We found that ventromedial prefrontal cortex (vmPFC) encoded expected subjective value. Further, activity in dorsal anterior cingulate (dACC) and anterior insula (aI) reflected both effort discounting as well as a subjective value prediction error signal derived from trial history. While prior studies have identified these regions as being involved in effort-based decision making, these data demonstrate their specific role in the formation and maintenance of subjective value estimates as relevant information becomes available.


2019 ◽  
Vol 2019 ◽  
pp. 1-10 ◽  
Author(s):  
Maya G. Mosner ◽  
R. Edward McLaurin ◽  
Jessica L. Kinard ◽  
Shabnam Hakimi ◽  
Jacob Parelman ◽  
...  

Few studies have explored neural mechanisms of reward learning in ASD despite evidence of behavioral impairments of predictive abilities in ASD. To investigate the neural correlates of reward prediction errors in ASD, 16 adults with ASD and 14 typically developing controls performed a prediction error task during fMRI scanning. Results revealed greater activation in the ASD group in the left paracingulate gyrus during signed prediction errors and the left insula and right frontal pole during thresholded unsigned prediction errors. Findings support atypical neural processing of reward prediction errors in ASD in frontostriatal regions critical for prediction coding and reward learning. Results provide a neural basis for impairments in reward learning that may contribute to traits common in ASD (e.g., intolerance of unpredictability).


2017 ◽  
Vol 47 (7) ◽  
pp. 1246-1258 ◽  
Author(s):  
T. U. Hauser ◽  
R. Iannaccone ◽  
R. J. Dolan ◽  
J. Ball ◽  
J. Hättenschwiler ◽  
...  

BackgroundObsessive–compulsive disorder (OCD) has been linked to functional abnormalities in fronto-striatal networks as well as impairments in decision making and learning. Little is known about the neurocognitive mechanisms causing these decision-making and learning deficits in OCD, and how they relate to dysfunction in fronto-striatal networks.MethodWe investigated neural mechanisms of decision making in OCD patients, including early and late onset of disorder, in terms of reward prediction errors (RPEs) using functional magnetic resonance imaging. RPEs index a mismatch between expected and received outcomes, encoded by the dopaminergic system, and are known to drive learning and decision making in humans and animals. We used reinforcement learning models and RPE signals to infer the learning mechanisms and to compare behavioural parameters and neural RPE responses of the OCD patients with those of healthy matched controls.ResultsPatients with OCD showed significantly increased RPE responses in the anterior cingulate cortex (ACC) and the putamen compared with controls. OCD patients also had a significantly lower perseveration parameter than controls.ConclusionsEnhanced RPE signals in the ACC and putamen extend previous findings of fronto-striatal deficits in OCD. These abnormally strong RPEs suggest a hyper-responsive learning network in patients with OCD, which might explain their indecisiveness and intolerance of uncertainty.


eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
Geert-Jan Will ◽  
Robb B Rutledge ◽  
Michael Moutoussis ◽  
Raymond J Dolan

Self-esteem is shaped by the appraisals we receive from others. Here, we characterize neural and computational mechanisms underlying this form of social influence. We introduce a computational model that captures fluctuations in self-esteem engendered by prediction errors that quantify the difference between expected and received social feedback. Using functional MRI, we show these social prediction errors correlate with activity in ventral striatum/subgenual anterior cingulate cortex, while updates in self-esteem resulting from these errors co-varied with activity in ventromedial prefrontal cortex (vmPFC). We linked computational parameters to psychiatric symptoms using canonical correlation analysis to identify an ‘interpersonal vulnerability’ dimension. Vulnerability modulated the expression of prediction error responses in anterior insula and insula-vmPFC connectivity during self-esteem updates. Our findings indicate that updating of self-evaluative beliefs relies on learning mechanisms akin to those used in learning about others. Enhanced insula-vmPFC connectivity during updating of those beliefs may represent a marker for psychiatric vulnerability.


2015 ◽  
Vol 113 (10) ◽  
pp. 3459-3461 ◽  
Author(s):  
Chong Chen

Our understanding of the neural basis of reinforcement learning and intelligence, two key factors contributing to human strivings, has progressed significantly recently. However, the overlap of these two lines of research, namely, how intelligence affects neural responses during reinforcement learning, remains uninvestigated. A mini-review of three existing studies suggests that higher IQ (especially fluid IQ) may enhance the neural signal of positive prediction error in dorsolateral prefrontal cortex, dorsal anterior cingulate cortex, and striatum, several brain substrates of reinforcement learning or intelligence.


2020 ◽  
Vol 17 (5) ◽  
pp. 452-459
Author(s):  
Byung-Hoon Kim ◽  
Yu-Bin Shin ◽  
Sunghyon Kyeong ◽  
Seon-Koo Lee ◽  
Jae-Jin Kim

Objective Little has been explored about a reflection towards self-image in schizophrenia, though it can be related to heterogeneous symptoms of the illness. We identified the neural basis of ambivalence towards ideal self-image in patients with schizophrenia.Methods 20 patients with schizophrenia and 20 healthy controls underwent functional MRI while the self-image reflection tasks of determining whether to agree with sentences describing their actual or ideal self-image that contained one of the adjective pairs with opposite valence. The interaction between the group and ideal ambivalence score was examined, and group differences in functional connectivity related to ambivalence towards ideal self-image were further studied.Results The interaction of group-by-ideal ambivalence score was shown in the dorsal anterior cingulate cortex and dorsolateral prefrontal cortex, where activities were positively correlated with the level of ideal self-image ambivalence in patients, but not in controls. Task-related decrease in functional connectivity was shown between the orbitofrontal cortex and cerebellum in patients.Conclusion The process of reflecting on ambivalent ideal self-image in schizophrenia may be related to aberrant prefrontal activity and connectivity. Abnormality in the prefrontal regions that take part in cognitive conflict monitoring and value judgment may underlie the pathophysiology of increased ambivalence towards ideal self-image.


2017 ◽  
Author(s):  
Amitai Shenhav ◽  
Mark A. Straccia ◽  
Jonathan D. Cohen ◽  
Matthew M. Botvinick

AbstractDecision-making is typically studied as a sequential process from the selection of what to attend (e.g., between possible tasks, stimuli, or stimulus attributes) to the selection of which actions to take based on the attended information. However, people often gather information across these levels in parallel. For instance, even as they choose their actions, they may continue to evaluate how much to attend other tasks or dimensions of information within a task. We scanned participants while they made such parallel evaluations, simultaneously weighing how much to attend two dynamic stimulus attributes and which response to give based on the attended information. Regions of prefrontal cortex tracked information about the stimulus attributes in dissociable ways, related to either the predicted reward (ventromedial prefrontal cortex) or the degree to which that attribute was being attended (dorsal anterior cingulate, dACC). Within dACC, adjacent regions tracked uncertainty at different levels of the decision, regarding what to attend versus how to respond. These findings bridge research on perceptual and value-based decision-making, demonstrating that people dynamically integrate information in parallel across different levels of decision making.Naturalistic decisions allow an individual to weigh their options within a particular task (e.g., how best to word the introduction to a paper) while also weighing how much to attend other tasks (e.g., responding to e-mails). These different types of decision-making have a hierarchical but reciprocal relationship: Decisions at higher levels inform the focus of attention at lower levels (e.g., whether to select between citations or email addresses) while, at the same time, information at lower levels (e.g., the salience of an incoming email) informs decisions regarding which task to attend. Critically, recent studies suggest that decisions across these levels may occur in parallel, continuously informed by information that is integrated from the environment and from one’s internal milieu1,2.Research on cognitive control and perceptual decision-making has examined how responses are selected when attentional targets are clearly defined (e.g., based on instruction to attend a stimulus dimension), including cases in which responding requires accumulating information regarding a noisy percept (e.g., evidence favoring a left or right response)3-7. Separate research on value-based decision-making has examined how individuals select which stimulus dimension(s) to attend in order to maximize their expected rewards8-11. However, it remains unclear how the accumulation of evidence to select high-level goals and/or attentional targets interacts with the simultaneous accumulation of evidence to select responses according to those goals (e.g., based on the perceptual properties of the stimuli). Recent work has highlighted the importance of such interactions to understanding task selection12-15, multi-attribute decision-making16-18, foraging behavior19-21, cognitive effort22,23, and self-control24-27.While these interactions remain poorly understood, previous research has identified candidate neural mechanisms associated with multi-attribute value-based decision-making11,28,29 and with selecting a response based on noisy information from an instructed attentional target3–5. These research areas have implicated the ventromedial prefrontal cortex (vmPFC) in tracking the value of potential targets of attention (e.g., stimulus attributes)8,11 and the dorsal anterior cingulate cortex (dACC) in tracking an individual’s uncertainty regarding which response to select30–32. It has been further proposed that dACC may differentiate between uncertainty at each of these parallel levels of decision-making (e.g., at the level of task goals or strategies vs. specific motor actions), and that these may be separately encoded at different locations along the dACC’s rostrocaudal axis32,33. However, neural activity within and across these prefrontal regions has not yet been examined in a setting in which information is weighed at both levels within and across trials.Here we use a value-based perceptual decision-making task to examine how people integrate different dynamic sources of information to decide (a) which perceptual attribute to attend and (b) how to respond based on the evidence for that attribute. Participants performed a task in which they regularly faced a conflict between attending the stimulus attribute that offered the greater reward or the attribute that was more perceptually salient (akin to persevering in writing one’s paper when an enticing email awaits). We demonstrate that dACC and vmPFC track evidence for the two attributes in dissociable ways. Across these regions, vmPFC weighs attribute evidence by the reward it predicts and dACC weighs it by its attentional priority (i.e., the degree to which that attribute drives choice). Within dACC, adjacent regions differentiated between uncertainty at the two levels of the decision, regarding what to attend (rostral dACC) versus how to respond (caudal dACC).


Author(s):  
Brianne Disabato ◽  
Isabelle E. Bauer ◽  
Jair C. Soares ◽  
Yvette Sheline

Unipolar major depressive disorder (MDD) and bipolar disorder (BD) are among the world’s leading causes of disability. This chapter highlights the importance of neuroimaging in understanding their neural mechanisms. Depression affects limbic-corticostriatopallidothalamic regions. Structurally, depressed subjects showed increased volume of lesions in white matter (WMH) and decreased gray matter in prefrontal-striatum, orbitofrontal, anterior cingulate cortices, and hippocampus. Functionally, depressed subjects showed abnormal activation in amygdala and medial prefrontal cortex and dsyconnectivity in executive and emotional networks. BD was associated with frontocingulate, limbic-striatal, and hippocampus abnormalities. Specifically, BD subjects showed increased WMH in frontocortical and subcortical areas and altered microstructure in limbic-striatal, cingulate, thalamus, corpus callosum, and prefrontal regions. Functionally, abnormal activations in dorsolateral prefrontal and ventrolimbic regions, hypoconnectivity in the cinguloinsularopercular, mesoparalimbic, and cerebellar networks, and hyperconnectivity in affective and executive networks were also observed. These studies show congruence. Full integration of them would allow better understanding of mood disorders.


Sign in / Sign up

Export Citation Format

Share Document