scholarly journals Oxytocin modulates neurocomputational mechanisms underlying prosocial reinforcement learning

2021 ◽  
Author(s):  
Daniel Martins ◽  
Patricia Lockwood ◽  
Jo Cutler ◽  
Rosalyn J. Moran ◽  
Yannis Paloyelis
Keyword(s):  
Reinforcement Learning ◽  
Computational Models ◽  
Anterior Cingulate ◽  
Best Interests ◽  
Prediction Errors ◽  
Subgenual Anterior Cingulate Cortex ◽  
How People Learn ◽  
Dose Dependent ◽  
Different Doses

Humans often act in the best interests of others. However, how we learn which actions result in good outcomes for other people and the neurochemical systems that support this "prosocial learning" remain poorly understood. Using computational models of reinforcement learning, functional magnetic resonance imaging and dynamic causal modelling, we examined how different doses of intranasal oxytocin, a neuropeptide linked to social cognition, impact how people learn to benefit others (prosocial learning) and whether this influence could be dissociated from how we learn to benefit ourselves (self-oriented learning). We show that a low dose of oxytocin prevented decreases in prosocial performance over time, despite no impact on self-oriented learning. Critically, oxytocin produced dose-dependent changes in the encoding of prediction errors (PE) in the midbrain-subgenual anterior cingulate cortex (sgACC) pathway specifically during prosocial learning. Our findings reveal a new role of oxytocin in prosocial learning by modulating computations of PEs in the midbrain-sgACC pathway.

scholarly journals Neurocomputational mechanisms of prosocial learning and links to empathy

2016 ◽  
Vol 113 (35) ◽  
pp. 9763-9768 ◽  
Author(s):  
Patricia L. Lockwood ◽  
Matthew A. J. Apps ◽  
Vincent Valton ◽  
Essi Viding ◽  
Jonathan P. Roiser
Keyword(s):  
Reinforcement Learning ◽  
Prosocial Behavior ◽  
Learning Theory ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Posterior Portion ◽  
Subgenual Anterior Cingulate Cortex ◽  
Actual Outcome ◽  
The Difference ◽  
Do So

Reinforcement learning theory powerfully characterizes how we learn to benefit ourselves. In this theory, prediction errors—the difference between a predicted and actual outcome of a choice—drive learning. However, we do not operate in a social vacuum. To behave prosocially we must learn the consequences of our actions for other people. Empathy, the ability to vicariously experience and understand the affect of others, is hypothesized to be a critical facilitator of prosocial behaviors, but the link between empathy and prosocial behavior is still unclear. During functional magnetic resonance imaging (fMRI) participants chose between different stimuli that were probabilistically associated with rewards for themselves (self), another person (prosocial), or no one (control). Using computational modeling, we show that people can learn to obtain rewards for others but do so more slowly than when learning to obtain rewards for themselves. fMRI revealed that activity in a posterior portion of the subgenual anterior cingulate cortex/basal forebrain (sgACC) drives learning only when we are acting in a prosocial context and signals a prosocial prediction error conforming to classical principles of reinforcement learning theory. However, there is also substantial variability in the neural and behavioral efficiency of prosocial learning, which is predicted by trait empathy. More empathic people learn more quickly when benefitting others, and their sgACC response is the most selective for prosocial learning. We thus reveal a computational mechanism driving prosocial learning in humans. This framework could provide insights into atypical prosocial behavior in those with disorders of social cognition.

scholarly journals Precision weighting of cortical unsigned prediction errors is mediated by dopamine and benefits learning

2018 ◽  
Author(s):  
J. Haarsma ◽  
P.C. Fletcher ◽  
H. Ziauddeen ◽  
T.J. Spencer ◽  
K.M.J. Diederen ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Predictive Coding ◽  
Learning Task ◽  
Anterior Cingulate ◽  
Dopamine Antagonist ◽  
Prediction Errors ◽  
Data Set ◽  
Health And Illness ◽  
Hierarchical Bayesian Inference

AbstractThe predictive coding framework construes the brain as performing a specific form of hierarchical Bayesian inference. In this framework the precision of cortical unsigned prediction error (surprise) signals is proposed to play a key role in learning and decision-making, and to be controlled by dopamine. To test this hypothesis, we re-analysed an existing data-set from healthy individuals who received a dopamine agonist, antagonist or placebo and who performed an associative learning task under different levels of outcome precision. Computational reinforcement-learning modelling of behaviour provided support for precision-weighting of unsigned prediction errors. Functional MRI revealed coding of unsigned prediction errors relative to their precision in bilateral superior frontal gyri and dorsal anterior cingulate. Cortical precision-weighting was (i) perturbed by the dopamine antagonist sulpiride, and (ii) associated with task performance. These findings have important implications for understanding the role of dopamine in reinforcement learning and predictive coding in health and illness.

scholarly journals Neural and computational processes underlying dynamic changes in self-esteem

eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
Geert-Jan Will ◽  
Robb B Rutledge ◽  
Michael Moutoussis ◽  
Raymond J Dolan
Keyword(s):  
Psychiatric Symptoms ◽  
Self Esteem ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Social Feedback ◽  
Learning Mechanisms ◽  
Subgenual Anterior Cingulate Cortex ◽  
The Difference ◽  
Interpersonal Vulnerability ◽  
Social Prediction

Self-esteem is shaped by the appraisals we receive from others. Here, we characterize neural and computational mechanisms underlying this form of social influence. We introduce a computational model that captures fluctuations in self-esteem engendered by prediction errors that quantify the difference between expected and received social feedback. Using functional MRI, we show these social prediction errors correlate with activity in ventral striatum/subgenual anterior cingulate cortex, while updates in self-esteem resulting from these errors co-varied with activity in ventromedial prefrontal cortex (vmPFC). We linked computational parameters to psychiatric symptoms using canonical correlation analysis to identify an ‘interpersonal vulnerability’ dimension. Vulnerability modulated the expression of prediction error responses in anterior insula and insula-vmPFC connectivity during self-esteem updates. Our findings indicate that updating of self-evaluative beliefs relies on learning mechanisms akin to those used in learning about others. Enhanced insula-vmPFC connectivity during updating of those beliefs may represent a marker for psychiatric vulnerability.

scholarly journals The best laid plans: Computational principles of ACC

2020 ◽  
Author(s):  
Clay B. Holroyd ◽  
Tom Verguts
Keyword(s):  
Reinforcement Learning ◽  
Anterior Cingulate Cortex ◽  
Hierarchical Model ◽  
Computational Models ◽  
Cingulate Cortex ◽  
Anterior Cingulate ◽  
Hierarchical Reinforcement Learning ◽  
The Past ◽  
Model Based ◽  
High Level

Despite continual debate for the past thirty years about the function of anterior cingulate cortex (ACC), its key contribution to neurocognition remains unknown. Here we review computational models that illustrate three core principles of ACC function (related to hierarchy, world models and cost), as well as four constraints on the neural implementation of these principles (related to modularity, binding, encoding and learning and regulation). These observations suggest a role for ACC in hierarchical model-based hierarchical reinforcement learning, which instantiates a mechanism for motivating the execution of high-level plans.

scholarly journals From internal models toward metacognitive AI

2021 ◽  
Author(s):  
Mitsuo Kawato ◽  
Aurelio Cortese
Keyword(s):  
Reinforcement Learning ◽  
Computational Models ◽  
Monitoring Network ◽  
Internal Models ◽  
Inverse Model ◽  
Small Samples ◽  
Prediction Errors ◽  
Hierarchical Reinforcement Learning ◽  
Reward Prediction ◽  
Higher Cognitive Functions

AbstractIn several papers published in Biological Cybernetics in the 1980s and 1990s, Kawato and colleagues proposed computational models explaining how internal models are acquired in the cerebellum. These models were later supported by neurophysiological experiments using monkeys and neuroimaging experiments involving humans. These early studies influenced neuroscience from basic, sensory-motor control to higher cognitive functions. One of the most perplexing enigmas related to internal models is to understand the neural mechanisms that enable animals to learn large-dimensional problems with so few trials. Consciousness and metacognition—the ability to monitor one’s own thoughts, may be part of the solution to this enigma. Based on literature reviews of the past 20 years, here we propose a computational neuroscience model of metacognition. The model comprises a modular hierarchical reinforcement-learning architecture of parallel and layered, generative-inverse model pairs. In the prefrontal cortex, a distributed executive network called the “cognitive reality monitoring network” (CRMN) orchestrates conscious involvement of generative-inverse model pairs in perception and action. Based on mismatches between computations by generative and inverse models, as well as reward prediction errors, CRMN computes a “responsibility signal” that gates selection and learning of pairs in perception, action, and reinforcement learning. A high responsibility signal is given to the pairs that best capture the external world, that are competent in movements (small mismatch), and that are capable of reinforcement learning (small reward-prediction error). CRMN selects pairs with higher responsibility signals as objects of metacognition, and consciousness is determined by the entropy of responsibility signals across all pairs. This model could lead to new-generation AI, which exhibits metacognition, consciousness, dimension reduction, selection of modules and corresponding representations, and learning from small samples. It may also lead to the development of a new scientific paradigm that enables the causal study of consciousness by combining CRMN and decoded neurofeedback.

Endogenous cholecystokinin drives gallbladder emptying in dogs

1986 ◽  
Vol 251 (4) ◽  
pp. G553-G558
Author(s):  
K. Shiratori ◽  
S. Watanabe ◽  
W. Y. Chey ◽  
K. Y. Lee ◽  
T. M. Chang
Keyword(s):  
Corn Oil ◽  
Venous Blood ◽  
Gallbladder Emptying ◽  
Dependent Manner ◽  
Gallbladder Volume ◽  
Dose Dependent ◽  
Intraduodenal Administration ◽  
Different Doses ◽  
Cck Release

We investigated the effect of fat in the duodenum on the gallbladder emptying in seven dogs prepared with gastric, duodenal, and gallbladder cannulas. Gallbladder volume was measured at 15-min intervals, and venous blood samples were obtained at regular intervals for 2.5 h. Intraduodenal administration of Lipomul (pH 5.0, corn oil) in three different doses (1.1, 2.2, and 4.4 mmol/10 min) resulted in significant increases in gallbladder emptying in a dose-dependent manner (r = 0.8668, P less than 0.001). Likewise, the increase in integrated cholecystokinin (CCK) release in response to Lipomul was also dose dependent (r = 0.7334, P less than 0.001). A statistically significant correlation was found between integrated CCK release and gallbladder emptying in response to Lipomul (P less than 0.001). To determine the role of circulating endogenous CCK on gallbladder emptying effects of intravenous administration of proglumide and a rabbit anti-CCK serum on gallbladder emptying were studied. Gallbladder emptying was virtually abolished by the antiserum. Proglumide not only abolished the emptying but also increased gallbladder volume. Thus we conclude that in dogs the gallbladder emptying in response to fat in the upper small intestine depends on increased circulating endogenous CCK.

scholarly journals The Role of the Subgenual Anterior Cingulate Cortex and Amygdala in Environmental Sensitivity to Infant Crying

PLoS ONE ◽  
2016 ◽  
Vol 11 (8) ◽  
pp. e0161181 ◽  
Author(s):  
Isabella Mutschler ◽  
Tonio Ball ◽  
Ursula Kirmse ◽  
Birgit Wieckhorst ◽  
Michael Pluess ◽  
...  
Keyword(s):  
Anterior Cingulate Cortex ◽  
Cingulate Cortex ◽  
Anterior Cingulate ◽  
Environmental Sensitivity ◽  
Infant Crying ◽  
Subgenual Anterior Cingulate Cortex

The Role of 5-HT1A/BAutoreceptors in the Antinociceptive Effect of Systemic Administration of Acetaminophen

2003 ◽  
Vol 98 (3) ◽  
pp. 741-747 ◽  
Author(s):  
Aránzazu Roca-Vinardell ◽  
Antonio Ortega-Alvaro ◽  
Juan Gibert-Rahola ◽  
Juan A. Micó
Keyword(s):  
Antinociceptive Effect ◽  
Serotonin Release ◽  
Hot Plate ◽  
Hot Plate Test ◽  
Plate Test ◽  
Way 100635 ◽  
New Strategy ◽  
Dose Dependent ◽  
Different Doses

Background It has been proposed that serotonin participates in the central antinociceptive effect of acetaminophen. The serotonin activity in the brainstem is primarily under the control of 5-HT1A somatodendritic receptors, although some data also suggest the involvement of 5-HT1B receptors. In the presence of serotonin, the blockade of 5-HT(1A/B) receptors at the level of the raphe nuclei leads to an increase in serotonin release in terminal areas, thus improving serotonin functions. This study examines the involvement of 5-HT(1A/B) receptors in the antinociceptive effect of acetaminophen in mice. Methods The effects of acetaminophen (600 mg/kg intraperitoneal) followed by different doses of antagonists (WAY 100635 [0.2-0.8 mg/kg subcutaneous] and SB 216641 [0.2-0.8 mg/kg subcutaneous]) or agonists (8-OH-DPAT [0.25-1 mg/kg subcutaneous] and CP 93129 [0.125-0.5 mg/kg subcutaneous]) of 5-HT1A and 5-HT1B receptors, respectively, were determined in the hot-plate test in mice. Results Acetaminophen (300-800 mg/kg) showed a dose-dependent antinociceptive effect in the hot-plate test in mice. WAY 100635 (0.2-0.8 mg/kg; 5-HT1A antagonist) induced an increase in the antinociceptive effect of 600 mg/kg acetaminophen, but this increase was not dose related. Conversely, 8-OH-DPAT (0.25-1 mg/kg; 5-HT1A agonist) decreased the antinociceptive effect of acetaminophen. SB 216641 (0.2-0.8 mg/kg; 5-HT1B antagonist) induced a dose-related increase in the antinociceptive effect of acetaminophen, and CP 93129 (0.25 mg/kg; 5-HT1B agonist) significantly decreased the antinociceptive effect of acetaminophen. Conclusions These results suggest that the combination of acetaminophen with compounds having 5-HT1A and 5-HT1B antagonist properties could be a new strategy to improve the analgesia of acetaminophen, thanks to its mild serotonergic properties.

scholarly journals Neural signatures of model-free learning when avoiding harm to self and other

2019 ◽  
Author(s):  
Patricia L. Lockwood ◽  
Miriam Klein-Flügge ◽  
Ayat Abdurahman ◽  
Molly J. Crockett
Keyword(s):  
Anterior Cingulate ◽  
Prediction Errors ◽  
Learning Mechanisms ◽  
Neural Basis ◽  
Self And Other ◽  
Subgenual Anterior Cingulate Cortex ◽  
Model Free ◽  
Moral Learning ◽  
Dorsolateral Prefrontal ◽  
Moral Behaviour

AbstractMoral behaviour requires learning how our actions help or harm others. Theoretical accounts of learning propose a key division between ‘model-free’ algorithms that efficiently cache outcome values in actions and ‘model-based’ algorithms that prospectively map actions to outcomes, a distinction that may be critical for moral learning. Here, we tested the engagement of these learning mechanisms and their neural basis as participants learned to avoid painful electric shocks for themselves and a stranger. We found that model-free learning was prioritized when avoiding harm to others compared to oneself. Model-free prediction errors for others relative to self were tracked in the thalamus/caudate at the time of the outcome. At the time of choice, a signature of model-free moral learning was associated with responses in subgenual anterior cingulate cortex (sgACC), and resisting this model-free influence was predicted by stronger connectivity between sgACC and dorsolateral prefrontal cortex. Finally, multiple behavioural and neural correlates of model-free moral learning varied with individual differences in moral judgment. Our findings suggest moral learning favours efficiency over flexibility and is underpinned by specific neural mechanisms.

Sign in / Sign up

Export Citation Format

Share Document