scholarly journals Trace Imbalance in Reinforcement and Punishment Systems Can Mis-reinforce Implicit Choices Leading to Anxiety

2020 ◽  
Author(s):  
Yuki Sakai ◽  
Yutaka Sakai ◽  
Yoshinari Abe ◽  
Jin Narumoto ◽  
Saori C. Tanaka
Keyword(s):  
Reinforcement Learning ◽  
Memory Trace ◽  
Healthy Population ◽  
Prediction Errors ◽  
Obsessive Compulsive ◽  
Compulsive Disorder ◽  
Learning Framework ◽  
Negative Prediction ◽  
Neural Signature ◽  
Healthy Participants

AbstractNobody wants to experience anxiety. However, anxiety may be induced by our own implicit choices that are mis-reinforced by some imbalance in reinforcement learning. Here we focused on obsessive-compulsive disorder (OCD) as a candidate for implicitly learned anxiety. Simulations in the reinforcement learning framework showed that agents implicitly learn to become anxious when the memory trace signal for past actions decays differently for positive and negative prediction errors. In empirical data, we confirmed that OCD patients showed extremely imbalanced traces, which were normalized by serotonin enhancers. We also used fMRI to identify the neural signature of OCD and healthy participants with imbalanced traces. Beyond the spectrum of clinical phenotypes, these behavioral and neural characteristics can be generalized to variations in the healthy population.

scholarly journals Atypical action updating in a dynamic environment associated with adolescent obsessive-compulsive disorder

2021 ◽  
Author(s):  
Aleya A Aziz Marzuki ◽  
Matilde Vaghi ◽  
Anna Conway-Morris ◽  
Muzaffer Kaser ◽  
Akeem Sule ◽  
...  
Keyword(s):  
Obsessive Compulsive Disorder ◽  
Young Patients ◽  
Dynamic Environment ◽  
Prediction Errors ◽  
Obsessive Compulsive ◽  
Compulsive Disorder ◽  
Error Related Negativity ◽  
The Face ◽  
Cognitive Information ◽  
Confidence Ratings

Background Computational research had determined that adults with obsessive-compulsive disorder (OCD) display heightened action updating in response to noise in the environment and neglect meta-cognitive information (such as confidence) when making decisions. These features are proposed to underlie patients compulsions despite knowledge they are irrational. Nonetheless, it is unclear whether this extends to adolescents with OCD as research in this population is lacking. Thus, this study aimed to investigate the interplay between action and confidence in adolescents with OCD. Methods Twenty-seven adolescents with OCD and 46 controls completed a predictive-inference task, designed to probe how subjects actions and confidence ratings fluctuate in response to unexpected outcomes. We investigated how subjects update actions in response to prediction errors (indexing mismatches between expectations and outcomes) and used parameters from a Bayesian model to predict how confidence and action evolve over time. Confidence-action association strength was assessed using a regression model. We also investigated the effects of serotonergic medication. Results Adolescents with OCD showed significantly increased learning rates, particularly following small prediction errors. Results were driven primarily by unmedicated patients. Confidence ratings appeared equivalent between groups, although model-based analysis revealed that patients confidence was less affected by prediction errors compared to controls. Patients and controls did not differ in the extent to which they updated actions and confidence in tandem. Conclusions Adolescents with OCD showed enhanced action adjustments, especially in the face of small prediction errors, consistent with previous research establishing just-right compulsions, enhanced error-related negativity, and greater decision-uncertainty in paediatric-OCD. These tendencies were ameliorated in patients receiving serotonergic medication, emphasising the importance of early intervention in preventing disorder-related cognitive deficits. Confidence ratings were equivalent between young patients and controls, mirroring findings in adults OCD research.

scholarly journals Increased fronto-striatal reward prediction errors moderate decision making in obsessive–compulsive disorder

2017 ◽  
Vol 47 (7) ◽  
pp. 1246-1258 ◽  
Author(s):  
T. U. Hauser ◽  
R. Iannaccone ◽  
R. J. Dolan ◽  
J. Ball ◽  
J. Hättenschwiler ◽  
...  
Keyword(s):  
Decision Making ◽  
Obsessive Compulsive Disorder ◽  
Late Onset ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Learning Mechanisms ◽  
Obsessive Compulsive ◽  
Compulsive Disorder ◽  
Reward Prediction ◽  
Reinforcement Learning Models

BackgroundObsessive–compulsive disorder (OCD) has been linked to functional abnormalities in fronto-striatal networks as well as impairments in decision making and learning. Little is known about the neurocognitive mechanisms causing these decision-making and learning deficits in OCD, and how they relate to dysfunction in fronto-striatal networks.MethodWe investigated neural mechanisms of decision making in OCD patients, including early and late onset of disorder, in terms of reward prediction errors (RPEs) using functional magnetic resonance imaging. RPEs index a mismatch between expected and received outcomes, encoded by the dopaminergic system, and are known to drive learning and decision making in humans and animals. We used reinforcement learning models and RPE signals to infer the learning mechanisms and to compare behavioural parameters and neural RPE responses of the OCD patients with those of healthy matched controls.ResultsPatients with OCD showed significantly increased RPE responses in the anterior cingulate cortex (ACC) and the putamen compared with controls. OCD patients also had a significantly lower perseveration parameter than controls.ConclusionsEnhanced RPE signals in the ACC and putamen extend previous findings of fronto-striatal deficits in OCD. These abnormally strong RPEs suggest a hyper-responsive learning network in patients with OCD, which might explain their indecisiveness and intolerance of uncertainty.

scholarly journals A Comparison of Manifestations and Impact of Reassurance Seeking among Japanese Individuals with OCD and Depression

2014 ◽  
Vol 43 (5) ◽  
pp. 623-634 ◽  
Author(s):  
Osamu Kobori ◽  
Yoko Sawamiya ◽  
Masaomi Iyo ◽  
Eiji Shimizu
Keyword(s):  
Obsessive Compulsive Disorder ◽  
Obsessive Compulsive ◽  
Reassurance Seeking ◽  
Compulsive Disorder ◽  
Key Aspects ◽  
Healthy Participants

Background: One of the most common interpersonal reactions to threat and anxiety is to seek reassurance from a trusted person. The Reassurance Seeking Questionnaire (ReSQ) measures several key aspects of reassurance seeking behaviour, including frequency, trust of sources, intensity, carefulness, and the emotional consequences of reassurance seeking. Aims: The current study compares patterns and consequences of reassurance seeking in obsessive-compulsive disorder (OCD) and depression. Method: ReSQ scores were compared for three groups: 32 individuals with OCD, 17 individuals with depression, and 24 healthy comparison participants. Results: We found that individuals with OCD tended to seek reassurance more intensely and employ self-reassurance more frequently than individuals with depression or healthy participants, and that if reassurance was not provided, they tended to feel a greater urge to seek additional reassurance. Conclusions: This study is the first to quantitatively elucidate differences in reassurance seeking between OCD and depression.

Common Neural Networks Between ocd and Behavioural Addictions: Is ocd a Behavioral Addiction?

2017 ◽  
Vol 41 (S1) ◽  
pp. S21-S22 ◽  
Author(s):  
G. Grassi ◽  
S. Pallanti
Keyword(s):  
Decision Making ◽  
Probabilistic Reasoning ◽  
Behavioral Addiction ◽  
Obsessive Compulsive ◽  
Risky Decision ◽  
Compulsive Disorder ◽  
The Common ◽  
Competing Interest ◽  
Healthy Participants ◽  
Behavioural Addictions

The stereotypical portrait of an obsessive–compulsive patient is an excessively self-controlled, risk aversive individual that acts in order to avoid potential loss or punishments. Although this portrait fits well with several clinical studies showing increased harm-avoidance in obsessive–compulsive disorder (OCD), more recent clinical, neuropsychological and neuroimaging studies challenged this idea and described a different portrait of OCD, showing several commonalities between OCD and addictions such as impulsivity, reward dysfunction and impaired decision-making. The results of these studies conflict with the stereotypical OCD portrait of doubtfulness and risk-aversiveness. In fact, these findings are prototypical for addiction and have led some authors in the last years to view OCD as a behavioral addiction. In our recently published article, we investigated the behavioral addiction model of obsessive (OCD), by assessing three core dimensions of addiction in patients with OCD and healthy participants. Similar to the common findings in addiction, OCD patients demonstrated increased impulsivity, risky decision-making, and biased probabilistic reasoning compared to healthy controls. During the presentation we will discuss the behavioral addiction model of OCD by focusing on common neuropsychological and neurobiological circuitries.Disclosure of interestThe authors have not supplied their declaration of competing interest.

The Comorbidity of Traumatic Life Events in Childhood Age With Adult Obsessive-Compulsive Disorder

2016 ◽  
Vol 33 (S1) ◽  
pp. S497-S497 ◽  
Author(s):  
S. Kıvılcım ◽  
H. Erensoy ◽  
D.B. Tonguç ◽  
G. Sarıdogan ◽  
K. Ebru
Keyword(s):  
Obsessive Compulsive Disorder ◽  
Life Events ◽  
Emotional Abuse ◽  
Epidemiological Studies ◽  
Healthy Population ◽  
Healthy Controls ◽  
Obsessive Compulsive ◽  
Compulsive Disorder ◽  
Traumatic Life Events ◽  
Statistical Level

PurposeObsessive-Compulsive Disorder (OCD) is a mental disorder characterized by obsessions and/or compulsions. Although some epidemiological studies take part in literature, which claim that traumatic life events in childhood ages are observed more in patients with OCD compared to healthy population, the number of these studies is limited. In this study, it is aimed to compare OCD patients with healthy volunteers in terms of traumatic life events in childhood ages.MethodWith 25 consecutive patients who are diagnosed as OCD and whose treatment continues, 25 healthy controls equivalent in terms of sociodemographic features are included in the study. Sociodemographic Data Form, Childhood Age Trauma Quarter (CTQ) and Maudley Obsessive Compulsive Question List (MOCQL) are applied to the participants. Significance Value in statistical level is accepted as P < 0.05.FindingsIn OCD patient group, CTQ scores are found high in statistical level compared to healthy controls. It has been determined that there is a significant relationship between total score of MOCQL slowness subscale scores, subscale scores of sexual and emotional abuse, MOCQL rumination subscale scores and CTQ sexual abuse scores.ResultCompared to healthy controls, more findings of traumatic life event in childhood age are observed within OCD patients.Disclosure of interestThe authors have not supplied their declaration of competing interest.

scholarly journals Learning at variable attentional load requires cooperation between working memory, meta-learning and attention-augmented reinforcement learning

2020 ◽  
Author(s):  
Thilo Womelsdorf ◽  
Marcus R. Watson ◽  
Paul Tiesinga
Keyword(s):  
Working Memory ◽  
Reinforcement Learning ◽  
Cognitive Flexibility ◽  
Real World ◽  
Learning Strategy ◽  
Prediction Errors ◽  
Negative Prediction ◽  
Attentional Load ◽  
Meta Learning ◽  
Efficient Learning

AbstractFlexible learning of changing reward contingencies can be realized with different strategies. A fast learning strategy involves using working memory of recently rewarded objects to guide choices. A slower learning strategy uses prediction errors to gradually update value expectations to improve choices. How the fast and slow strategies work together in scenarios with real-world stimulus complexity is not well known. Here, we disentangle their relative contributions in rhesus monkeys while they learned the relevance of object features at variable attentional load. We found that learning behavior across six subjects is consistently best predicted with a model combining (i) fast working memory (ii) slower reinforcement learning from differently weighted positive and negative prediction errors, as well as (iii) selective suppression of non-chosen feature values and (iv) a meta-learning mechanism adjusting exploration rates based on a memory trace of recent errors. These mechanisms cooperate differently at low and high attentional loads. While working memory was essential for efficient learning at lower attentional loads, enhanced weighting of negative prediction errors and meta-learning were essential for efficient learning at higher attentional loads. Together, these findings pinpoint a canonical set of learning mechanisms and demonstrate how they cooperate when subjects flexibly adjust to environments with variable real-world attentional demands.Significance statementLearning which visual features are relevant for achieving our goals is challenging in real-world scenarios with multiple distracting features and feature dimensions. It is known that in such scenarios learning benefits significantly from attentional prioritization. Here we show that beyond attention, flexible learning uses a working memory system, a separate learning gain for avoiding negative outcomes, and a meta-learning process that adaptively increases exploration rates whenever errors accumulate. These subcomponent processes of cognitive flexibility depend on distinct learning signals that operate at varying timescales, including the most recent reward outcome (for working memory), memories of recent outcomes (for adjusting exploration), and reward prediction errors (for attention augmented reinforcement learning). These results illustrate the specific mechanisms that cooperate during cognitive flexibility.

scholarly journals Moral judgments in obsessive-compulsive disorder: a narrative mini-review

2020 ◽  
Vol 3 (1) ◽  
pp. 11-23
Author(s):  
Mehrnaz Hosseinzadeh ◽  
Elma Azhdehakosh ◽  
Adib Valibeygi
Keyword(s):  
Obsessive Compulsive Disorder ◽  
Behavioral Problems ◽  
Intrusive Thoughts ◽  
Moral Dilemmas ◽  
Moral Judgments ◽  
Dual Process ◽  
Healthy Population ◽  
Moral Norms ◽  
Obsessive Compulsive ◽  
Compulsive Disorder

Obsessive-compulsive disorder (OCD) is a prevalent mental disorder characterized by intrusive thoughts (obsessions) and ensuing rituals (compulsions). Although OC patients exhibit various cognitive and behavioral problems, rigid and hypersensitive moral judgments are known to be one of the most striking problems in these patients. There is evidence indicating that OC patients often tend to make deontological judgments in moral dilemmas, significantly more than the healthy population. Therefore, numerous studies are dedicated to understanding the underlying cognitive processes responsible for such variation of moral judgments in OCD, which are reviewed and discussed in the current paper. First, it is previously discussed that abnormal moral judgments in OCD are due to executive dysfunctions. These dysfunctions include impaired cognitive control resulting in the domination of strong, uncontrolled emotional responses, impaired cognitive flexibility resulting in the inability to switch between aspects of a scenario, and decreased capacity and overload of working memory and its inability to resist the interfering information. The dual-process theory also emphasizes and acknowledges the role of executive functions in moral judgments. Second, it is thought that disobeying moral norms results in the abnormal feeling of deontological guilt in OC patients, to which these patients are highly sensitive. Feeling of guilt is also thought to be correlated with OCD symptomatology. The third impairment contributing to abnormal moral judgments in OCD is known to be the abnormal feeling of disgust for moral violations and immoral unwanted intrusive thoughts, which is regarded as one of the major causes of OCD symptoms. Finally, the abnormal fear of responsibility and being criticized due to not acting morally is regarded as one of the primary impairments contributing to the abnormal moral judgments in OCD. In conclusion, this review sheds light on the most striking cognitive and affective impairments contributing to abnormal moral judgments in OCD.

Learning at Variable Attentional Load Requires Cooperation of Working Memory, Meta-learning and Attention-augmented Reinforcement Learning

2021 ◽  
pp. 1-29
Author(s):  
Thilo Womelsdorf ◽  
Marcus R. Watson ◽  
Paul Tiesinga
Keyword(s):  
Working Memory ◽  
Reinforcement Learning ◽  
Real World ◽  
Learning Strategy ◽  
Prediction Errors ◽  
Learning Mechanisms ◽  
Negative Prediction ◽  
Attentional Load ◽  
Meta Learning ◽  
Efficient Learning

Abstract Flexible learning of changing reward contingencies can be realized with different strategies. A fast learning strategy involves using working memory of recently rewarded objects to guide choices. A slower learning strategy uses prediction errors to gradually update value expectations to improve choices. How the fast and slow strategies work together in scenarios with real-world stimulus complexity is not well known. Here, we aim to disentangle their relative contributions in rhesus monkeys while they learned the relevance of object features at variable attentional load. We found that learning behavior across six monkeys is consistently best predicted with a model combining (i) fast working memory and (ii) slower reinforcement learning from differently weighted positive and negative prediction errors as well as (iii) selective suppression of nonchosen feature values and (iv) a meta-learning mechanism that enhances exploration rates based on a memory trace of recent errors. The optimal model parameter settings suggest that these mechanisms cooperate differently at low and high attentional loads. Whereas working memory was essential for efficient learning at lower attentional loads, enhanced weighting of negative prediction errors and meta-learning were essential for efficient learning at higher attentional loads. Together, these findings pinpoint a canonical set of learning mechanisms and suggest how they may cooperate when subjects flexibly adjust to environments with variable real-world attentional demands.

Inversion of the “Unrealistic Optimism” Bias Contributes to Overestimation of Threat in Obsessive-Compulsive Disorder

2009 ◽  
Vol 37 (2) ◽  
pp. 179-193 ◽  
Author(s):  
Steffen Moritz ◽  
Lena Jelinek
Keyword(s):  
Obsessive Compulsive Disorder ◽  
Internet Survey ◽  
Average Risk ◽  
Negative Events ◽  
Unrealistic Optimism ◽  
Obsessive Compulsive ◽  
Optimism Bias ◽  
Compulsive Disorder ◽  
The Common ◽  
Healthy Participants

Background: Overestimation of threat (OET) is ascribed a pathogenetic role in obsessive-compulsive disorder (OCD). Aims: We hypothesized that OCD patients overestimate their personal but not the average risk for OCD-related events. Specifically, an attenuation of the common “unrealistic optimism” bias (UO) was expected for OCD patients. UO refers to the phenomenon that the subjective likelihood to personally experience a positive event is enhanced compared to other persons and vice versa for negative events. Method: Fifty-three participants with OCD as well as 40 healthy and 23 psychiatric controls participated in an internet survey. They were asked several questions about different event types (positive, negative, and OCD-related): the probability that this event will happen to oneself (block 1), to another person (block 2), comparison between oneself versus another person (block 3), appraisal of consequences (block 4), and prior encounters with event (block 5). Results: No evidence was obtained in OCD to overestimate the overall probability of negative or OCD-related events. However, whereas healthy participants displayed an UO bias, OCD participants perceived themselves as more vulnerable to experience OCD-related events. Conclusions: Results indicate that OCD is associated with inflated personal vulnerability and that this bias is not fully available to the consciousness of OCD participants.

scholarly journals Valence biases factual and counterfactual learning in opposite directions

2016 ◽  
Author(s):  
Stefano Palminteri ◽  
Germain Lefebvre ◽  
Emma J. Kilford ◽  
Sarah-Jayne Blakemore
Keyword(s):  
Reinforcement Learning ◽  
Computational Model ◽  
Prediction Error ◽  
Prediction Errors ◽  
Learning Tasks ◽  
Negative Prediction ◽  
Account Information ◽  
Factual Learning

AbstractPrevious studies suggest that factual learning, that is, learning from obtained outcomes, is biased, such that participants preferentially take into account positive, as compared to negative, prediction errors. However, whether or not the prediction error valence also affects counterfactual learning, that is, learning from forgone outcomes, is unknown. To address this question, we analysed the performance of two cohorts of participants on reinforcement learning tasks using a computational model that was adapted to test if prediction error valance influences learning. Concerning factual learning, we replicated previous findings of a valence-induced bias, whereby participants learned preferentially from positive, relative to negative, prediction errors. In contrast, for counterfactual learning, we found the opposite valence-induced bias: negative prediction errors were preferentially taken into account relative to positive ones. When considering valence-induced bias in the context of both factual and counterfactual learning, it appears that people tend to preferentially take into account information that confirms their current choice

Sign in / Sign up

Export Citation Format

Share Document