scholarly journals Learning when effort matters: Neural dynamics underlying updating and adaptation to changes in performance efficacy

2020 ◽  
Author(s):  
Ivan Grahek ◽  
Romy Frömer ◽  
Amitai Shenhav
Keyword(s):  
Cognitive Control ◽  
Learning Algorithm ◽  
Neural Mechanisms ◽  
Prediction Errors ◽  
Proactive Control ◽  
Efficacy Expectations ◽  
Learning And Adaptation ◽  
How People Learn ◽  
Improved Performance ◽  
Efficacy Prediction

AbstractTo determine how much cognitive control to invest in a task, people need to consider whether exerting control matters for obtaining potential rewards. In particular, they need to account for the efficacy of their performance – the degree to which potential rewards are determined by their performance or by independent factors (e.g., random chance). Yet it remains unclear how people learn about their performance efficacy in a given environment. Here, we examined the neural and computational mechanisms through which people (a) learn and dynamically update efficacy expectations in a changing environment, and (b) proactively adjust control allocation based on their current efficacy expectations. We recorded EEG in 40 participants performing an incentivized cognitive control task, while their performance efficacy (the likelihood that reward for a given trial would be determined by performance or at random) dynamically varied over time. We found that participants continuously updated their self- reported efficacy expectations based on recent feedback, and that these updates were well described by a standard prediction error-based reinforcement learning algorithm. Paralleling findings on updating of expected rewards, we found that model-based estimates of efficacy prediction errors were encoded by the feedback-related P3b. Updated expectations of efficacy in turn influenced levels of effort exerted on subsequent trials, reflected in greater proactive control (indexed by the contingent negative variation [CNV]) and improved performance when participants expected their performance to be more efficacious. These findings demonstrate that learning and adaptation to the efficacy of one’s environment is underpinned by similar computations and neural mechanisms as are involved in learning about potential reward.

scholarly journals Antagonism between brain regions relevant for cognitive control and emotional memory facilitates the generation of humorous ideas

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Florian Bitsch ◽  
Philipp Berger ◽  
Andreas Fink ◽  
Arne Nagels ◽  
Benjamin Straube ◽  
...  
Keyword(s):  
Cognitive Control ◽  
Positive Emotions ◽  
Emotional Memory ◽  
Brain Regions ◽  
Neural Mechanisms ◽  
Functional Magnetic Resonance ◽  
Frontal Brain ◽  
Successful Resolution ◽  
Neural Underpinnings ◽  
The Brain

AbstractThe ability to generate humor gives rise to positive emotions and thus facilitate the successful resolution of adversity. Although there is consensus that inhibitory processes might be related to broaden the way of thinking, the neural underpinnings of these mechanisms are largely unknown. Here, we use functional Magnetic Resonance Imaging, a humorous alternative uses task and a stroop task, to investigate the brain mechanisms underlying the emergence of humorous ideas in 24 subjects. Neuroimaging results indicate that greater cognitive control abilities are associated with increased activation in the amygdala, the hippocampus and the superior and medial frontal gyrus during the generation of humorous ideas. Examining the neural mechanisms more closely shows that the hypoactivation of frontal brain regions is associated with an hyperactivation in the amygdala and vice versa. This antagonistic connectivity is concurrently linked with an increased number of humorous ideas and enhanced amygdala responses during the task. Our data therefore suggests that a neural antagonism previously related to the emergence and regulation of negative affective responses, is linked with the generation of emotionally positive ideas and may represent an important neural pathway supporting mental health.

Neural mechanisms of the cognitive control of emotion

2006 ◽  
Vol 18 (3-4) ◽  
pp. 144-153 ◽  
Author(s):  
Melissa J. Green ◽  
Gin S. Malhi
Keyword(s):  
Emotion Regulation ◽  
Cognitive Control ◽  
Psychiatric Disorders ◽  
Psychological Health ◽  
Emotion Dysregulation ◽  
Emotional Experience ◽  
Cognitive Strategies ◽  
Emotion Perception ◽  
Neural Mechanisms ◽  
Emotion Generation

Background:Emotion regulation involves the initiation of new emotional responses and continual alteration of current emotions in response to rapidly changing environmental and social stimuli. The capacity to effectively implement emotion regulation strategies is essential for psychological health; impairments in the ability to regulate emotions may be critical to the development of clinical levels of depression, anxiety and mania.Objective:This review provides a summary of findings from current research examining the neural mechanisms of emotion regulation by means of conscious cognitive strategies of reappraisal. These findings are considered in the context of related concepts of emotion perception and emotion generation, with discussion of the likely cognitive neuropsychological contributions to emotion regulation and the implications for psychiatric disorders.Results:Convergent evidence implicates an inhibitory role of prefrontal cortex and cingulate regions upon subcortical and cortical emotion generation systems in the cognitive control of emotional experience. Concurrent modulation of cortical activity by the peripheral nervous system is highlighted by recent studies using simultaneous physiological and neuroimaging techniques. Individual differences in emotion perception, generation of affect and neuropsychological skills are likely to have direct consequences for emotion regulation.Conclusions:Emotion regulation relies on synergy within brain stem, limbic and cortical processes that promote the adaptive perception, generation and regulation of affect. Aberrant emotion processing in any of these stages may disrupt this self-sustaining regulatory system, with the potential to manifest in distinct forms of emotion dysregulation as seen in major psychiatric disorders such as depression, bipolar disorder and schizophrenia.

scholarly journals Selective network discovery via deep reinforcement learning on embedded spaces

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Peter Morales ◽  
Rajmonda Sulo Caceres ◽  
Tina Eliassi-Rad
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Sequential Decision ◽  
Network Discovery ◽  
Learning Tasks ◽  
Partially Observed ◽  
Decision Making Problem ◽  
Resource Collection ◽  
Improved Performance ◽  
Discovery Algorithms

AbstractComplex networks are often either too large for full exploration, partially accessible, or partially observed. Downstream learning tasks on these incomplete networks can produce low quality results. In addition, reducing the incompleteness of the network can be costly and nontrivial. As a result, network discovery algorithms optimized for specific downstream learning tasks given resource collection constraints are of great interest. In this paper, we formulate the task-specific network discovery problem as a sequential decision-making problem. Our downstream task is selective harvesting, the optimal collection of vertices with a particular attribute. We propose a framework, called network actor critic (NAC), which learns a policy and notion of future reward in an offline setting via a deep reinforcement learning algorithm. The NAC paradigm utilizes a task-specific network embedding to reduce the state space complexity. A detailed comparative analysis of popular network embeddings is presented with respect to their role in supporting offline planning. Furthermore, a quantitative study is presented on various synthetic and real benchmarks using NAC and several baselines. We show that offline models of reward and network discovery policies lead to significantly improved performance when compared to competitive online discovery algorithms. Finally, we outline learning regimes where planning is critical in addressing sparse and changing reward signals.

scholarly journals Neural Dynamics of Cognitive Control over Working Memory Capture of Attention

2019 ◽  
Vol 31 (7) ◽  
pp. 1079-1090 ◽  
Author(s):  
Peter S. Whitehead ◽  
Mathilde M. Ooi ◽  
Tobias Egner ◽  
Marty G. Woldorff
Keyword(s):  
Working Memory ◽  
Visual Search ◽  
Cognitive Control ◽  
Attentional Capture ◽  
Search Display ◽  
Neural Mechanisms ◽  
High Temporal Resolution ◽  
Control Mechanisms ◽  
Attentional Orienting ◽  
Behavioral Studies

The contents of working memory (WM) guide visual attention toward matching features, with visual search being faster when the target and a feature of an item held in WM spatially overlap (validly cued) than when they occur at different locations (invalidly cued). Recent behavioral studies have indicated that attentional capture by WM content can be modulated by cognitive control: When WM cues are reliably helpful to visual search (predictably valid), capture is enhanced, but when reliably detrimental (predictably invalid), capture is attenuated. The neural mechanisms underlying this effect are not well understood, however. Here, we leveraged the high temporal resolution of ERPs time-locked to the onset of the search display to determine how and at what processing stage cognitive control modulates the search process. We manipulated predictability by grouping trials into unpredictable (50% valid/invalid) and predictable (100% valid, 100% invalid) blocks. Behavioral results confirmed that predictability modulated WM-related capture. Comparison of ERPs to the search arrays showed that the N2pc, a posteriorly distributed signature of initial attentional orienting toward a lateralized target, was not impacted by target validity predictability. However, a longer latency, more anterior, lateralized effect—here, termed the “contralateral attention-related negativity”—was reduced under predictable conditions. This reduction interacted with validity, with substantially greater reduction for invalid than valid trials. These data suggest cognitive control over attentional capture by WM content does not affect the initial attentional-orienting process but can reduce the need to marshal later control mechanisms for processing relevant items in the visual world.

scholarly journals Towards a decision support tool for intensive care discharge: machine learning algorithm development using electronic healthcare data from MIMIC-III and Bristol, UK

BMJ Open ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. e025925 ◽  
Author(s):  
Christopher J McWilliams ◽  
Daniel J Lawson ◽  
Raul Santos-Rodriguez ◽  
Iain D Gilchrist ◽  
Alan Champneys ◽  
...  
Keyword(s):  
Intensive Care ◽  
Decision Support ◽  
Medical Information ◽  
Learning Algorithm ◽  
Adverse Outcomes ◽  
Royal Infirmary ◽  
Automated Method ◽  
Improved Performance ◽  
Discharge Criteria ◽  
Mimic Iii

ObjectiveThe primary objective is to develop an automated method for detecting patients that are ready for discharge from intensive care.DesignWe used two datasets of routinely collected patient data to test and improve on a set of previously proposed discharge criteria.SettingBristol Royal Infirmary general intensive care unit (GICU).PatientsTwo cohorts derived from historical datasets: 1870 intensive care patients from GICU in Bristol, and 7592 from Medical Information Mart for Intensive Care (MIMIC)-III.ResultsIn both cohorts few successfully discharged patients met all of the discharge criteria. Both a random forest and a logistic classifier, trained using multiple-source cross-validation, demonstrated improved performance over the original criteria and generalised well between the cohorts. The classifiers showed good agreement on which features were most predictive of readiness-for-discharge, and these were generally consistent with clinical experience. By weighting the discharge criteria according to feature importance from the logistic model we showed improved performance over the original criteria, while retaining good interpretability.ConclusionsOur findings indicate the feasibility of the proposed approach to ready-for-discharge classification, which could complement other risk models of specific adverse outcomes in a future decision support system. Avenues for improvement to produce a clinically useful tool are identified.

Comparison of Single-Trace and Multiple-Trace Polarity Determination for Surface Microseismic Data Using Deep Learning

2020 ◽  
Vol 91 (3) ◽  
pp. 1794-1803
Author(s):  
Xiao Tian ◽  
Wei Zhang ◽  
Xiong Zhang ◽  
Jie Zhang ◽  
Qingshan Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Learning Algorithm ◽  
Microseismic Monitoring ◽  
P Wave ◽  
Prediction Errors ◽  
Focal Mechanism Solutions ◽  
Deep Learning Algorithm ◽  
Training Samples ◽  
Polarity Determination ◽  
Single Trace

Abstract For surface microseismic monitoring, determination of the P-wave first-motion polarity is important because (1) it has been widely used to determine focal mechanisms and (2) the location accuracy of the diffraction-stack-based method is improved greatly using polarization correction. The convolutional neural network (CNN) is a form of deep learning algorithm that can be applied to predict the polarity of a seismogram automatically. However, the existing network designed for polarity detection utilizes only individual trace information. In this study, we design a multitrace-based CNN (MT-CNN) architecture using several neighbor traces combined as training samples, which could utilize the polarity information of neighbor sensors in the surface microseismic array. We use 17,227 field seismograms with labeled polarities to train two different neural networks that predict the polarities by a single trace or by multiple traces. The performance of the test set and field example of two CNN architectures shows that the MT-CNN significantly produces fewer polarity prediction errors and leads to more accurate focal mechanism solutions for microseismic events.

scholarly journals The Effect of Parkinson's Disease on the Dynamics of On-line and Proactive Cognitive Control during Action Selection

2010 ◽  
Vol 22 (9) ◽  
pp. 2058-2073 ◽  
Author(s):  
Scott A. Wylie ◽  
K. Richard Ridderinkhof ◽  
Theodore R. Bashore ◽  
Wery P. M. van den Wildenberg
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Basal Ganglia ◽  
Cognitive Control ◽  
Incorrect Response ◽  
Motor Symptom ◽  
Control Mechanisms ◽  
Proactive Control ◽  
On Line ◽  
Response Activation

Processing irrelevant visual information sometimes activates incorrect response impulses. The engagement of cognitive control mechanisms to suppress these impulses and make proactive adjustments to reduce the future impact of incorrect impulses may rely on the integrity of frontal–basal ganglia circuitry. Using a Simon task, we investigated the effects of basal ganglia dysfunction produced by Parkinson's disease (PD) on both on-line (within-trial) and proactive (between-trial) control efforts to reduce interference produced by the activation of an incorrect response. As a novel feature, we applied distributional analyses, guided by the activation–suppression model, to differentiate the strength of incorrect response activation and the proficiency of suppression engaged to counter this activation. For situations requiring on-line control, PD (n = 52) and healthy control (n = 30) groups showed similar mean interference effects (i.e., Simon effects) on reaction time (RT) and accuracy. Distributional analyses showed that although the strength of incorrect response impulses was similar between the groups PD patients were less proficient at suppressing these impulses. Both groups demonstrated equivalent and effective proactive control of response interference on mean RT and accuracy rates. However, PD patients were less effective at reducing the strength of incorrect response activation proactively. Among PD patients, motor symptom severity was associated with difficulties in on-line, but not in proactive, control of response impulses. These results suggest that basal ganglia dysfunction produced by PD has selective effects on cognitive control mechanisms engaged to resolve response conflict, with primary deficits in the on-line suppression of incorrect responses occurring in the context of a relatively spared ability to adjust control proactively to minimize future conflict.

scholarly journals Neural Mechanisms of Reward Prediction Error in Autism Spectrum Disorder

2019 ◽  
Vol 2019 ◽  
pp. 1-10 ◽  
Author(s):  
Maya G. Mosner ◽  
R. Edward McLaurin ◽  
Jessica L. Kinard ◽  
Shabnam Hakimi ◽  
Jacob Parelman ◽  
...  
Keyword(s):  
Prediction Error ◽  
Autism Spectrum ◽  
Neural Mechanisms ◽  
Reward Learning ◽  
Prediction Errors ◽  
Frontal Pole ◽  
Neural Basis ◽  
Reward Prediction ◽  
Behavioral Impairments ◽  
Adults With Asd

Few studies have explored neural mechanisms of reward learning in ASD despite evidence of behavioral impairments of predictive abilities in ASD. To investigate the neural correlates of reward prediction errors in ASD, 16 adults with ASD and 14 typically developing controls performed a prediction error task during fMRI scanning. Results revealed greater activation in the ASD group in the left paracingulate gyrus during signed prediction errors and the left insula and right frontal pole during thresholded unsigned prediction errors. Findings support atypical neural processing of reward prediction errors in ASD in frontostriatal regions critical for prediction coding and reward learning. Results provide a neural basis for impairments in reward learning that may contribute to traits common in ASD (e.g., intolerance of unpredictability).

Sign in / Sign up

Export Citation Format

Share Document