On the reliability of behavioral measures of cognitive control: retest reliability of task-inhibition effect, task-preparation effect, Stroop-like interference, and conflict adaptation effect

AbstractThis study examined the reliability (retest and split-half) of four common behavioral measures of cognitive control. In Experiment 1 (N = 96), we examined N – 2 task repetition costs as a marker of task-level inhibition, and the cue-stimulus interval (CSI) effect as a marker of time-based task preparation. In Experiment 2 (N = 48), we examined a Stroop-like face-name interference effect as a measure of distractor interference control, and the sequential congruency effect (“conflict adaptation effect”) as a measure of conflict-triggered adaptation of cognitive control. In both experiments, the measures were assessed in two sessions on the same day, separated by a 10 min-long unrelated filler task. We observed substantial experimental effects with medium to large effect sizes. At the same time, split-half reliabilities were moderate, and retest reliabilities were poor, for most measures, except for the CSI effect. Retest reliability of the Stroop-like effect was improved when considering only trials preceded by congruent trials. Together, the data suggest that these cognitive control measures are well suited for assessing group-level effects of cognitive control. Yet, except for the CSI effect, these measures do not seem suitable for reliably assessing interindividual differences in the strength of cognitive control, and therefore are not suited for correlational approaches. We discuss possible reasons for the discrepancy between robustness at the group level and reliability at the level of interindividual differences.

Download Full-text

Dynamic Adjustments of Cognitive Control: Oscillatory Correlates of the Conflict Adaptation Effect

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_00474 ◽

2013 ◽

Vol 25 (12) ◽

pp. 2167-2178 ◽

Cited By ~ 37

Author(s):

Bernhard Pastötter ◽

Gesine Dreisbach ◽

Karl-Heinz T. Bäuml

Keyword(s):

Cognitive Control ◽

Adaptation Effect ◽

Previous Trial ◽

Conflict Adaptation ◽

Response Priming ◽

Theta Power ◽

Current Trial ◽

Beta Power ◽

A Current ◽

Adaptation Effects

It is a prominent idea that cognitive control mediates conflict adaptation, in that response conflict in a previous trial triggers control adjustments that reduce conflict in a current trial. In the present EEG study, we investigated the dynamics of cognitive control in a response-priming task by examining the effects of previous trial conflict on intertrial and current trial oscillatory brain activities, both on the electrode and the source level. Behavioral results showed conflict adaptation effects for RTs and response accuracy. Physiological results showed sustained intertrial effects in left parietal theta power, originating in the left inferior parietal cortex, and midcentral beta power, originating in the left and right (pre)motor cortex. Moreover, physiological analysis revealed a current trial conflict adaptation effect in midfrontal theta power, originating in the ACC. Correlational analyses showed that intertrial effects predicted conflict-induced midfrontal theta power in currently incongruent trials. In addition, conflict adaptation effects in midfrontal theta power and RTs were positively related. Together, these findings point to a dynamic cognitive control system that, as a function of previous trial type, up- and down-regulates attention and preparatory motor activities in anticipation of the next trial.

Download Full-text

The conflict adaptation effect in Parkinson's disease: a study with EEG

Klinische Neurophysiologie ◽

10.1055/s-0033-1337259 ◽

2013 ◽

Vol 44 (01) ◽

Author(s):

N Rustamov ◽

R Rodriguez-Raecke ◽

B Kopp ◽

L Timm ◽

R Dengler ◽

...

Keyword(s):

Parkinson’S Disease ◽

Parkinson's Disease ◽

Adaptation Effect ◽

Conflict Adaptation

Download Full-text

Right Ventrolateral Prefrontal Cortex Mediates Individual Differences in Conflict-driven Cognitive Control

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_00064 ◽

2011 ◽

Vol 23 (12) ◽

pp. 3903-3913 ◽

Cited By ~ 45

Author(s):

Tobias Egner

Keyword(s):

Conflict Resolution ◽

Individual Differences ◽

Cognitive Control ◽

Inferior Frontal Gyrus ◽

Group Analysis ◽

Individual Variability ◽

Primary Role ◽

Conflict Adaptation ◽

Traditional Group ◽

Behavioral Expression

Conflict adaptation—a conflict-triggered improvement in the resolution of conflicting stimulus or response representations—has become a widely used probe of cognitive control processes in both healthy and clinical populations. Previous fMRI studies have localized activation foci associated with conflict resolution to dorsolateral PFC (dlPFC). The traditional group analysis approach employed in these studies highlights regions that are, on average, activated during conflict resolution, but does not necessarily reveal areas mediating individual differences in conflict resolution, because between-subject variance is treated as noise. Here, we employed a complementary approach to elucidate the neural bases of variability in the proficiency of conflict-driven cognitive control. We analyzed two independent fMRI data sets of face–word Stroop tasks by using individual variability in the behavioral expression of conflict adaptation as the metric against which brain activation was regressed while controlling for individual differences in mean RT and Stroop interference. Across the two experiments, a replicable neural substrate of individual variation in conflict adaptation was found in ventrolateral PFC (vlPFC), specifically, in the right inferior frontal gyrus, pars orbitalis (BA 47). Unbiased regression estimates showed that variability in activity in this region accounted for ∼40% of the variance in behavioral expression of conflict adaptation across subjects, thus documenting a heretofore unsuspected key role for vlPFC in mediating conflict-driven adjustments in cognitive control. We speculate that vlPFC plays a primary role in conflict control that is supplemented by dlPFC recruitment under conditions of suboptimal performance.

Download Full-text

Learning from the Reliability Paradox: How Theoretically Informed Generative Models Can Advance the Social, Behavioral, and Brain Sciences

10.31234/osf.io/xr7y3 ◽

2020 ◽

Cited By ~ 4

Author(s):

Nathaniel Haines ◽

Peter D. Kvam ◽

Louis H. Irving ◽

Colin Smith ◽

Theodore P. Beauchaine ◽

...

Keyword(s):

Individual Differences ◽

Theory Development ◽

Generative Models ◽

Parameter Estimates ◽

Group Level ◽

Retest Reliability ◽

Individual Level ◽

Generative Modeling ◽

Behavioral Tasks ◽

Test Retest Reliability

Behavioral tasks (e.g., Stroop task) that produce replicable group-level effects (e.g., Stroop effect) often fail to reliably capture individual differences between participants (e.g., low test-retest reliability). This “reliability paradox” has led many researchers to conclude that most behavioral tasks cannot be used to develop and advance theories of individual differences. However, these conclusions are derived from statistical models that provide only superficial summary descriptions of behavioral data, thereby ignoring theoretically-relevant data-generating mechanisms that underly individual-level behavior. More generally, such descriptive methods lack the flexibility to test and develop increasingly complex theories of individual differences. To resolve this theory-description gap, we present generative modeling approaches, which involve using background knowledge to specify how behavior is generated at the individual level, and in turn how the distributions of individual-level mechanisms are characterized at the group level—all in a single joint model. Generative modeling shifts our focus away from estimating descriptive statistical “effects” toward estimating psychologically meaningful parameters, while simultaneously accounting for measurement error that would otherwise attenuate individual difference correlations. Using simulations and empirical data from the Implicit Association Test and Stroop, Flanker, Posner Cueing, and Delay Discounting tasks, we demonstrate how generative models yield (1) higher test-retest reliability estimates, and (2) more theoretically informative parameter estimates relative to traditional statistical approaches. Our results reclaim optimism regarding the utility of behavioral paradigms for testing and advancing theories of individual differences, and emphasize the importance of formally specifying and checking model assumptions to reduce theory-description gaps and facilitate principled theory development.

Download Full-text

Cognitive Control Deficits in Alcohol Dependence Are a Trait- and State-Dependent Biomarker: An ERP Study

Frontiers in Psychiatry ◽

10.3389/fpsyt.2020.606891 ◽

2020 ◽

Vol 11 ◽

Author(s):

Xiaohong Liu ◽

Hongliang Zhou ◽

Chenguang Jiang ◽

Yanling Xue ◽

Zhenhe Zhou ◽

...

Keyword(s):

Cognitive Control ◽

Alcohol Dependence ◽

Control Function ◽

Event Related Potential ◽

Oddball Paradigm ◽

Group Level ◽

Auditory Oddball ◽

State Dependent ◽

Simple Effect ◽

Time Point

Alcohol dependence (AD) presents cognitive control deficits. Event-related potential (ERP) P300 reflects cognitive control-related processing. The aim of this study was to investigate whether cognitive control deficits are a trait biomarker or a state biomarker in AD. Participants included 30 AD patients and 30 healthy controls (HCs). All participants were measured with P300 evoked by a three-stimulus auditory oddball paradigm at a normal state (time 1, i.e., just after the last alcohol intake) and abstinence (time 2, i.e., just after a 4-week abstinence). The results showed that for P3a and P3b amplitude, the interaction effect for group × time point was significant, the simple effect for group at time 1 level and time 2 level was significant, and the simple effect for time point at AD group level was significant; however, the simple effect for time point at HC group level was not significant. Above results indicated that compared to HCs, AD patients present reductions of P3a/3b amplitude, and after 4-week alcohol abstinence, although P3a/3b amplitudes were improved, they were still lower than those of HCs. For P3a and P3b latencies, no significant differences were observed. These findings conclude that AD patients present cognitive control deficits that are reflected by P3a/3b and that cognitive control deficits in AD are trait- and state-dependent. The implication of these findings is helpful to understand the psychological and neural processes for AD, and these findings suggest that improving the cognitive control function may impact the treatment effect for AD.

Download Full-text

Neural Connectivity Subtypes Predict Discrete Attentional-Bias Profiles Among Heterogeneous Anxiety Patients

Clinical Psychological Science ◽

10.1177/2167702620906149 ◽

2020 ◽

Vol 8 (3) ◽

pp. 491-505 ◽

Cited By ~ 2

Author(s):

Rebecca B. Price ◽

Adriene M. Beltz ◽

Mary L. Woody ◽

Logan Cummings ◽

Danielle Gilchrist ◽

...

Keyword(s):

Functional Connectivity ◽

Selective Attention ◽

Attentional Bias ◽

Sensory Processing ◽

Brain Regions ◽

Data Driven ◽

Neural Connectivity ◽

Group Level ◽

Behavioral Measures ◽

Substantial Heterogeneity

On average, anxious patients show altered attention to threat—including early vigilance toward threat and later avoidance of threat—accompanied by altered functional connectivity across brain regions. However, substantial heterogeneity within clinical, neural, and attentional features of anxiety is overlooked in typical group-level comparisons. We used a well-validated method for data-driven parsing of neural connectivity to reveal connectivity-based subgroups among 60 adults with transdiagnostic anxiety. Subgroups were externally compared on attentional patterns derived from independent behavioral measures. Two subgroups emerged. Subgroup A (68% of patients) showed stronger executive network influences on sensory processing regions and a paradigmatic “vigilance–avoidance” pattern on external behavioral measures. Subgroup B was defined by a larger number of limbic influences on sensory regions and exhibited a more atypical and inconsistent attentional profile. Neural connectivity-based categorization revealed an atypical, limbic-driven pattern of connectivity in a subset of anxious patients that generalized to atypical patterns of selective attention.

Download Full-text