scholarly journals A Registered Report of Error-Related Negativity and Reward Positivity as Biomarkers of Depression: P-Curving the Evidence

2019 ◽  
Author(s):  
Peter E Clayson ◽  
Kaylie Amanda Carbine ◽  
Michael J. Larson
Keyword(s):  
Statistical Power ◽  
Performance Monitoring ◽  
Meta Analysis ◽  
A Priori ◽  
Depression Symptoms ◽  
Selective Reporting ◽  
Evidential Value ◽  
Error Related Negativity ◽  
Reward Positivity ◽  
Size Heterogeneity

Performance-monitoring event-related brain potentials (ERPs), such as the error-related negativity (ERN) and reward positivity (RewP), are advocated as biomarkers of depression symptoms and risk. However, a recent meta-analysis indicated effect size heterogeneity in the ERN and RewP literatures. Hence, advocating these ERPs as biomarkers of depression might be premature or possibly misguided due to the selective reporting of significant analyses on the part of researchers (e.g., p-hacking or omission of non-significant findings). The present study quantified the degree of selective reporting and the evidential value for a true relationship between depression and ERN and RewP using a p-curve analysis. We predicted that the ERN and RewP literatures would fail to show evidential value for a relationship between each ERP and depression. Contrary to expectations, both literatures showed evidential value, albeit weak. The statistical power of the included ERN studies was between 20% and 25%, and the statistical power of the RewP was around 27%. Taken together, these findings provide support for a relationship between these ERPs and depression, which strengthens claims that these ERPs represent candidate biomarkers of depression symptoms and risk. In light of the evidence for these relationships being weak, some recommendations moving forward include conducting a priori power analyses, increasing sample sizes to improve statistical power, assessing the internal consistency of ERP scores, and carefully planning statistical approaches to maximize power.

Power-Enhanced Funnel Plots for Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 43-49 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Relevant Information ◽  
Selective Reporting ◽  
Graphical Display ◽  
Funnel Plot ◽  
Evidential Value ◽  
Graphical Displays ◽  
Funnel Plots ◽  
Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

Replication of the Superstition and Performance Study by

2014 ◽  
Vol 45 (3) ◽  
pp. 239-245 ◽  
Author(s):  
Robert J. Calin-Jageman ◽  
Tracy L. Caldwell
Keyword(s):  
Task Difficulty ◽  
Statistical Power ◽  
Meta Analysis ◽  
A Priori ◽  
Significant Heterogeneity ◽  
Performance Study ◽  
Improve Performance ◽  
Research Designs ◽  
Series Of Experiments ◽  
And Performance

A recent series of experiments suggests that fostering superstitions can substantially improve performance on a variety of motor and cognitive tasks ( Damisch, Stoberock, & Mussweiler, 2010 ). We conducted two high-powered and precise replications of one of these experiments, examining if telling participants they had a lucky golf ball could improve their performance on a 10-shot golf task relative to controls. We found that the effect of superstition on performance is elusive: Participants told they had a lucky ball performed almost identically to controls. Our failure to replicate the target study was not due to lack of impact, lack of statistical power, differences in task difficulty, nor differences in participant belief in luck. A meta-analysis indicates significant heterogeneity in the effect of superstition on performance. This could be due to an unknown moderator, but no effect was observed among the studies with the strongest research designs (e.g., high power, a priori sampling plan).

scholarly journals Noise in the process: an assessment of the evidential value of mediation effects in marketing journals

2021 ◽  
Author(s):  
Aaron Charlton ◽  
Amanda Kay Montoya ◽  
John Price ◽  
Joseph Hilgard
Keyword(s):  
Low Power ◽  
Multiple Testing ◽  
Statistical Power ◽  
A Priori ◽  
Marketing Research ◽  
Real World Data ◽  
Mediation Effects ◽  
Mediation Analyses ◽  
Evidential Value ◽  
Adequate Power

Mediation analysis plays a central role in marketing research due to its usefulness in helping to explain complex processes. Like other forms of inference, mediation analyses are susceptible to false positive results. This is particularly true when analytic decisions are based on the data, rather than a priori hypotheses. To assess the collective evidential value of mediation analyses in marketing, we used an approach first implemented by Götz and colleagues (2021) that (1) measures the relative proximity of confidence intervals to zero (RP) and (2) aggregates a related set of RP scores into a single distribution. For our analysis, we compared the RP distribution of top marketing journals (2018-20) to simulations of low power, adequate power, and null effects. We also compared the marketing journals to real-world data from Journal of Personality and Social Psychology (JPSP) (2018-20). We found that, in terms of evidential value, mediation analyses in marketing substantially deviated from simulations of adequate power and JPSP but were similar to simulations of low power and null effects. We propose study preregistration, corrections for multiple testing, and increased statistical power as solutions to increase evidence quality going forward.

scholarly journals Mindfulness meditators do not show differences in electrophysiological measures of error processing

2018 ◽  
Author(s):  
Neil W Bailey ◽  
Kavya Raj ◽  
Gabrielle Freedman ◽  
Bernadette M Fitzgibbon ◽  
Nigel C Rogasch ◽  
...  
Keyword(s):  
Statistical Power ◽  
Performance Monitoring ◽  
Bayes Factor ◽  
Mindfulness Meditation ◽  
Self Regulation ◽  
Medium Effect ◽  
Error Processing ◽  
Meditation Practice ◽  
Error Related Negativity ◽  
Alternative Hypotheses

AbstractMindfulness meditation may improve attention and self-regulation. One component of attention and self-regulation that may allow these improvements is performance monitoring. Neural correlates of performance monitoring can be objectively measured with electroencephalogram (EEG) via the error related negativity (ERN) and error positivity (Pe). Previous research assessing the ERN and Pe in meditators has resulted in inconsistent findings; some have reported alteration in peak amplitudes from both very brief meditation practice and long-term meditation practice, while others have failed to provide evidence for differences in the ERN or Pe. However, recently developed EEG analysis techniques allow for more rigorous analyses than have been used in past investigations. The current study measured the ERN and Pe, as well as post-error alpha suppression, during a Go/Nogo task, and emotional and colour Stroop tasks. The measures were compared between 22 experienced meditators (mean of 8 years of practice) and 20 healthy controls. The results suggested no differences in the ERN, Pe, or post-error alpha suppression (all p > 0.05), even when varying multiple analysis parameters. The study showed equivalent statistical power to previous research, and > 85% power to detect medium effect sizes present in previous research. Bayes Factor analysis indicated the null hypotheses were > 3.5 more likely than any of the alternative hypotheses for the ERN or Pe. These results suggest that meditation may not alter neural activity related to error processing, despite prior research suggesting that it does.

scholarly journals Differentiating electrophysiological indices of internal and external performance monitoring: Relationship with perfectionism and locus of control

2019 ◽  
Author(s):  
Alexandra M. Muir ◽  
Kaylie A. Carbine ◽  
Jayden Goodwin ◽  
Ariana Hedges-Muncy ◽  
Tanja Endrass ◽  
...  
Keyword(s):  
Locus Of Control ◽  
Personality Traits ◽  
Individual Difference ◽  
Performance Monitoring ◽  
Future Research ◽  
Control Scale ◽  
Error Related Negativity ◽  
Reward Positivity ◽  
And Gender ◽  
The Impact

AbstractThe impact of individual differences on performance monitoring and psychopathology is a question of active debate. Personality traits associated with psychopathology may be related to poor internal performance monitoring (as measured by the error-related negativity [ERN]) but intact external performance monitoring (as measured by the reward positivity [RewP]), suggesting that there are underlying neural differences between internal and external performance monitoring processes. We tested the relationships between individual difference measures of perfectionism, locus of control, and ERN, Pe, and RewP component difference amplitude in a healthy undergraduate sample. A total of 128 participants (69 female, M(SD)age= 20.6(2.0) years) completed two tasks: a modified version of the Eriksen Flanker and a doors gambling task along with the Frost Multidimensional Perfectionism scale, the Rotter Locus of Control scale, and the Levenson Multidimensional Locus of Control scale to quantify perfectionism and locus of control traits, respectively. Linear regressions adjusting for age and gender showed that neither ΔERN nor ΔRewP amplitude were significantly moderated by perfectionism or locus of control scores. Findings suggest that, in psychiatrically-healthy individuals, there is not a strong link between perfectionism, locus of control, and ERN or RewP amplitude. Future research on individual difference measures in people with psychopathology may provide further insight into how these personality traits affect performance monitoring.

Drug-eluting stents in patients at high risk of restenosis: Assessment for France

2011 ◽  
Vol 27 (2) ◽  
pp. 108-117 ◽  
Author(s):  
Emmanuelle Schapiro-Dufour ◽  
Michel Cucherat ◽  
Elodie Velzenberger ◽  
Hubert Galmiche ◽  
Catherine Denis ◽  
...  
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
A Priori ◽  
Bare Metal Stents ◽  
Vessel Diameter ◽  
Target Lesion ◽  
Drug Eluting Stents ◽  
Metal Stents ◽  
Patient Category ◽  
Drug Eluting

Background: In unselected patients, the incidence of restenosis is lower after placement of drug-eluting stents (DES) than bare-metal stents (BMS) without difference in safety at a time horizon of 4 years. However, DES appears less effective in “off label” patients.Objectives: The aim of the study was to assess available evidence of DES efficacy and safety by patient category to establish when DES placement may be recommended for reimbursement by the French national health insurance.Methods: Based on a systematic review by patient category (January 2002 to August 2009), two health technology assessment (HTA) reports and thirty-eight clinical studies not covered by the HTA reports (eleven meta-analysis including ours, eleven randomized trials and sixteen cohort studies) were selected. After assessment of the methodological quality, the studies mostly comparing DES with BMS were reviewed by a panel of health professionals who defined a priori the most relevant end points of safety and efficacy.Results: Seven to fourteen patients treated with DES were needed to avoid one target lesion revascularization (TLR) in patients with lesions >15 mm long, vessel diameter <3 mm, or diabetes, and with some complex lesions (total coronary occlusion, BMS in-stent restenosis multivessel disease, unprotected left main stenosis). DES appeared as safe as other alternatives over a follow-up of up to 4 years when dual antiplatelet therapy was continued for at least 1 year, but statistical power remains limited to conclude for some clinical features.Conclusions: For reimbursement, DES use should be limited to certain categories of patients. Treatment of particular cases requires a multidisciplinary approach.

scholarly journals Evidential Value That Exercise Improves BMIz-Score in Overweight and Obese Children and Adolescents

2015 ◽  
Vol 2015 ◽  
pp. 1-5 ◽  
Author(s):  
George A. Kelley ◽  
Kristi S. Kelley
Keyword(s):  
Children And Adolescents ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Selective Reporting ◽  
Obese Children ◽  
Evidential Value ◽  
Chi Squared ◽  
Using Data ◽  
Exercise Induced ◽  
Rule Out

Background.Given the cardiovascular disease (CVD) related importance of understanding the true effects of exercise on adiposity in overweight and obese children and adolescents, this study examined whether there is evidential value to rule out excessive and inappropriate reporting of statistically significant results, a major problem in the published literature, with respect to exercise-induced improvements in BMIz-score among overweight and obese children and adolescents.Methods.Using data from a previous meta-analysis of 10 published studies that included 835 overweight and obese children and adolescents, a novel, recently developed approach (p-curve) was used to test for evidential value and rule out selective reporting of findings. Chi-squared tests (χ2) were used to test for statistical significance with alpha (p) values <0.05 considered statistically significant.Results.Six of 10 findings (60%) were statistically significant. Statistically significant right-skew to rule out selective reporting was found (χ2= 38.8,p= 0.0001). Conversely, studies neither lacked evidential value (χ2= 6.8,p= 0.87) nor lacked evidential value and were intenselyp-hacked (χ2= 4.3,p= 0.98).Conclusion.Evidential value results confirm that exercise reduces BMIz-score in overweight and obese children and adolescents, an important therapeutic strategy for treating and preventing CVD.

Response-locked component of error monitoring in psychopathy: A systematic review and meta-analysis of error-related negativity/positivity

2020 ◽  
Author(s):  
William Vallet ◽  
Cecilia Neige ◽  
Sabine MOUCHET-MAGES ◽  
Jerome Brunelin ◽  
Simon Grondin
Keyword(s):  
Systematic Review ◽  
Performance Monitoring ◽  
Meta Analysis ◽  
Factorial Analysis ◽  
Error Monitoring ◽  
Affective Dimension ◽  
Error Related Negativity ◽  
Error Positivity ◽  
Following Error ◽  
Analysis Of Error

Background: Evidence suggests that individuals with psychopathy display difficulties to adapt their behavior in accordance with the demands of the environment and show altered performance monitoring. However, studies investigating electrophysiological markers of error monitoring (e.g., the error-related negativity (ERN) and the error-positivity (Pe)) in this population reported mixed results. To explain discrepancies observed between studies, we hypothesized that psychopathy dimensions influence electrophysiological outcomes and we predicted that individuals with impulsive-antisocial features would display abnormal ERN compared to individuals with interpersonal-affective features. Methods: Based on the PRISMA guidelines, we conducted a systematic review and meta-analysis of studies investigating ERN and Pe components in individuals with psychopathy compared to controls. A factorial analysis was undertaken to investigate the role of psychopathy dimensions on ERN. Results: Among the 206 retrieved studies, 15 were included in the meta- analysis. Individuals with psychopathy (n = 817) showed a reduced ERN (Cohen's d = 0.18) and Pe amplitude (d = -0.22) compared to control. The factorial analysis indicates a dissociation regarding the dimensional construct of psychopathy. The impulsive-antisocial dimension was linked to reduced ERN amplitude (d = 0.22) whereas the interpersonal-affective dimension was related to increased ERN amplitude compare to controls (d = -0.17). Conclusion: Individuals with psychopathy displayed abnormal ERN and Pe amplitudes following error commission. In addition, models reported that individuals with psychopathic traits relating more specifically to the interpersonal-affective dimension shows efficient error-monitoring systems and increased ERN component while those with marked impulsive-antisocial dimension displayed decreased ERN and altered performance monitoring.

scholarly journals Assessing and adjusting for publication bias in the relationship between anxiety and the error-related negativity

2018 ◽  
Author(s):  
Blair Saunders ◽  
Michael Inzlicht
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Meta Analysis ◽  
Event Related Potential ◽  
Medium Effect ◽  
Future Research ◽  
Error Related Negativity ◽  
Trim And Fill ◽  
Neural Reactivity

Many clinical neuroscience investigations have suggested that trait anxiety is associated with increased neural reactivity to mistakes in the form of an event-related potential called the error-related negativity (ERN). Several recent meta-analyses indicated that the anxiety-ERN association was of a small-to-medium effect size, however, these prior investigations did not comprehensively adjust effect sizes for publication bias. Here, in an updated meta-analysis (k=58, N=3819), we found support for an uncorrected effect size of r =-.19, and applied a range of methods to test for and correct publication bias (trim-and-fill, PET, PEESE, Peters’ test, three-parameter selection model). The majority of bias-correction methods suggested that the correlation between anxiety and the ERN is non-zero, but smaller than the uncorrected effect size (average adjusted effect size: r =-.12, range: r =-.05 to -.18). Moderation analyses also revealed more robust effects for clinical anxiety and anxious samples characterised by worry, however, it should be noted that these larger effects were also associated with elevated indicators of publication bias relative to the overall analysis. Mixed anxiety and sub-clinical anxiety were not associated with the amplitude of the ERN. Our results suggest that the anxiety-ERN relationship survives multiple corrections for publication bias, albeit not among all sub-types and populations of anxiety. Nevertheless, only 50% of the studies included in our analysis reported significant results, indicating that future research exploring the anxiety-ERN relationship would benefit from increased statistical power.

scholarly journals Most oxytocin administration studies are statistically underpowered to reliably detect (or reject) a wide range of effect sizes

2020 ◽  
Author(s):  
Daniel S Quintana
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Evidential Value ◽  
Statistical Equivalence ◽  
Wide Range ◽  
Noticeable Improvement ◽  
Meta Analyses ◽  
Study Designs ◽  
Analytical Approaches

The neuropeptide oxytocin has attracted substantial research interest for its role in behaviour and cognition; however, the evidence for its effects have been mixed. Meta-analysis is viewed as the gold-standard for synthesizing evidence, but the evidential value of a meta-analysis is dependent on the evidential value of the studies it synthesizes, and the analytical approaches used to derive conclusions. To assess the evidential value of oxytocin administration meta-analyses, this study calculated the statistical power of 107 studies from 35 meta-analyses and assessed the statistical equivalence of reported results. The mean statistical power across all studies was 12.2% and there has been no noticeable improvement in power over an eight-year period. None of the 26 non-significant meta-analyses were statistically equivalent, assuming a smallest effect size of interest of 0.1. Altogether, most oxytocin treatment study designs are statistically underpowered to either detect or reject a wide range of effect sizes that scholars may find worthwhile.

Sign in / Sign up

Export Citation Format

Share Document