Power-Enhanced Funnel Plots for Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 43-49 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Relevant Information ◽  
Selective Reporting ◽  
Graphical Display ◽  
Funnel Plot ◽  
Evidential Value ◽  
Graphical Displays ◽  
Funnel Plots ◽  
Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

Visual Inference for the Funnel Plot in Meta-Analysis

2019 ◽  
Vol 227 (1) ◽  
pp. 83-89 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Visual Inspection ◽  
Type I Error ◽  
Statistical Tests ◽  
Meta Analysis ◽  
Real Data ◽  
Type I ◽  
Funnel Plot ◽  
Funnel Plots ◽  
Meta Analyses ◽  
Open Nature

Abstract. The funnel plot is widely used in meta-analyses to assess potential publication bias. However, experimental evidence suggests that informal, mere visual, inspection of funnel plots is frequently prone to incorrect conclusions, and formal statistical tests (Egger regression and others) entirely focus on funnel plot asymmetry. We suggest using the visual inference framework with funnel plots routinely, including for didactic purposes. In this framework, the type I error is controlled by design, while the explorative, holistic, and open nature of visual graph inspection is preserved. Specifically, the funnel plot of the actually observed data is presented simultaneously, in a lineup, with null funnel plots showing data simulated under the null hypothesis. Only when the real data funnel plot is identifiable from all the funnel plots presented, funnel plot-based conclusions might be warranted. Software to implement visual funnel plot inference is provided via a tailored R function.

scholarly journals A Registered Report of Error-Related Negativity and Reward Positivity as Biomarkers of Depression: P-Curving the Evidence

2019 ◽  
Author(s):  
Peter E Clayson ◽  
Kaylie Amanda Carbine ◽  
Michael J. Larson
Keyword(s):  
Statistical Power ◽  
Performance Monitoring ◽  
Meta Analysis ◽  
A Priori ◽  
Depression Symptoms ◽  
Selective Reporting ◽  
Evidential Value ◽  
Error Related Negativity ◽  
Reward Positivity ◽  
Size Heterogeneity

Performance-monitoring event-related brain potentials (ERPs), such as the error-related negativity (ERN) and reward positivity (RewP), are advocated as biomarkers of depression symptoms and risk. However, a recent meta-analysis indicated effect size heterogeneity in the ERN and RewP literatures. Hence, advocating these ERPs as biomarkers of depression might be premature or possibly misguided due to the selective reporting of significant analyses on the part of researchers (e.g., p-hacking or omission of non-significant findings). The present study quantified the degree of selective reporting and the evidential value for a true relationship between depression and ERN and RewP using a p-curve analysis. We predicted that the ERN and RewP literatures would fail to show evidential value for a relationship between each ERP and depression. Contrary to expectations, both literatures showed evidential value, albeit weak. The statistical power of the included ERN studies was between 20% and 25%, and the statistical power of the RewP was around 27%. Taken together, these findings provide support for a relationship between these ERPs and depression, which strengthens claims that these ERPs represent candidate biomarkers of depression symptoms and risk. In light of the evidence for these relationships being weak, some recommendations moving forward include conducting a priori power analyses, increasing sample sizes to improve statistical power, assessing the internal consistency of ERP scores, and carefully planning statistical approaches to maximize power.

scholarly journals Visual representations of meta-analyses of multiple outcomes: Extensions to forest plots, funnel plots, and caterpillar plots

Methodology ◽  
2020 ◽  
Vol 16 (4) ◽  
pp. 299-315
Author(s):  
Belén Fernández-Castilla ◽  
Lies Declercq ◽  
Laleh Jamshidi ◽  
Susan Natasha Beretvas ◽  
Patrick Onghena ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Effect Sizes ◽  
Forest Plot ◽  
Sampling Variance ◽  
Individual Effect ◽  
Outcome Reporting Bias ◽  
Funnel Plot ◽  
Multiple Effect ◽  
Funnel Plots ◽  
Meta Analyses

Meta-analytic datasets can be large, especially when in primary studies multiple effect sizes are reported. The visualization of meta-analytic data is therefore useful to summarize data and understand information reported in primary studies. The gold standard figures in meta-analysis are forest and funnel plots. However, none of these plots can yet account for the existence of multiple effect sizes within primary studies. This manuscript describes extensions to the funnel plot, forest plot and caterpillar plot to adapt them to three-level meta-analyses. For forest plots, we propose to plot the study-specific effects and their precision, and to add additional confidence intervals that reflect the sampling variance of individual effect sizes. For caterpillar plots and funnel plots, we recommend to plot individual effect sizes and averaged study-effect sizes in two separate graphs. For the funnel plot, plotting separate graphs might improve the detection of both publication bias and/or selective outcome reporting bias.

scholarly journals Most oxytocin administration studies are statistically underpowered to reliably detect (or reject) a wide range of effect sizes

2020 ◽  
Author(s):  
Daniel S Quintana
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Evidential Value ◽  
Statistical Equivalence ◽  
Wide Range ◽  
Noticeable Improvement ◽  
Meta Analyses ◽  
Study Designs ◽  
Analytical Approaches

The neuropeptide oxytocin has attracted substantial research interest for its role in behaviour and cognition; however, the evidence for its effects have been mixed. Meta-analysis is viewed as the gold-standard for synthesizing evidence, but the evidential value of a meta-analysis is dependent on the evidential value of the studies it synthesizes, and the analytical approaches used to derive conclusions. To assess the evidential value of oxytocin administration meta-analyses, this study calculated the statistical power of 107 studies from 35 meta-analyses and assessed the statistical equivalence of reported results. The mean statistical power across all studies was 12.2% and there has been no noticeable improvement in power over an eight-year period. None of the 26 non-significant meta-analyses were statistically equivalent, assuming a smallest effect size of interest of 0.1. Altogether, most oxytocin treatment study designs are statistically underpowered to either detect or reject a wide range of effect sizes that scholars may find worthwhile.

Which Data to Meta-Analyze, and How?

2019 ◽  
Vol 227 (1) ◽  
pp. 64-82 ◽  
Author(s):  
Martin Voracek ◽  
Michael Kossmeier ◽  
Ulrich S. Tran
Keyword(s):  
Empirical Research ◽  
Meta Analysis ◽  
Curve Analysis ◽  
Research Progress ◽  
Primary Data ◽  
Statistical Procedures ◽  
Graphical Displays ◽  
Primary Data Analysis ◽  
Meta Analyses ◽  
Data Analytic

Abstract. Which data to analyze, and how, are fundamental questions of all empirical research. As there are always numerous flexibilities in data-analytic decisions (a “garden of forking paths”), this poses perennial problems to all empirical research. Specification-curve analysis and multiverse analysis have recently been proposed as solutions to these issues. Building on the structural analogies between primary data analysis and meta-analysis, we transform and adapt these approaches to the meta-analytic level, in tandem with combinatorial meta-analysis. We explain the rationale of this idea, suggest descriptive and inferential statistical procedures, as well as graphical displays, provide code for meta-analytic practitioners to generate and use these, and present a fully worked real example from digit ratio (2D:4D) research, totaling 1,592 meta-analytic specifications. Specification-curve and multiverse meta-analysis holds promise to resolve conflicting meta-analyses, contested evidence, controversial empirical literatures, and polarized research, and to mitigate the associated detrimental effects of these phenomena on research progress.

scholarly journals The Galaxy Plot: A New Visualization Tool for Bivariate Meta-Analysis Studies

2020 ◽  
Vol 189 (8) ◽  
pp. 861-869 ◽  
Author(s):  
Chuan Hong ◽  
Rui Duan ◽  
Lingzhen Zeng ◽  
Rebecca A Hubbard ◽  
Thomas Lumley ◽  
...  
Keyword(s):  
Dimensional Space ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Standard Errors ◽  
Visualization Tool ◽  
The Galaxy ◽  
Funnel Plots ◽  
Complex Features ◽  
Bivariate Outcomes ◽  
Meta Analyses

Abstract Funnel plots have been widely used to detect small-study effects in the results of univariate meta-analyses. However, there is no existing visualization tool that is the counterpart of the funnel plot in the multivariate setting. We propose a new visualization method, the galaxy plot, which can simultaneously present the effect sizes of bivariate outcomes and their standard errors in a 2-dimensional space. We illustrate the use of the galaxy plot with 2 case studies, including a meta-analysis of hypertension trials with studies from 1979–1991 (Hypertension. 2005;45(5):907–913) and a meta-analysis of structured telephone support or noninvasive telemonitoring with studies from 1966–2015 (Heart. 2017;103(4):255–257). The galaxy plot is an intuitive visualization tool that can aid in interpreting results of multivariate meta-analysis. It preserves all of the information presented by separate funnel plots for each outcome while elucidating more complex features that may only be revealed by examining the joint distribution of the bivariate outcomes.

Does Mindfulness Training Without Explicit Ethics-Based Instruction Promote Prosocial Behaviors? A Meta-Analysis

2020 ◽  
Vol 46 (8) ◽  
pp. 1247-1269 ◽  
Author(s):  
Daniel R. Berry ◽  
Jonathan P. Hoerr ◽  
Selena Cesko ◽  
Amir Alayoubi ◽  
Kevin Carpio ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Mindfulness Training ◽  
Selective Reporting ◽  
Single Session ◽  
Based Instruction ◽  
Meta Analyses ◽  
Scholarly Discourse ◽  
Contemplative Traditions ◽  
Difference Effect ◽  
Prosocial Action

Scholarly discourse has raised concerns about the gravitas of secular mindfulness trainings in promoting prosocial outgrowths, as these trainings lack ethics-based concepts found in contemplative traditions. Random-effects meta-analyses were conducted to test whether mindfulness trainings absent explicit ethics-based instructions promote prosocial action. There was a range of small to medium standardized mean difference effect sizes of mindfulness training on overt acts of prosociality when compared with active and inactive controls, k = 29, N = 3,100, g = .426, 95% confidence interval (CI)( g) = [.304, .549]. Reliable effect size estimates were found for single-session interventions that measured prosocial behavior immediately after training. Mindfulness training also reliably promotes compassionate (but not instrumental or generous) helping and reliably reduces prejudice and retaliation. Publication bias analyses indicated that the reliability of these findings was not wholly dependent on selective reporting. Implications for the science of secular mindfulness training on prosocial action are discussed.

scholarly journals META ANALISIS: PENGARUH ANEMIA IBU HAMIL TERHADAP BERAT BAYI LAHIR RENDAH

2020 ◽  
Vol 11 (2) ◽  
pp. 163-177
Author(s):  
Aditianti Aditianti ◽  
Sri Poedji Hastoety Djaiman
Keyword(s):  
Pregnant Women ◽  
Young Women ◽  
Odds Ratio ◽  
Health Workers ◽  
Meta Analysis ◽  
Random Effect ◽  
Forest Plot ◽  
Funnel Plot ◽  
Funnel Plots

Abstract Background: The prevalence of low birth weight (LBW) in Indonesia shows a decrease, but the risk factor for anemia in pregnant women has increased sharply and this has an impact on increasing the prevalence of LBW. Objective: This study aimed to determine the risk of anemia in pregnant women to the prevalence of LBW in several countries. Methods: This study was a meta-analysis using PRISMA. Eleven of the 122,000 studies met criteria for the analysis. Presentation of the data used a forest plot with a random effect statistical model. Results: The combined odds ratio (OR) showed that the effect of anemia in pregnant women on LBW was 1.49 times higher than that of non-anemia mothers (95% CI: 1.26-4.60; p <0.001). The variance was 53,7%. The results of the funnel plots from 11 studies were not evenly distributed so that the information obtained was homogeneous, focusing more on the middle value. Conclusion: There was an effect of anemia in pregnant women with the prevalence of LBW. Detection of anemia in pregnant women needs to be done as early as possible by involving the role of health workers and cadres. Outreach activities for young women at schools and Posyandu must be carried out regularly and continuously.   Keywords: Anemia, LBW, Pregnancy     Abstrak Latar belakang: Prevalensi berat bayi lahir rendah (BBLR ) di Indonesia menunjukkan penurunan namun faktor risiko anemia pada ibu hamil meningkat tajam dan hal ini berdampak pada peningkatan kejadian BBLR. Tujuan: Studi ini bertujuan untuk mengetahui besarnya risiko ibu hamil anemia terhadap kejadian BBLR di beberapa negara. Metode: Studi ini merupakan meta analisis menggunakan PRISMA. Sebelas dari 122.000 studi masuk dalam kriteria untuk dianalisis. Penyajian data menggunakan forest plot dengan model statistik random effect. Hasil: Besar odds ratio (OR) gabungan menunjukkan bahwa pengaruh ibu hamil anemia terhadap BBLR 1,49 kali lebih tinggi dibandingkan ibu yang tidak anemia (95%CI: 1,26-4,60; p<0,001). Besarnya varian 53,7 persen. Hasil funnel plot dari 11 studi ini tidak tersebar secara merata sehingga informasi yang diperoleh homogen, lebih fokus pada nilai tengah. Kesimpulan: Terdapat pengaruh anemia pada ibu hamil dengan kejadian BBLR. Deteksi anemia pada ibu hamil perlu dilakukan sedini mungkin dengan melibatkan peran tenaga keseharan dan kader. Penyuluhan bagi remaja putri di sekolah dan posyandu harus dilakukan secara berkala dan berkesinambungan   Kata kunci: Anemia, BBLR, Kehamilan

scholarly journals Quantifying the advantages of conducting a prospective meta-analysis (PMA): a case study of early childhood obesity prevention

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
A. L. Seidler ◽  
◽  
K. E. Hunter ◽  
D. Espinoza ◽  
S. Mihrshahi ◽  
...  
Keyword(s):  
Early Childhood ◽  
Childhood Obesity ◽  
External Validity ◽  
Statistical Power ◽  
Meta Analysis ◽  
Intervention Design ◽  
Intervention Effects ◽  
Early Childhood Obesity ◽  
Limited Power ◽  
Meta Analyses

Abstract Background For prospective meta-analyses (PMAs), eligible studies are identified, and the PMA hypotheses, selection criteria, and analysis methods are pre-specified before the results of any of the studies are known. This reduces publication bias and selective outcome reporting and provides a unique opportunity for outcome standardisation/harmonisation. We conducted a world-first PMA of four trials investigating interventions to prevent early childhood obesity. The aims of this study were to quantitatively analyse the effects of prospective planning on variations across trials, outcome harmonisation, and the power to detect intervention effects, and to derive recommendations for future PMA. Methods We examined intervention design, participant characteristics, and outcomes collected across the four trials included in the EPOCH PMA using their registration records, protocol publications, and variable lists. The outcomes that trials planned to collect prior to inclusion in the PMA were compared to the outcomes that trials collected after PMA inclusion. We analysed the proportion of matching outcome definitions across trials, the number of outcomes per trial, and how collaboration increased the statistical power to detect intervention effects. Results The included trials varied in intervention design and participants, this improved external validity and the ability to perform subgroup analyses for the meta-analysis. While individual trials had limited power to detect the main intervention effect (BMI z-score), synthesising data substantially increased statistical power. Prospective planning led to an increase in the number of collected outcome categories (e.g. weight, child’s diet, sleep), and greater outcome harmonisation. Prior to PMA inclusion, only 18% of outcome categories were included in all trials. After PMA inclusion, this increased to 91% of outcome categories. However, while trials mostly collected the same outcome categories after PMA inclusion, some inconsistencies in how the outcomes were measured remained (such as measuring physical activity by hours of outside play versus using an activity monitor). Conclusion Prospective planning led to greater outcome harmonisation and greater power to detect intervention effects, while maintaining acceptable variation in trial designs and populations, which improved external validity. Recommendations for future PMA include more detailed harmonisation of outcome measures and careful pre-specification of analyses to avoid research waste by unnecessary over-collection of data.

Systematic reviews and meta-analysis of prognosis research studies

2019 ◽  
pp. 208-244
Author(s):  
Richard D Riley ◽  
Karel GM Moons ◽  
Thomas PA Debray ◽  
Douglas G Altman ◽  
Gary S Collins
Keyword(s):  
Prognostic Factor ◽  
Systematic Reviews ◽  
Clinical Decision Making ◽  
Meta Analysis ◽  
Clinical Decision ◽  
Relevant Information ◽  
Included Study ◽  
Prognosis Research ◽  
Meta Analyses ◽  
Research Studies

Systematic reviews and meta-analyses identify, evaluate, and summarize prognosis research studies and their findings. The chapter provides a guide to the key components and methods for conducting a systematic review and meta-analysis for each of the four types of prognosis studies. The CHARMS checklist is introduced as a guide to identifying clear review objectives and design, and to extracting the relevant information from each included study. Many existing prognosis studies are at high risk of bias, because (for example) of selective recruitment and reporting. Tools for examining quality of studies are discussed—the QUIPS for prognostic factor research and PROBAST for prognostic model research. The statistical principles of meta-analysis are described, and the key statistics that can be synthesized are outlined. Challenges are identified, such as the potential for publication bias and substantial heterogeneity in published prognostic factor cut points and methods of prognostic factor measurement. Despite these challenges the chapter emphasizes the crucial importance of prognosis reviews for evidence-based guidelines and clinical decision making.

Sign in / Sign up

Export Citation Format

Share Document