scholarly journals Visual representations of meta-analyses of multiple outcomes: Extensions to forest plots, funnel plots, and caterpillar plots

Methodology ◽  
2020 ◽  
Vol 16 (4) ◽  
pp. 299-315
Author(s):  
Belén Fernández-Castilla ◽  
Lies Declercq ◽  
Laleh Jamshidi ◽  
Susan Natasha Beretvas ◽  
Patrick Onghena ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Effect Sizes ◽  
Forest Plot ◽  
Sampling Variance ◽  
Individual Effect ◽  
Outcome Reporting Bias ◽  
Funnel Plot ◽  
Multiple Effect ◽  
Funnel Plots ◽  
Meta Analyses

Meta-analytic datasets can be large, especially when in primary studies multiple effect sizes are reported. The visualization of meta-analytic data is therefore useful to summarize data and understand information reported in primary studies. The gold standard figures in meta-analysis are forest and funnel plots. However, none of these plots can yet account for the existence of multiple effect sizes within primary studies. This manuscript describes extensions to the funnel plot, forest plot and caterpillar plot to adapt them to three-level meta-analyses. For forest plots, we propose to plot the study-specific effects and their precision, and to add additional confidence intervals that reflect the sampling variance of individual effect sizes. For caterpillar plots and funnel plots, we recommend to plot individual effect sizes and averaged study-effect sizes in two separate graphs. For the funnel plot, plotting separate graphs might improve the detection of both publication bias and/or selective outcome reporting bias.

Power-Enhanced Funnel Plots for Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 43-49 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Relevant Information ◽  
Selective Reporting ◽  
Graphical Display ◽  
Funnel Plot ◽  
Evidential Value ◽  
Graphical Displays ◽  
Funnel Plots ◽  
Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

2019 ◽  
Author(s):  
Shinichi Nakagawa ◽  
Malgorzata Lagisz ◽  
Rose E O'Dea ◽  
Joanna Rutkowska ◽  
Yefeng Yang ◽  
...  
Keyword(s):  
Meta Analysis ◽  
R Package ◽  
Effect Sizes ◽  
Forest Plot ◽  
Point Estimates ◽  
Aggregate Effect ◽  
The Individual ◽  
Meta Analyses ◽  
Heterogeneous Effect ◽  
Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

scholarly journals The Galaxy Plot: A New Visualization Tool for Bivariate Meta-Analysis Studies

2020 ◽  
Vol 189 (8) ◽  
pp. 861-869 ◽  
Author(s):  
Chuan Hong ◽  
Rui Duan ◽  
Lingzhen Zeng ◽  
Rebecca A Hubbard ◽  
Thomas Lumley ◽  
...  
Keyword(s):  
Dimensional Space ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Standard Errors ◽  
Visualization Tool ◽  
The Galaxy ◽  
Funnel Plots ◽  
Complex Features ◽  
Bivariate Outcomes ◽  
Meta Analyses

Abstract Funnel plots have been widely used to detect small-study effects in the results of univariate meta-analyses. However, there is no existing visualization tool that is the counterpart of the funnel plot in the multivariate setting. We propose a new visualization method, the galaxy plot, which can simultaneously present the effect sizes of bivariate outcomes and their standard errors in a 2-dimensional space. We illustrate the use of the galaxy plot with 2 case studies, including a meta-analysis of hypertension trials with studies from 1979–1991 (Hypertension. 2005;45(5):907–913) and a meta-analysis of structured telephone support or noninvasive telemonitoring with studies from 1966–2015 (Heart. 2017;103(4):255–257). The galaxy plot is an intuitive visualization tool that can aid in interpreting results of multivariate meta-analysis. It preserves all of the information presented by separate funnel plots for each outcome while elucidating more complex features that may only be revealed by examining the joint distribution of the bivariate outcomes.

scholarly journals META ANALISIS: PENGARUH ANEMIA IBU HAMIL TERHADAP BERAT BAYI LAHIR RENDAH

2020 ◽  
Vol 11 (2) ◽  
pp. 163-177
Author(s):  
Aditianti Aditianti ◽  
Sri Poedji Hastoety Djaiman
Keyword(s):  
Pregnant Women ◽  
Young Women ◽  
Odds Ratio ◽  
Health Workers ◽  
Meta Analysis ◽  
Random Effect ◽  
Forest Plot ◽  
Funnel Plot ◽  
Funnel Plots

Abstract Background: The prevalence of low birth weight (LBW) in Indonesia shows a decrease, but the risk factor for anemia in pregnant women has increased sharply and this has an impact on increasing the prevalence of LBW. Objective: This study aimed to determine the risk of anemia in pregnant women to the prevalence of LBW in several countries. Methods: This study was a meta-analysis using PRISMA. Eleven of the 122,000 studies met criteria for the analysis. Presentation of the data used a forest plot with a random effect statistical model. Results: The combined odds ratio (OR) showed that the effect of anemia in pregnant women on LBW was 1.49 times higher than that of non-anemia mothers (95% CI: 1.26-4.60; p <0.001). The variance was 53,7%. The results of the funnel plots from 11 studies were not evenly distributed so that the information obtained was homogeneous, focusing more on the middle value. Conclusion: There was an effect of anemia in pregnant women with the prevalence of LBW. Detection of anemia in pregnant women needs to be done as early as possible by involving the role of health workers and cadres. Outreach activities for young women at schools and Posyandu must be carried out regularly and continuously.   Keywords: Anemia, LBW, Pregnancy     Abstrak Latar belakang: Prevalensi berat bayi lahir rendah (BBLR ) di Indonesia menunjukkan penurunan namun faktor risiko anemia pada ibu hamil meningkat tajam dan hal ini berdampak pada peningkatan kejadian BBLR. Tujuan: Studi ini bertujuan untuk mengetahui besarnya risiko ibu hamil anemia terhadap kejadian BBLR di beberapa negara. Metode: Studi ini merupakan meta analisis menggunakan PRISMA. Sebelas dari 122.000 studi masuk dalam kriteria untuk dianalisis. Penyajian data menggunakan forest plot dengan model statistik random effect. Hasil: Besar odds ratio (OR) gabungan menunjukkan bahwa pengaruh ibu hamil anemia terhadap BBLR 1,49 kali lebih tinggi dibandingkan ibu yang tidak anemia (95%CI: 1,26-4,60; p<0,001). Besarnya varian 53,7 persen. Hasil funnel plot dari 11 studi ini tidak tersebar secara merata sehingga informasi yang diperoleh homogen, lebih fokus pada nilai tengah. Kesimpulan: Terdapat pengaruh anemia pada ibu hamil dengan kejadian BBLR. Deteksi anemia pada ibu hamil perlu dilakukan sedini mungkin dengan melibatkan peran tenaga keseharan dan kader. Penyuluhan bagi remaja putri di sekolah dan posyandu harus dilakukan secara berkala dan berkesinambungan   Kata kunci: Anemia, BBLR, Kehamilan

Visual Inference for the Funnel Plot in Meta-Analysis

2019 ◽  
Vol 227 (1) ◽  
pp. 83-89 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Visual Inspection ◽  
Type I Error ◽  
Statistical Tests ◽  
Meta Analysis ◽  
Real Data ◽  
Type I ◽  
Funnel Plot ◽  
Funnel Plots ◽  
Meta Analyses ◽  
Open Nature

Abstract. The funnel plot is widely used in meta-analyses to assess potential publication bias. However, experimental evidence suggests that informal, mere visual, inspection of funnel plots is frequently prone to incorrect conclusions, and formal statistical tests (Egger regression and others) entirely focus on funnel plot asymmetry. We suggest using the visual inference framework with funnel plots routinely, including for didactic purposes. In this framework, the type I error is controlled by design, while the explorative, holistic, and open nature of visual graph inspection is preserved. Specifically, the funnel plot of the actually observed data is presented simultaneously, in a lineup, with null funnel plots showing data simulated under the null hypothesis. Only when the real data funnel plot is identifiable from all the funnel plots presented, funnel plot-based conclusions might be warranted. Software to implement visual funnel plot inference is provided via a tailored R function.

Combining results from several studies: systematic reviews and meta-analyses

2017 ◽  
Author(s):  
Mark Elwood
Keyword(s):  
Systematic Reviews ◽  
Testing Effect ◽  
Meta Analysis ◽  
Cochrane Collaboration ◽  
Outcome Reporting Bias ◽  
Regression Methods ◽  
Inverse Variance ◽  
Funnel Plots ◽  
Meta Analyses ◽  
The Cochrane Collaboration

This chapter explains systematic reviews, the PRISMA format, and meta-analysis. It discusses publication bias, outcome reporting bias, funnel plots, the issue of false positive results in small studies, along with search strategies, electronic databases, PubMed, and the Cochrane collaboration. It discusses the assessment of quality, risks of bias, limitations of meta-analysis, heterogeneity testing, effect modification, and meta-regression methods. In part two, it explains statistical methods for meta-analyses are presented, including the Mantel-Haenszel and Peto methods for individual patient data, the inverse variance weighted method using final results, and random effects methods. Forest plots and tests of heterogeneity are explained.

scholarly journals Reproducibility of Individual Effect Sizes in Meta-Analyses in Psychology

2019 ◽  
Author(s):  
Esther Maassen ◽  
Marcel A. L. M. van Assen ◽  
Michele B. Nuijten ◽  
Anton Olsson-Collentine ◽  
Jelte M. Wicherts
Keyword(s):  
Meta Analysis ◽  
Effect Sizes ◽  
Primary Study ◽  
Individual Effect ◽  
Meta Analyses

To determine the reproducibility of psychological meta-analyses, we investigated whether we could reproduce 500 primary study effect sizes drawn from 33 published meta-analyses based on the information given in the meta-analyses, and whether recomputations of primary study effect sizes altered the overall results of the meta-analysis.

The alchemy of confirmation bias transmutes expectations into bilingual advantages: A tale of two new meta-analyses

2020 ◽  
Vol 73 (8) ◽  
pp. 1290-1299 ◽  
Author(s):  
Kenneth R Paap ◽  
Lauren Mason ◽  
Brandon Zimiga ◽  
Yocelyne Ayala-Silva ◽  
Matthew Frost
Keyword(s):  
Meta Analysis ◽  
Confirmation Bias ◽  
Effect Sizes ◽  
Bilingual Advantage ◽  
Search Terms ◽  
Colour Word ◽  
Funnel Plots ◽  
The Mean ◽  
Meta Analyses ◽  
Clear Support

Five recent meta-analyses of the bilingual advantage in executive functioning hypothesis have converged on the outcome that the mean effect size is very small and that the incidence of statistically significant bilingual advantages is very low (about 15% of all comparisons). Those analyses that used the PET-PEESE method to correct for publication bias show mean effect sizes that are not different from zero and sometimes negative. In contrast, van den Noort and colleagues provide a sixth review of 46 studies published before October 31, 2018, that appears to produce a very different outcome, namely that more than half the studies yield clear support for the bilingual advantage hypothesis. We show that the deviance is due in part to search terms that yielded far fewer relevant studies, but more importantly to a subjective method of evaluating the results of each study that enables confirmation biases on the part of study authors and meta-analysts to substantially distort the objective pattern of results. A seventh meta-analysis, by Armstrong and colleagues, reports significant bilingual advantages of g = 0.48 for 32 samples using Simon and Stroop colour–word interference tasks that tested older adults. However, all effects were entered into the funnel plots as positive even though many were negative (bilingual disadvantages). This and other striking anomalies are consistent with the view that confirmation bias can suspend critical judgement and promulgate errors. Meta-analyses that use preregistration and a many-labs collaboration can better control for both publication and experimenter biases.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews &amp; Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals A protocol for the systematic review and meta-analysis of thigmotactic behaviour in the open field test in rodent models associated with persistent pain

2021 ◽  
Vol 5 (1) ◽  
pp. e100135
Author(s):  
Xue Ying Zhang ◽  
Jan Vollert ◽  
Emily S Sena ◽  
Andrew SC Rice ◽  
Nadia Soliman
Keyword(s):  
Systematic Review ◽  
Outcome Measure ◽  
Meta Analysis ◽  
Persistent Pain ◽  
Effect Sizes ◽  
Bias Assessment ◽  
Animal Pain ◽  
Trim And Fill ◽  
Meta Analyses ◽  
The Impact

ObjectiveThigmotaxis is an innate predator avoidance behaviour of rodents and is enhanced when animals are under stress. It is characterised by the preference of a rodent to seek shelter, rather than expose itself to the aversive open area. The behaviour has been proposed to be a measurable construct that can address the impact of pain on rodent behaviour. This systematic review will assess whether thigmotaxis can be influenced by experimental persistent pain and attenuated by pharmacological interventions in rodents.Search strategyWe will conduct search on three electronic databases to identify studies in which thigmotaxis was used as an outcome measure contextualised to a rodent model associated with persistent pain. All studies published until the date of the search will be considered.Screening and annotationTwo independent reviewers will screen studies based on the order of (1) titles and abstracts, and (2) full texts.Data management and reportingFor meta-analysis, we will extract thigmotactic behavioural data and calculate effect sizes. Effect sizes will be combined using a random-effects model. We will assess heterogeneity and identify sources of heterogeneity. A risk-of-bias assessment will be conducted to evaluate study quality. Publication bias will be assessed using funnel plots, Egger’s regression and trim-and-fill analysis. We will also extract stimulus-evoked limb withdrawal data to assess its correlation with thigmotaxis in the same animals. The evidence obtained will provide a comprehensive understanding of the strengths and limitations of using thigmotactic outcome measure in animal pain research so that future experimental designs can be optimised. We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines and disseminate the review findings through publication and conference presentation.

Sign in / Sign up

Export Citation Format

Share Document