Erratum to: BootES: An R package for bootstrap confidence intervals on effect sizes

We provide two simple metrics that could be reported routinely in random-effects meta-analyses to convey evidence strength for scientifically meaningful effects under effect heterogeneity (i.e., a nonzero estimated variance of the true effect distribution). First, given a chosen threshold of meaningful effect size, meta-analyses could report the estimated proportion of true effect sizes above this threshold. Second, meta-analyses could estimate the proportion of effect sizes below a second, possibly symmetric, threshold in the opposite direction from the estimated mean. These metrics could help identify if: (1) there are few effects of scientifically meaningful size despite a "statistically significant" pooled point estimate; (2) there are some large effects despite an apparently null point estimate; or (3) strong effects in the direction opposite the pooled estimate regularly also occur (and thus, potential effect modifiers should be examined). These metrics should be presented with confidence intervals, which can be obtained analytically or, under weaker assumptions, using bias-corrected and accelerated (BCa) bootstrapping. Additionally, these metrics inform relative comparison of evidence strength across related meta-analyses. We illustrate with applied examples and provide an R package to compute the metrics and confidence intervals.

Download Full-text

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

10.32942/osf.io/epqa7 ◽

2019 ◽

Author(s):

Shinichi Nakagawa ◽

Malgorzata Lagisz ◽

Rose E O'Dea ◽

Joanna Rutkowska ◽

Yefeng Yang ◽

...

Keyword(s):

Meta Analysis ◽

R Package ◽

Effect Sizes ◽

Forest Plot ◽

Point Estimates ◽

Aggregate Effect ◽

The Individual ◽

Meta Analyses ◽

Heterogeneous Effect ◽

Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

Download Full-text

Two-Condition Within-Participant Statistical Mediation Analysis: A Path-Analytic Framework

10.31234/osf.io/4su2j ◽

2019 ◽

Author(s):

Amanda Kay Montoya ◽

Andrew F. Hayes

Keyword(s):

Confidence Intervals ◽

Indirect Effect ◽

Mediation Analysis ◽

Analytic Approach ◽

Analytic Framework ◽

Hypothesis Tests ◽

Bootstrap Confidence Intervals ◽

Complex Models ◽

Path Analytic ◽

Statistical Mediation Analysis

Researchers interested in testing mediation often use designs where participants are measured on a dependent variable Y and a mediator M in both of two different circumstances. The dominant approach to assessing mediation in such a design, proposed by Judd, Kenny, and McClelland (2001), relies on a series of hypothesis tests about components of the mediation model and is not based on an estimate of or formal inference about the indirect effect. In this paper we recast Judd et al.’s approach in the path-analytic framework that is now commonly used in between-participant mediation analysis. By so doing, it is apparent how to estimate the indirect effect of a within-participant manipulation on some outcome through a mediator as the product of paths of influence. This path analytic approach eliminates the need for discrete hypothesis tests about components of the model to support a claim of mediation, as Judd et al’s method requires, because it relies only on an inference about the product of paths— the indirect effect. We generalize methods of inference for the indirect effect widely used in between-participant designs to this within-participant version of mediation analysis, including bootstrap confidence intervals and Monte Carlo confidence intervals. Using this path analytic approach, we extend the method to models with multiple mediators operating in parallel and serially and discuss the comparison of indirect effects in these more complex models. We offer macros and code for SPSS, SAS, and Mplus that conduct these analyses.

Download Full-text