Decision rules for selecting effect sizes in meta-analysis: A review and reanalysis of psychotherapy outcome studies.

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

Download Full-text

Identification of and Correction for Publication Bias: Comment

10.31222/osf.io/dh87m ◽

2019 ◽

Author(s):

Amanda Kvarven ◽

Eirik Strømland ◽

Magnus Johannesson

Keyword(s):

Publication Bias ◽

False Positive ◽

Large Scale ◽

Meta Analysis ◽

False Positive Rate ◽

Effect Sizes ◽

Replication Studies ◽

Moderate Reduction ◽

Positive Rate ◽

Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

Download Full-text

Effectiveness of psychological interventions to improve emotion regulation in youth - a meta-analysis

10.31234/osf.io/3yxcp ◽

2019 ◽

Author(s):

Bettina Moltrecht ◽

Jessica Deighton ◽

Praveetha Patalay ◽

Julian Childs

Keyword(s):

Emotion Regulation ◽

Emotion Dysregulation ◽

Psychological Intervention ◽

Meta Analysis ◽

Age Groups ◽

Effect Sizes ◽

What Works ◽

Large Heterogeneity ◽

Larger Sample

Background: Research investigating the role of emotion regulation (ER) in the development and treatment of psychopathology has increased in recent years. Evidence suggests that an increased focus on ER in treatment can improve existing interventions. Most ER research has neglected young people, therefore the present meta-analysis summarizes the evidence for existing psychosocial intervention and their effectiveness to improve ER in youth. Methods: A systematic review and meta-analysis was conducted according to the PRISMA guidelines. Twenty-one randomized-control-trials (RCTs) assessed changes in ER following a psychological intervention in youth exhibiting various psychopathological symptoms.Results: We found moderate effect sizes for current interventions to decrease emotion dysregulation in youth (g=-.46) and small effect sizes to improve emotion regulation (g=0.36). Significant differences between studies including intervention components, ER measures and populations studied resulted in large heterogeneity. Conclusion: This is the first meta-analysis that summarizes the effectiveness for existing interventions to improve ER in youth. The results suggest that interventions can enhance ER in youth, and that these improvements correlate with improvements in psychopathology. More RCTs including larger sample sizes, different age groups and psychopathologies are needed to increase our understanding of what works for who and when.

Download Full-text

A Meta-Analysis of the Facial Feedback Literature: Effects of Facial Feedback on Emotional Experience Are Small and Variable

10.31234/osf.io/svjru ◽

2017 ◽

Cited By ~ 2

Author(s):

Nicholas Alvaro Coles ◽

Jeff T. Larsen ◽

Heather Lench

Keyword(s):

Publication Bias ◽

Emotional Experience ◽

Meta Analysis ◽

Effect Sizes ◽

Detection Methods ◽

Feedback Effects ◽

Facial Feedback ◽

Facial Movements ◽

Facial Feedback Hypothesis ◽

Variance Estimates

The facial feedback hypothesis suggests that an individual’s experience of emotion is influenced by feedback from their facial movements. To evaluate the cumulative evidence for this hypothesis, we conducted a meta-analysis on 286 effect sizes derived from 138 studies that manipulated facial feedback and collected emotion self-reports. Using random effects meta-regression with robust variance estimates, we found that the overall effect of facial feedback was significant, but small. Results also indicated that feedback effects are stronger in some circumstances than others. We examined 12 potential moderators, and three were associated with differences in effect sizes. 1. Type of emotional outcome: Facial feedback influenced emotional experience (e.g., reported amusement) and, to a greater degree, affective judgments of a stimulus (e.g., the objective funniness of a cartoon). Three publication bias detection methods did not reveal evidence of publication bias in studies examining the effects of facial feedback on emotional experience, but all three methods revealed evidence of publication bias in studies examining affective judgments. 2. Presence of emotional stimuli: Facial feedback effects on emotional experience were larger in the absence of emotionally evocative stimuli (e.g., cartoons). 3. Type of stimuli: When participants were presented with emotionally evocative stimuli, facial feedback effects were larger in the presence of some types of stimuli (e.g., emotional sentences) than others (e.g., pictures). The available evidence supports the facial feedback hypothesis’ central claim that facial feedback influences emotional experience, although these effects tend to be small and heterogeneous.

Download Full-text