Second-order meta-analysis synthesizing the evidence on associations between school leadership and different school outcomes

2020 ◽  
pp. 174114322093545
Author(s):  
Cheng Yong Tan ◽  
Lin Gao ◽  
Meijia Shi
Keyword(s):  
School Leadership ◽  
Leadership Practices ◽  
Teaching And Learning ◽  
School Effectiveness ◽  
Meta Analysis ◽  
Second Order ◽  
Effect Sizes ◽  
Positive Effects ◽  
Leading Schools ◽  
Meta Analyses

The present study addresses the question of whether school leadership matters. It employs second-order meta-analysis to synthesize results from 12 first-order meta-analyses examining school leadership effects published 2003–2019. These meta-analyses collectively examined 512 primary studies published across four decades (1978–2019). Results showed that the overall mean effect size for school leadership was small in magnitude ( r = .33). Effect sizes for leadership models were larger than those for leadership practices, thereby indicating the utility of examining models as compared to practices for understanding leadership influence. Relatedly, findings of significant positive effects for eight different school leadership practices underscore the need to examine comprehensively the scope of school leaders’ work beyond that related to teaching-and-learning. Additionally, leaders require myriad competencies and skills including how to galvanize, motivate and equip teachers to achieve school goals. The substantially larger mean effect sizes for organizational and teacher as compared to student outcomes challenge the assertion by some that principals are less consequential than teachers in contributing to school effectiveness. Indeed, the larger effect sizes for principals as compared to other types of leaders reflect the key role they play in leading schools.

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

2019 ◽  
Author(s):  
Shinichi Nakagawa ◽  
Malgorzata Lagisz ◽  
Rose E O'Dea ◽  
Joanna Rutkowska ◽  
Yefeng Yang ◽  
...  
Keyword(s):  
Meta Analysis ◽  
R Package ◽  
Effect Sizes ◽  
Forest Plot ◽  
Point Estimates ◽  
Aggregate Effect ◽  
The Individual ◽  
Meta Analyses ◽  
Heterogeneous Effect ◽  
Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals Effective Components of School-Based Prevention Programs for Child Abuse: A Meta-Analytic Review

2021 ◽  
Author(s):  
Jeanne Gubbels ◽  
Claudia E. van der Put ◽  
Geert-Jan J. M. Stams ◽  
Mark Assink
Keyword(s):  
Child Abuse ◽  
Program Effectiveness ◽  
Meta Analysis ◽  
Prevention Programs ◽  
Effect Sizes ◽  
Positive Effects ◽  
School Based ◽  
Analytic Review ◽  
Program Components ◽  
Self Protection

AbstractSchool-based programs seem promising for child abuse prevention. However, research mainly focused on sexual child abuse and knowledge is lacking on how individual program components contribute to the effectiveness of school-based prevention programs for any form of child abuse. This study aimed to examine the overall effect of these school-based programs on (a) children’s child abuse-related knowledge and (b) self-protection skills by conducting two three-level meta-analyses. Furthermore, moderator analyses were performed to identify how program components and delivery techniques were associated with effectiveness. A literature search yielded 34 studies (158 effect sizes; N = 11,798) examining knowledge of child abuse and 22 studies (99 effect sizes; N = 7804) examining self-protection skills. A significant overall effect was found of school-based programs on both knowledge (d = 0.572, 95% CI [0.408, 0.737], p < 0.001) and self-protection skills (d = 0.528, 95% CI [0.262, 0.794], p < 0.001). The results of the first meta-analysis on children’s child abuse knowledge suggest that program effects were larger in programs addressing social–emotional skills of children (d = 0.909 for programs with this component versus d = 0.489 for programs without this component) and self-blame (d = 0.776 versus d = 0.412), and when puppets (d = 1.096 versus d = 0.500) and games or quizzes (d = 0.966 versus d = 0.494) were used. The second meta-analysis on children’s self-protections skills revealed that no individual components or techniques were associated with increased effectiveness. Several other study and program characteristics did moderate the overall effects and are discussed. In general, school-based prevention programs show positive effects on both knowledge and self-protection skills, and the results imply that program effectiveness can be improved by implementing specific components and techniques.

scholarly journals A protocol for the systematic review and meta-analysis of thigmotactic behaviour in the open field test in rodent models associated with persistent pain

2021 ◽  
Vol 5 (1) ◽  
pp. e100135
Author(s):  
Xue Ying Zhang ◽  
Jan Vollert ◽  
Emily S Sena ◽  
Andrew SC Rice ◽  
Nadia Soliman
Keyword(s):  
Systematic Review ◽  
Outcome Measure ◽  
Meta Analysis ◽  
Persistent Pain ◽  
Effect Sizes ◽  
Bias Assessment ◽  
Animal Pain ◽  
Trim And Fill ◽  
Meta Analyses ◽  
The Impact

ObjectiveThigmotaxis is an innate predator avoidance behaviour of rodents and is enhanced when animals are under stress. It is characterised by the preference of a rodent to seek shelter, rather than expose itself to the aversive open area. The behaviour has been proposed to be a measurable construct that can address the impact of pain on rodent behaviour. This systematic review will assess whether thigmotaxis can be influenced by experimental persistent pain and attenuated by pharmacological interventions in rodents.Search strategyWe will conduct search on three electronic databases to identify studies in which thigmotaxis was used as an outcome measure contextualised to a rodent model associated with persistent pain. All studies published until the date of the search will be considered.Screening and annotationTwo independent reviewers will screen studies based on the order of (1) titles and abstracts, and (2) full texts.Data management and reportingFor meta-analysis, we will extract thigmotactic behavioural data and calculate effect sizes. Effect sizes will be combined using a random-effects model. We will assess heterogeneity and identify sources of heterogeneity. A risk-of-bias assessment will be conducted to evaluate study quality. Publication bias will be assessed using funnel plots, Egger’s regression and trim-and-fill analysis. We will also extract stimulus-evoked limb withdrawal data to assess its correlation with thigmotaxis in the same animals. The evidence obtained will provide a comprehensive understanding of the strengths and limitations of using thigmotactic outcome measure in animal pain research so that future experimental designs can be optimised. We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines and disseminate the review findings through publication and conference presentation.

scholarly journals Pre-post effect sizes should be avoided in meta-analyses

2016 ◽  
Vol 26 (4) ◽  
pp. 364-368 ◽  
Author(s):  
P. Cuijpers ◽  
E. Weitz ◽  
I. A. Cristea ◽  
J. Twisk
Keyword(s):  
Comparison Group ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Behaviour Therapy ◽  
Natural Processes ◽  
Cognitive Behaviour ◽  
Post Test ◽  
The Difference ◽  
Meta Analyses ◽  
Highly Cited

AimsThe standardised mean difference (SMD) is one of the most used effect sizes to indicate the effects of treatments. It indicates the difference between a treatment and comparison group after treatment has ended, in terms of standard deviations. Some meta-analyses, including several highly cited and influential ones, use the pre-post SMD, indicating the difference between baseline and post-test within one (treatment group).MethodsIn this paper, we argue that these pre-post SMDs should be avoided in meta-analyses and we describe the arguments why pre-post SMDs can result in biased outcomes.ResultsOne important reason why pre-post SMDs should be avoided is that the scores on baseline and post-test are not independent of each other. The value for the correlation should be used in the calculation of the SMD, while this value is typically not known. We used data from an ‘individual patient data’ meta-analysis of trials comparing cognitive behaviour therapy and anti-depressive medication, to show that this problem can lead to considerable errors in the estimation of the SMDs. Another even more important reason why pre-post SMDs should be avoided in meta-analyses is that they are influenced by natural processes and characteristics of the patients and settings, and these cannot be discerned from the effects of the intervention. Between-group SMDs are much better because they control for such variables and these variables only affect the between group SMD when they are related to the effects of the intervention.ConclusionsWe conclude that pre-post SMDs should be avoided in meta-analyses as using them probably results in biased outcomes.

Meta-analysis of effect sizes reported at multiple time points: A multivariate approach

2012 ◽  
Vol 9 (5) ◽  
pp. 610-620 ◽  
Author(s):  
Thomas A Trikalinos ◽  
Ingram Olkin
Keyword(s):  
Random Effects ◽  
Treatment Effects ◽  
Multivariate Analyses ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Multiple Time ◽  
Time Points ◽  
Fixed And Random Effects ◽  
Multiple Time Points ◽  
Meta Analyses

Background Many comparative studies report results at multiple time points. Such data are correlated because they pertain to the same patients, but are typically meta-analyzed as separate quantitative syntheses at each time point, ignoring the correlations between time points. Purpose To develop a meta-analytic approach that estimates treatment effects at successive time points and takes account of the stochastic dependencies of those effects. Methods We present both fixed and random effects methods for multivariate meta-analysis of effect sizes reported at multiple time points. We provide formulas for calculating the covariance (and correlations) of the effect sizes at successive time points for four common metrics (log odds ratio, log risk ratio, risk difference, and arcsine difference) based on data reported in the primary studies. We work through an example of a meta-analysis of 17 randomized trials of radiotherapy and chemotherapy versus radiotherapy alone for the postoperative treatment of patients with malignant gliomas, where in each trial survival is assessed at 6, 12, 18, and 24 months post randomization. We also provide software code for the main analyses described in the article. Results We discuss the estimation of fixed and random effects models and explore five options for the structure of the covariance matrix of the random effects. In the example, we compare separate (univariate) meta-analyses at each of the four time points with joint analyses across all four time points using the proposed methods. Although results of univariate and multivariate analyses are generally similar in the example, there are small differences in the magnitude of the effect sizes and the corresponding standard errors. We also discuss conditional multivariate analyses where one compares treatment effects at later time points given observed data at earlier time points. Limitations Simulation and empirical studies are needed to clarify the gains of multivariate analyses compared with separate meta-analyses under a variety of conditions. Conclusions Data reported at multiple time points are multivariate in nature and are efficiently analyzed using multivariate methods. The latter are an attractive alternative or complement to performing separate meta-analyses.

Mindfulness- and Acceptance-Based Interventions for Stroke Survivors: A Systematic Review and Meta-Analysis

2021 ◽  
pp. 003435522110432
Author(s):  
Areum Han
Keyword(s):  
Systematic Review ◽  
Depressive Symptoms ◽  
Meta Analysis ◽  
Random Effect ◽  
Random Effect Model ◽  
Mental Fatigue ◽  
Stroke Survivors ◽  
Positive Effects ◽  
Meta Analyses

Objective: Mindfulness- and acceptance-based intervention (MABI) is an emerging evidenced-based practice, but no systematic review incorporating meta-analyses for MABIs in stroke survivors has been conducted. The objective of this systematic review was to measure the effectiveness of MABIs on outcomes in people with stroke. Method: Three electronic databases, including PubMed, CINAHL, and PsycINFO, were searched to identify relevant studies published in peer-reviewed journals. The methodological quality of the included studies was assessed. Data were extracted and combined in a meta-analysis with a random-effect model to compute the size of the intervention effect. Results: A total of 11 studies met the eligibility criteria. Meta-analyses found a small-to-moderate effect of MABIs on depressive symptoms (standardized mean difference [SMD] = 0.39, 95% confidence interval [CI] = [0.12, 0.66]) and a large effect on mental fatigue (SMD = 1.22, 95% CI = [0.57, 1.87]). No statistically significant effect of MABIs on anxiety, quality of life, and mindfulness was found, but there was a trend in favor of MABIs overall. Conclusions: This meta-analysis found positive effects of MABIs on depressive symptoms and mental fatigue in stroke survivors, but future high-quality studies are needed to guarantee treatment effects of MABIs on varied outcomes in stroke survivors.

scholarly journals A Statistical Method for Synthesizing Meta-Analyses

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Liansheng Larry Tang ◽  
Michael Caudy ◽  
Faye Taxman
Keyword(s):  
Statistical Method ◽  
Effect Size ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Weighted Mean ◽  
Summary Effect ◽  
Different Types ◽  
Meta Analyses ◽  
Size Estimates ◽  
Similar Search

Multiple meta-analyses may use similar search criteria and focus on the same topic of interest, but they may yield different or sometimes discordant results. The lack of statistical methods for synthesizing these findings makes it challenging to properly interpret the results from multiple meta-analyses, especially when their results are conflicting. In this paper, we first introduce a method to synthesize the meta-analytic results when multiple meta-analyses use the same type of summary effect estimates. When meta-analyses use different types of effect sizes, the meta-analysis results cannot be directly combined. We propose a two-step frequentist procedure to first convert the effect size estimates to the same metric and then summarize them with a weighted mean estimate. Our proposed method offers several advantages over existing methods by Hemming et al. (2012). First, different types of summary effect sizes are considered. Second, our method provides the same overall effect size as conducting a meta-analysis on all individual studies from multiple meta-analyses. We illustrate the application of the proposed methods in two examples and discuss their implications for the field of meta-analysis.

Physical Abuse, Emotional Abuse, and Neglect and Childhood Behavior Problems: A Meta-Analysis of Studies in Mainland China

2018 ◽  
Vol 21 (1) ◽  
pp. 206-224 ◽  
Author(s):  
Naixue Cui ◽  
Jianghong Liu
Keyword(s):  
Child Maltreatment ◽  
Behavior Problems ◽  
Emotional Abuse ◽  
Meta Analysis ◽  
Mainland China ◽  
Effect Sizes ◽  
Childhood Behavior ◽  
Abuse And Neglect ◽  
Meta Analyses ◽  
Childhood Behavior Problems

The relationship between three types of child maltreatment, including physical abuse, emotional abuse and neglect, and childhood behavior problems in Mainland China, has not been systematically examined. This meta-analysis reviewed findings from 42 studies conducted in 98,749 children in Mainland China and analyzed the pooled effect sizes of the associations between child maltreatment and childhood behavior problems, heterogeneity in study findings, and publication bias. In addition, this study explored cross-study similarities/differences by comparing the pooled estimates with findings from five existing meta-analyses. Equivalent small-to-moderate effect sizes emerged in the relationships between the three types of maltreatment and child externalizing and internalizing behaviors, except that emotional abuse related more to internalizing than externalizing behaviors. Considerable heterogeneity exists among the 42 studies. Weak evidence suggests that child gender and reporter of emotional abuse may moderate the strengths of the relationships between child maltreatment and behavior problems. No indication of publication bias emerged. Cross-study comparisons show that the pooled effect sizes in this meta-analysis are about equal to those reported in the five meta-analyses conducted in child and adult populations across the world. Findings urge relevant agencies in Mainland China to build an effective child protection system to prevent child maltreatment.

Systematic Review and Meta-Analysis of Classroom-Wide Social–Emotional Interventions for Preschool Children

2020 ◽  
pp. 027112142093557 ◽  
Author(s):  
Li Luo ◽  
Brian Reichow ◽  
Patricia Snyder ◽  
Jennifer Harrington ◽  
Joy Polignano
Keyword(s):  
Systematic Review ◽  
Preschool Children ◽  
Social Competence ◽  
Random Effects ◽  
Challenging Behavior ◽  
Emotional Competence ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Social Emotional ◽  
Meta Analyses

Background: All children benefit from intentional interactions and instruction to become socially and emotionally competent. Over the past 30 years, evidence-based intervention tactics and strategies have been integrated to establish comprehensive, multitiered, or hierarchical systems of support frameworks to guide social–emotional interventions for young children. Objectives: To review systematically the efficacy of classroom-wide social–emotional interventions for improving the social, emotional, and behavioral outcomes of preschool children and to use meta-analytic techniques to identify critical study characteristics associated with obtained effect sizes. Method: Four electronic databases (i.e., Academic Search Premier, Educational Resource Information Center, PsycINFO, and Education Full Text) were systematically searched in December 2015 and updated in January 2018. “Snowball methods” were used to locate additional relevant studies. Effect size estimates were pooled using random-effects meta-analyses for three child outcomes, and moderator analyses were conducted. Results: Thirty-nine studies involving 10,646 child participants met the inclusion criteria and were included in this systematic review, with 33 studies included in the meta-analyses. Random-effects meta-analyses showed improvements in social competence ( g = 0.42, 95% confidence interval [CI] = [0.28, 0.56]) and emotional competence ( g = 0.33, 95% CI = [0.10, 0.56]), and decreases in challenging behavior ( g = −0.31, 95% CI = [−0.43, −0.19]). For social competence and challenging behavior, moderator analyses suggested interventions with a family component had statistically significant and larger effect sizes than those without a family component. Studies in which classroom teachers served as the intervention agent produced statistically significant but smaller effect sizes than when researchers or others implemented the intervention for challenging behavior. Conclusion: This systematic review and meta-analysis support using comprehensive social–emotional interventions for all children in a preschool classroom to improve their social–emotional competence and reduce challenging behavior.

Sign in / Sign up

Export Citation Format

Share Document