scholarly journals A Guide to Conducting a Meta-Analysis with Non-Independent Effect Sizes

2019 ◽  
Author(s):  
Mike W.-L. Cheung
Keyword(s):  
Ad Hoc ◽  
Meta Analysis ◽  
Real Data ◽  
Effect Sizes ◽  
Independent Effect ◽  
New Techniques ◽  
Missed Opportunities ◽  
Research Questions ◽  
Conventional Models ◽  
Traditional Approaches

Conventional meta-analytic procedures assume that effect sizes are independent. When effect sizes are non-independent, conclusions based on these conventional models can be misleading or even wrong. Traditional approaches, such as averaging the effect sizes and selecting one effect size per study, are usually used to remove the dependence of the effect sizes. These ad-hoc approaches, however, may lead to missed opportunities to utilize all available data to address the relevant research questions. Both multivariate meta-analysis and three-level meta-analysis have been proposed to handle non-independent effect sizes. This paper gives a brief introduction to these new techniques for applied researchers. The first objective is to highlight the benefits of using these methods to address non-independent effect sizes. The second objective is to illustrate how to apply these techniques with real data in R and Mplus. Researchers may modify the sample R and Mplus code to fit their data.

scholarly journals Coordinate based meta-analysis of networks in neuroimaging studies

2018 ◽  
Author(s):  
CR Tench ◽  
Radu Tanasescu ◽  
CS Constantinescu ◽  
DP Auer ◽  
WJ Cottam
Keyword(s):  
Spatial Clustering ◽  
Meta Analysis ◽  
Simulated Data ◽  
Real Data ◽  
Network Effect ◽  
Effect Sizes ◽  
Multivariate Normal ◽  
Z Scores ◽  
Independent Effects ◽  
Multiple Clusters

AbstractMeta-analysis of published neuroimaging results is commonly performed using coordinate based meta-analysis (CBMA). Most commonly CBMA algorithms detect spatial clustering of reported coordinates across multiple studies by assuming that results relating to the common hypothesis fall in similar anatomical locations. The null hypothesis is that studies report uncorrelated results, which is simulated by random coordinates. It is assumed that multiple clusters are independent yet it is likely that multiple results reported per study are not, and in fact represent a network effect. Here the multiple reported effect sizes (reported peak Z scores) are assumed multivariate normal, and maximum likelihood used to estimate the parameters of the covariance matrix. The hypothesis is that the effect sizes are correlated. The parameters are covariance of effect size, considered as edges of a network, while clusters are considered as nodes. In this way coordinate based meta-analysis of networks (CBMAN) estimates a network of reported meta-effects, rather than multiple independent effects (clusters).CBMAN uses only the same data as CBMA, yet produces extra information in terms of the correlation between clusters. Here it is validated on numerically simulated data, and demonstrated on real data used previously to demonstrate CBMA. The CBMA and CBMAN clusters are similar, despite the very different hypothesis.

Characteristics of Effective School-Based, Teacher-Delivered Mental Health Services for Children

2019 ◽  
Vol 30 (4) ◽  
pp. 422-432
Author(s):  
Sunyoung Park ◽  
Samantha Guz ◽  
Anao Zhang ◽  
S. Natasha Beretvas ◽  
Cynthia Franklin ◽  
...  
Keyword(s):  
Mental Health ◽  
Mental Health Services ◽  
Health Services ◽  
Meta Analysis ◽  
Specific Treatment ◽  
Externalizing Disorders ◽  
Effect Sizes ◽  
Independent Effect ◽  
School Based ◽  
Treatment Characteristics

Purpose: The increasing need for school-based mental health services has altered teachers’ involvement in mental health services. Methods: This study presents a meta-analysis from a previous systematic review to identify which study characteristics result in effective treatment outcomes. Specific treatment characteristics analyzed in this study include type of intervention, treatment modality, length of treatment, and type of measurement. Effect sizes were coded by internalizing and externalizing disorders, depending on the symptoms the corresponding treatments were intended to address. A final sample size included 9 independent effect sizes of internalizing behaviors and 21 effect sizes of externalizing behaviors. Results: Internalizing disorders, social skill interventions, classroom modalities, and medium treatment length were moderating treatment characteristics. No significant effects were found for externalizing disorders. Conclusions: These results further add to the research on teacher’s role in school-based mental health services and provide important information for social workers who work in schools.

A Meta-Analysis on Computer Technology Intervention Effects on Mathematics Achievement for Low-Performing Students in K-12 Classrooms

2020 ◽  
pp. 073563312095206
Author(s):  
Hua Ran ◽  
Murat Kasli ◽  
Walter G. Secada
Keyword(s):  
Instructional Design ◽  
Mathematics Achievement ◽  
Computer Technology ◽  
Meta Analysis ◽  
Empirical Studies ◽  
Effect Sizes ◽  
Independent Effect ◽  
Technology Intervention ◽  
K 12 ◽  
Positive Effect

This meta-analysis extended the current literature regarding the effects of computer technology (CT) on mathematics achievement, with a particular focus on low-performing students. A total of 45 independent effect sizes extracted from 31 empirical studies based on a total of 2,044 low-performing students in K-12 classrooms were included in this meta-analysis. Consistent with previous reviews, this study suggested a statistically significant and positive effect of CT ([Formula: see text] = 0.56) on low-performing students’ mathematics achievement. Of four CT types, the largest CT effect was found with problem-solving system ([Formula: see text] = 0.86), followed by tutoring [Formula: see text] = 0.80), game-based intervention ([Formula: see text] = .58), and computerized practice ([Formula: see text] = .23). Furthermore, other moderators were found to explain variation in CT effects on low-performing students’ mathematics achievement. Study findings contributed to clarifying the effect of CT for low-performing students’ mathematics achievement. Implications for instructional design and practices are also discussed.

Toward Optimized Effectiveness of Employee Training Programs

2021 ◽  
Author(s):  
Dilek Uslu ◽  
Justin Marcus ◽  
Yasemin Kisbu-Sakarya
Keyword(s):  
Training Programs ◽  
Meta Analysis ◽  
Employee Training ◽  
Effect Sizes ◽  
Independent Effect ◽  
Workplace Training ◽  
Affective Outcomes ◽  
Training Outcomes ◽  
Training Interventions ◽  
Training Delivery

Abstract. Although organizations invest heavily on employee training, the effectiveness of employee training programs has not been well-established. In the current study, we examine the training delivery features of employee training programs to derive a better understanding of features that may be of best benefit in the improvement of employee affective outcomes. Specifically, and via the use of meta-analysis ( k = 79 studies totaling 107 independent effect sizes), we focus on two broad classes of affective employee training outcomes including attitudinal and motivational outcomes. Results evidence support for the effectiveness of employee workplace training interventions and indicate that employee training programs associated with attitudinal versus motivational outcomes require different features while being delivered to reach optimal effectiveness.

scholarly journals A Meta-analysis of Employment Discrimination Against Muslims and Arabs

2018 ◽  
Author(s):  
Timothy Bartkoski ◽  
Ellen Herrmann ◽  
Chelsea Witt ◽  
Cort Rudolph
Keyword(s):  
Meta Analysis ◽  
Experimental Studies ◽  
Research Literature ◽  
Effect Sizes ◽  
Independent Effect ◽  
Work Setting ◽  
Employment Decisions ◽  
Quantitative Review ◽  
Almost All ◽  
Future Work

Muslim and Arab individuals are discriminated against in almost all domains. Recently, there hasbeen a focus on examining the treatment of these groups in the work setting. Despite the great number of primary studies examining this issue, there has not yet been a quantitative review of the research literature. To fill this gap, this meta-analysis examined the presence and magnitude of hiring discrimination against Muslim and Arab individuals. Using 46 independent effect sizes from 26 sources, we found evidence of discrimination against Muslim and Arab people in employment judgments, behaviors, and decisions across multiple countries. Moderator analyses revealed that discrimination is stronger in field settings, when actual employment decisions are made, and when experimental studies used “Arab” (vs. “Muslim”) targets. However, primary studies provide inconsistent and inaccurate distinctions between Arabs and Muslims, therefore future work should be cautious in categorizing the exact aspect of identity being studied.

Current Methods for Meta-Analysis

2007 ◽  
Vol 215 (2) ◽  
pp. 90-103 ◽  
Author(s):  
Ralf Schulze
Keyword(s):  
Fixed Effects ◽  
Structural Equation ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Equation Modeling ◽  
Random Effects Model ◽  
New Methods ◽  
New Findings ◽  
Multivariate Procedures ◽  
Research Questions

Abstract. The bulk of conceptual and statistical developments as well as applications of meta-analysis have been published in the last 30 years. The methods for meta-analysis continue to be refined and new methods are applied to new types of research questions and data. Such current approaches, issues, and developments prevalent in the behavioral sciences are presented, reviewed, and discussed in this paper. The areas that are covered include: the fixed effects and random effects model of meta-analysis, new findings concerning effect sizes and their statistical properties, the comparison of different meta-analytic approaches, and multivariate procedures for meta-analysis. The latter also covers the stepwise combination of meta-analysis and structural equation modeling (MASEM).

scholarly journals A Meta-Analysis of the Effects of Mental Contrasting With Implementation Intentions on Goal Attainment

2021 ◽  
Vol 12 ◽  
Author(s):  
Guoxia Wang ◽  
Yi Wang ◽  
Xiaosong Gai
Keyword(s):  
Goal Attainment ◽  
Implementation Intentions ◽  
Meta Analysis ◽  
Empirical Studies ◽  
Self Regulation ◽  
Effect Sizes ◽  
Medium Effect ◽  
Independent Effect ◽  
Mental Contrasting ◽  
Medium Effect Size

Mental contrasting with implementation intentions (MCII) is a self-regulation strategy that enhances goal attainment. This meta-analysis evaluated the efficacy of MCII for goal attainment and explored potential moderators. A total of 21 empirical studies with 24 independent effect sizes (15,907 participants) were included in the analysis. Results showed that MCII to be effective for goal attainment with a small to medium effect size (g = 0.336). The effect was mainly moderated by intervention style. Specifically, studies with interventions based on interactions between participants and experimenters (g = 0.465) had stronger effects than studies with interventions based on interactions between participants and documents (g = 0.277). The results revealed that MCII is a brief and effective strategy for goal attainment with a small to moderate effect; however, because of some publication bias, the actual effect sizes may be smaller. Due to small number of studies in this meta-analysis, additional studies are needed to determine the role of moderator variables.

A meta-analytic review of the relation between impulsivity-related traits and family history of alcoholism

2018 ◽  
Author(s):  
◽  
Angela Maria Haeny
Keyword(s):  
Family History ◽  
Effect Size ◽  
Sensation Seeking ◽  
Meta Analysis ◽  
Alcohol Problems ◽  
Effect Sizes ◽  
Independent Effect ◽  
Increased Risk ◽  
Analytic Review ◽  
History Of

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT AUTHOR'S REQUEST.] Extensive research provides evidence that people with a family history of alcoholism are at risk for developing alcohol use disorder (AUD). Similarly, people with impulsivity-related traits are at increased risk for developing alcohol problems. Importantly, research suggests that impulsivity mediates the relation between family history of alcoholism and the development of alcohol problems. However, impulsivity is a heterogenous construct and has been assessed with a myriad of measures. The present work is a quantitative synthesis of the literature on the relation between family history of alcoholism and impulsivity-related traits and that also examines various potential moderators of this association. Sixty-nine independent effect sizes from 65 studies (N = 11,127) qualified for the meta-analysis. The overall effect size was small-to-moderate (d = .32 [95% CI: 0.25, 0.39], k = 69), and was moderated by offspring age (Z = 3.73, p less than .001), with the effect size increasing with age. When examining specific facets of impulsivity, a small effect was found for harm avoidance (d = -.26 [95% CI: -.41, -.11], k = 10) and was moderated by family history density (Q (1) = 4.12, p = .04) such that the effect was much larger among those with more than one alcoholic family member (d = -.66 [95% CI: -1.10, -.22], k = 3). A small-to-moderate effect size was found for sensation seeking (d = .30 [95% CI: .21, .40], k = 29) and was moderated by age (Z = 3.09, p = .002), with the effect increasing with age. The effect sizes for all other facets of impulsivity were not significant. Notably, there were much fewer studies investigating other facets of impulsivity (e.g., reward dependence, lack of perseverance, lack of planning) compared to sensation seeking, limiting power to detect larger effect sizes. Findings from this review suggest the need for additional studies investigating the relation between specific facets of impulsivity (e.g., positive and negative urgency) and family history of alcoholism. In addition, this review suggests that, to some degree, we can identify phenotypic risk beyond mere family history status and, thus, inform the development of interventions for individuals with a family history of alcoholism, targeting the specific types of impulsivity manifested.

scholarly journals Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias

2021 ◽  
Author(s):  
Hilde Elisabeth Maria Augusteijn ◽  
Robbie Cornelis Maria van Aert ◽  
Marcel A. L. M. van Assen
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Web Application ◽  
Meta Analysis ◽  
Real Data ◽  
Effect Sizes ◽  
Posterior Probabilities ◽  
True Effect Size ◽  
Heterogeneity Parameter ◽  
Size Heterogeneity

Publication bias remains to be a great challenge when conducting a meta-analysis. It may result in overestimated effect sizes, increased frequency of false positives, and over- or underestimation of the effect size heterogeneity parameter. A new method is introduced, Bayesian Meta-Analytic Snapshot (BMAS), which evaluates both effect size and its heterogeneity and corrects for potential publication bias. It evaluates the probability of the true effect size being zero, small, medium or large, and the probability of true heterogeneity being zero, small, medium or large. This approach, which provides an intuitive evaluation of uncertainty in the evaluation of effect size and heterogeneity, is illustrated with a real-data example, a simulation study, and a Shiny web application of BMAS.

Sign in / Sign up

Export Citation Format

Share Document