scholarly journals Comparison of Within- and Between-Series Effect Estimates in the Meta-Analysis of Multiple Baseline Studies

2021 ◽  
pp. 107699862110355
Author(s):  
Seang-Hwane Joo ◽  
Yan Wang ◽  
John Ferron ◽  
S. Natasha Beretvas ◽  
Mariola Moeyaert ◽  
...  
Keyword(s):  
Effect Size ◽  
Mean Squared Error ◽  
Meta Analysis ◽  
Small Sample ◽  
Effect Sizes ◽  
Alternative Methods ◽  
Multiple Baseline ◽  
Simulation Studies ◽  
Squared Error ◽  
Size Estimates

Multiple baseline (MB) designs are becoming more prevalent in educational and behavioral research, and as they do, there is growing interest in combining effect size estimates across studies. To further refine the meta-analytic methods of estimating the effect, this study developed and compared eight alternative methods of estimating intervention effects from a set of MB studies. The methods differed in the assumptions made and varied in whether they relied on within- or between-series comparisons, modeled raw data or effect sizes, and did or did not standardize. Small sample functioning was examined through two simulation studies, which showed that when data were consistent with assumptions the bias was consistently less than 5% of the effect size for each method, whereas root mean squared error varied substantially across methods. When assumptions were violated, substantial biases were found. Implications and limitations are discussed.

scholarly journals A Statistical Method for Synthesizing Meta-Analyses

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Liansheng Larry Tang ◽  
Michael Caudy ◽  
Faye Taxman
Keyword(s):  
Statistical Method ◽  
Effect Size ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Weighted Mean ◽  
Summary Effect ◽  
Different Types ◽  
Meta Analyses ◽  
Size Estimates ◽  
Similar Search

Multiple meta-analyses may use similar search criteria and focus on the same topic of interest, but they may yield different or sometimes discordant results. The lack of statistical methods for synthesizing these findings makes it challenging to properly interpret the results from multiple meta-analyses, especially when their results are conflicting. In this paper, we first introduce a method to synthesize the meta-analytic results when multiple meta-analyses use the same type of summary effect estimates. When meta-analyses use different types of effect sizes, the meta-analysis results cannot be directly combined. We propose a two-step frequentist procedure to first convert the effect size estimates to the same metric and then summarize them with a weighted mean estimate. Our proposed method offers several advantages over existing methods by Hemming et al. (2012). First, different types of summary effect sizes are considered. Second, our method provides the same overall effect size as conducting a meta-analysis on all individual studies from multiple meta-analyses. We illustrate the application of the proposed methods in two examples and discuss their implications for the field of meta-analysis.

scholarly journals The impact of studies with no events in both arms on meta-analysis of rare events: a simulation study using generalized linear mixed model

2021 ◽  
Author(s):  
Chang Xu ◽  
Lifeng Lin
Keyword(s):  
Effect Size ◽  
Mixed Model ◽  
Linear Mixed Model ◽  
Mean Squared Error ◽  
Meta Analysis ◽  
Unbiased Estimation ◽  
Double Zero ◽  
Squared Error ◽  
Meta Analyses ◽  
The Impact

AbstractObjectiveThe common approach to meta-analysis with double-zero studies is to remove such studies. Our previous work has confirmed that exclusion of these studies may impact the results. In this study, we undertook extensive simulations to investigate how the results of meta-analyses would be impacted in relation to the proportion of such studies.MethodsTwo standard generalized linear mixed models (GLMMs) were employed for the meta-analysis. The statistical properties of the two GLMMs were first examined in terms of percentage bias, mean squared error, and coverage. We then repeated all the meta-analyses after excluding double-zero studies. Direction of estimated effects and p-values for including against excluding double-zero studies were compared in nine ascending groups classified by the proportion of double-zero studies within a meta-analysis.ResultsBased on 50,000 simulated meta-analyses, the two GLMMs almost achieved unbiased estimation and reasonable coverage in most of the situations. When excluding double-zero studies, 0.00% to 4.47% of the meta-analyses changed the direction of effect size, and 0.61% to 8.78% changed direction of the significance of p-value. When the proportion of double-zero studies increased in a meta-analysis, the probability of the effect size changed the direction increased; when the proportion was about 40% to 60%, it has the largest impact on the change of p-values.ConclusionDouble-zero studies can impact the results of meta-analysis and excluding them may be problematic. The impact of such studies on meta-analysis varies by the proportion of such studies within a meta-analysis.

Systematic Reviews and Meta-Analyses of Crime Prevention Evaluations

2018 ◽  
Author(s):  
Anthony Petrosino ◽  
Claire Morgan ◽  
Trevor Fronius
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Effect Size ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Small Sample ◽  
Effect Sizes ◽  
What Works ◽  
Management Procedures ◽  
Meta Analyses

Systematic reviews and meta-analyses have become a focal point of evidence-based policy in criminology. Systematic reviews use explicit and transparent processes to identify, retrieve, code, analyze, and report on existing research studies bearing on a question of policy or practice. Meta-analysis can combine the results from the most rigorous evaluations identified in a systematic review to provide policymakers with the best evidence on what works for a variety of interventions relevant to reducing crime and making the justice system fairer and more effective. The steps of a systematic review using meta-analysis include specifying the topic area, developing management procedures, specifying the search strategy, developing eligibility criteria, extracting data from the studies, computing effect sizes, developing an analysis strategy, and interpreting and reporting the results. In a systematic review using meta-analysis, after identifying and coding eligible studies, the researchers create a measure of effect size for each experimental versus control contrast of interest in the study. Most commonly, reviewers do this by standardizing the difference between scores of the experimental and control groups, placing outcomes that are conceptually similar but measured differently (e.g., such as re-arrest or reconviction) on the same common scale or metric. Though these are different indices, they do measure a program’s effect on some construct (e.g., criminality). These effect sizes are usually averaged across all similar studies to provide a summary of program impact. The effect sizes also represent the dependent variable in the meta-analysis, and more advanced syntheses explore the role of potential moderating variables, such as sample size or other characteristics related to effect size. When done well and with full integrity, a systematic review using meta-analysis can provide the most comprehensive assessment of the available evaluative literature addressing the research question, as well as the most reliable statement about what works. Drawing from a larger body of research increases statistical power by reducing standard error; individual studies often use small sample sizes, which can result in large margins of error. In addition, conducting meta-analysis can be faster and less resource-intensive than replicating experimental studies. Using meta-analysis instead of relying on an individual program evaluation can help ensure that policy is guided by the totality of evidence, drawing upon a solid basis for generalizing outcomes.

scholarly journals The Relationship Between a Lifetime History of Sexual Victimization and Perinatal Depression: A Systematic Review and Meta-Analysis

2021 ◽  
pp. 152483802110216
Author(s):  
Brooke N. Lombardi ◽  
Todd M. Jensen ◽  
Anna B. Parisi ◽  
Melissa Jenkins ◽  
Sarah E. Bledsoe
Keyword(s):  
Systematic Review ◽  
Effect Size ◽  
Sexual Victimization ◽  
Perinatal Depression ◽  
Meta Analysis ◽  
Effect Size Estimate ◽  
Lifetime History ◽  
Size Estimate ◽  
History Of ◽  
Size Estimates

Background: The association between a lifetime history of sexual victimization and the well-being of women during the perinatal period has received increasing attention. However, research investigating this relationship has yet to be systematically reviewed or quantitatively synthesized. Aim: This systematic review and meta-analysis aims to calculate the pooled effect size estimate of the statistical association between a lifetime history of sexual victimization and perinatal depression (PND). Method: Four bibliographic databases were systematically searched, and reference harvesting was conducted to identify peer-reviewed articles that empirically examined associations between a lifetime history of sexual victimization and PND. A random effects model was used to ascertain an overall pooled effect size estimate in the form of an odds ratio and corresponding 95% confidence intervals (CIs). Subgroup analyses were also conducted to assess whether particular study features and sample characteristic (e.g., race and ethnicity) influenced the magnitude of effect size estimates. Results: This review included 36 studies, with 45 effect size estimates available for meta-analysis. Women with a lifetime history of sexual victimization had 51% greater odds of experiencing PND relative to women with no history of sexual victimization ( OR = 1.51, 95% CI [1.35, 1.67]). Effect size estimates varied considerably according to the PND instrument used in each study and the racial/ethnic composition of each sample. Conclusion: Findings provide compelling evidence for an association between a lifetime history of sexual victimization and PND. Future research should focus on screening practices and interventions that identify and support survivors of sexual victimization perinatally.

scholarly journals Cluster Wild Bootstrapping to Handle Dependent Effect Sizes in Meta-Analysis with a Small Number of Studies

2021 ◽  
Author(s):  
Megha Joshi ◽  
James E Pustejovsky ◽  
S. Natasha Beretvas
Keyword(s):  
Effect Size ◽  
Type I Error ◽  
Meta Analysis ◽  
Error Rates ◽  
Small Sample ◽  
Type I ◽  
Hypothesis Tests ◽  
Type I Error Rates ◽  
Meta Analyses ◽  
Small Sample Correction

The most common and well-known meta-regression models work under the assumption that there is only one effect size estimate per study and that the estimates are independent. However, meta-analytic reviews of social science research often include multiple effect size estimates per primary study, leading to dependence in the estimates. Some meta-analyses also include multiple studies conducted by the same lab or investigator, creating another potential source of dependence. An increasingly popular method to handle dependence is robust variance estimation (RVE), but this method can result in inflated Type I error rates when the number of studies is small. Small-sample correction methods for RVE have been shown to control Type I error rates adequately but may be overly conservative, especially for tests of multiple-contrast hypotheses. We evaluated an alternative method for handling dependence, cluster wild bootstrapping, which has been examined in the econometrics literature but not in the context of meta-analysis. Results from two simulation studies indicate that cluster wild bootstrapping maintains adequate Type I error rates and provides more power than extant small sample correction methods, particularly for multiple-contrast hypothesis tests. We recommend using cluster wild bootstrapping to conduct hypothesis tests for meta-analyses with a small number of studies. We have also created an R package that implements such tests.

scholarly journals Changes of Ankle Dorsiflexion Using Compression Tissue Flossing: A Systematic Review and Meta-Analysis

2020 ◽  
pp. 1-9
Author(s):  
Devin S. Kielur ◽  
Cameron J. Powden
Keyword(s):  
Confidence Interval ◽  
Range Of Motion ◽  
Effect Size ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Lower Extremity Injury ◽  
Level Of Evidence ◽  
Physically Active ◽  
Evidence Database

Context: Impaired dorsiflexion range of motion (DFROM) has been established as a predictor of lower-extremity injury. Compression tissue flossing (CTF) may address tissue restrictions associated with impaired DFROM; however, a consensus is yet to support these effects. Objectives: To summarize the available literature regarding CTF on DFROM in physically active individuals. Evidence Acquisition: PubMed and EBSCOhost (CINAHL, MEDLINE, and SPORTDiscus) were searched from 1965 to July 2019 for related articles using combination terms related to CTF and DRFOM. Articles were included if they measured the immediate effects of CTF on DFROM. Methodological quality was assessed using the Physiotherapy Evidence Database scale. The level of evidence was assessed using the Strength of Recommendation Taxonomy. The magnitude of CTF effects from pre-CTF to post-CTF and compared with a control of range of motion activities only were examined using Hedges g effect sizes and 95% confidence intervals. Randomeffects meta-analysis was performed to synthesize DFROM changes. Evidence Synthesis: A total of 6 studies were included in the analysis. The average Physiotherapy Evidence Database score was 60% (range = 30%–80%) with 4 out of 6 studies considered high quality and 2 as low quality. Meta-analysis indicated no DFROM improvements for CTF compared with range of motion activities only (effect size = 0.124; 95% confidence interval, −0.137 to 0.384; P = .352) and moderate improvements from pre-CTF to post-CTF (effect size = 0.455; 95% confidence interval, 0.022 to 0.889; P = .040). Conclusions: There is grade B evidence to suggest CTF may have no effect on DFROM when compared with a control of range of motion activities only and results in moderate improvements from pre-CTF to post-CTF. This suggests that DFROM improvements were most likely due to exercises completed rather than the band application.

Effect Sizes: Conventional Choices and Calculations

2013 ◽  
Author(s):  
Michael S. Rosenberg ◽  
Hannah R. Rothstein ◽  
Jessica Gurevitch
Keyword(s):  
Critical Point ◽  
Effect Size ◽  
Evolutionary Biology ◽  
Meta Analysis ◽  
Statistical Parameter ◽  
Effect Sizes ◽  
Common Effect ◽  
Inference Methods ◽  
Conventional Effect ◽  
Estimate Effect Size

One of the fundamental concepts in meta-analysis is that of the effect size. An effect size is a statistical parameter that can be used to compare, on the same scale, the results of different studies in which a common effect of interest has been measured. This chapter describes the conventional effect sizes most commonly encountered in ecology and evolutionary biology, and the types of data associated with them. While choice of a specific measure of effect size may influence the interpretation of results, it does not influence the actual inference methods of meta-analysis. One critical point to remember is that one cannot combine different measures of effect size in a single meta-analysis: once you have chosen how you are going to estimate effect size, you need to use it for all of the studies to be analyzed.

M. Meta-Analysis

2021 ◽  
pp. 163-165
Author(s):  
Noémie Laurens
Keyword(s):  
Literature Review ◽  
Empirical Research ◽  
Effect Size ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Effect Sizes ◽  
Primary Research ◽  
Unit Of Analysis ◽  
Research Studies

This chapter illustrates meta-analysis, which is a specific type of literature review, and more precisely a type of research synthesis, alongside traditional narrative reviews. Unlike in primary research, the unit of analysis of a meta-analysis is the results of individual studies. And unlike traditional reviews, meta-analysis only applies to: empirical research studies with quantitative findings hat are conceptually comparable and configured in similar statistical forms. What further distinguishes meta-analysis from other research syntheses is the method of synthesizing the results of studies — i.e. the use of statistics and, in particular, of effect sizes. An effect size represents the degree to which the phenomenon under study exists.

scholarly journals A-17 Category Test performance in individuals with alcohol versus methamphetamine dependence

2019 ◽  
Vol 34 (6) ◽  
pp. 876-876
Author(s):  
A Walker ◽  
A Hauson ◽  
S Sarkissians ◽  
A Pollard ◽  
C Flora-Tostado ◽  
...  
Keyword(s):  
Effect Size ◽  
Test Performance ◽  
Neuropsychological Functioning ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Significant Heterogeneity ◽  
Category Test ◽  
Methamphetamine Dependence ◽  
Significant Difference ◽  
Axis I

Abstract Objective The Category Test (CT) has consistently been found to be sensitive at detecting the effects of alcohol on the brain. However, this test has not been as widely used in examining the effects of methamphetamine. The current meta-analysis compared effect sizes of studies that have examined performance on the CT in alcohol versus methamphetamine dependent participants. Data selection Three researchers independently searched nine databases (e.g., PsycINFO, Pubmed, ProceedingsFirst), extracted required data, and calculated effect sizes. Inclusion criteria identified studies that had (a) compared alcohol or methamphetamine dependent groups to healthy controls and (b) matched groups on either age, education, or IQ (at least 2 out of 3). Studies were excluded if participants were reported to have Axis I diagnoses (other than alcohol or methamphetamine dependence) or comorbidities known to impact neuropsychological functioning. Sixteen articles were coded and analyzed for the current study. Data synthesis Alcohol studies showed a large effect size (g = 0.745, p < 0.001) while methamphetamine studies evidenced a moderate effect size (g = 0.406, p = 0.001); both without statistically significant heterogeneity (I2 = 0). Subgroup analysis revealed a statistically significant difference between the effect sizes from alcohol versus methamphetamine studies (Q-between = 5.647, p = 0.017). Conclusions The CT is sensitive to the effects of both alcohol and methamphetamine and should be considered when examining dependent patients who might exhibit problem solving, concept formation, and set loss difficulties in everyday living.

Sign in / Sign up

Export Citation Format

Share Document