Meta-Analysis with Heteroscedastic Effects

1993 ◽  
Vol 30 (2) ◽  
pp. 246-255 ◽  
Author(s):  
Murali Chandrashekaran ◽  
Beth A. Walker
Keyword(s):  
Simulation Experiment ◽  
Type I Error ◽  
Meta Analysis ◽  
Marketing Research ◽  
Estimation Procedure ◽  
Type I ◽  
Moderator Variables ◽  
Analytic Procedure ◽  
Wide Range ◽  
Analytic Dataset

To enhance the utility of meta-analysis as an integrative tool for marketing research, heteroscedastic MLE (HMLE), a maximum-likelihood-based estimation procedure, is proposed as a method that overcomes heteroscedasticity, a problem known to impair OLS estimates and threaten the validity of meta-analytic findings. The results of a Monté Carlo simulation experiment reveal that, under a wide range of heteroscedastic conditions, HMLE is more efficient and powerful than OLS and achieves these performance advantages without inflating type I error. Further, the relative performance of HMLE increases as heteroscedasticity becomes more severe. An empirical analysis of a meta-analytic dataset in marketing confirmed and extended these findings by illustrating how the enhanced efficiency and power of HMLE improve the ability to detect moderator variables and by demonstrating how the theoretical generalizations emerging from a meta-analysis are affected by the choice of the analytic procedure.

The Robustness of the Likelihood Ratio Chi-Square Test for Structural Equation Models: A Meta-Analysis

2001 ◽  
Vol 26 (1) ◽  
pp. 105-132 ◽  
Author(s):  
Douglas A. Powell ◽  
William D. Schafer
Keyword(s):  
Structural Equation ◽  
Structural Equation Models ◽  
Type I Error ◽  
Meta Analysis ◽  
Generalized Least Squares ◽  
Error Rates ◽  
Type I ◽  
Chi Square ◽  
Distribution Free ◽  
Projection Techniques

The robustness literature for the structural equation model was synthesized following the method of Harwell which employs meta-analysis as developed by Hedges and Vevea. The study focused on the explanation of empirical Type I error rates for six principal classes of estimators: two that assume multivariate normality (maximum likelihood and generalized least squares), elliptical estimators, two distribution-free estimators (asymptotic and others), and latent projection. Generally, the chi-square tests for overall model fit were found to be sensitive to non-normality and the size of the model for all estimators (with the possible exception of the elliptical estimators with respect to model size and the latent projection techniques with respect to non-normality). The asymptotic distribution-free (ADF) and latent projection techniques were also found to be sensitive to sample sizes. Distribution-free methods other than ADF showed, in general, much less sensitivity to all factors considered.

scholarly journals Cluster Wild Bootstrapping to Handle Dependent Effect Sizes in Meta-Analysis with a Small Number of Studies

2021 ◽  
Author(s):  
Megha Joshi ◽  
James E Pustejovsky ◽  
S. Natasha Beretvas
Keyword(s):  
Effect Size ◽  
Type I Error ◽  
Meta Analysis ◽  
Error Rates ◽  
Small Sample ◽  
Type I ◽  
Hypothesis Tests ◽  
Type I Error Rates ◽  
Meta Analyses ◽  
Small Sample Correction

The most common and well-known meta-regression models work under the assumption that there is only one effect size estimate per study and that the estimates are independent. However, meta-analytic reviews of social science research often include multiple effect size estimates per primary study, leading to dependence in the estimates. Some meta-analyses also include multiple studies conducted by the same lab or investigator, creating another potential source of dependence. An increasingly popular method to handle dependence is robust variance estimation (RVE), but this method can result in inflated Type I error rates when the number of studies is small. Small-sample correction methods for RVE have been shown to control Type I error rates adequately but may be overly conservative, especially for tests of multiple-contrast hypotheses. We evaluated an alternative method for handling dependence, cluster wild bootstrapping, which has been examined in the econometrics literature but not in the context of meta-analysis. Results from two simulation studies indicate that cluster wild bootstrapping maintains adequate Type I error rates and provides more power than extant small sample correction methods, particularly for multiple-contrast hypothesis tests. We recommend using cluster wild bootstrapping to conduct hypothesis tests for meta-analyses with a small number of studies. We have also created an R package that implements such tests.

scholarly journals A Likelihood Ratio Test For The Homogeneity of Between-Study Variance in Network Meta-Analysis

2021 ◽  
Author(s):  
Dapeng Hu ◽  
Chong Wang ◽  
Annette O'Connor
Keyword(s):  
Likelihood Ratio ◽  
Likelihood Ratio Test ◽  
Random Effects ◽  
Type I Error ◽  
Meta Analysis ◽  
Indirect Evidence ◽  
Point Estimate ◽  
Ratio Test ◽  
Type I ◽  
Statistical Heterogeneity

Abstract Background: Network meta-analysis (NMA) is a statistical method used to combine results from several clinical trials and simultaneously compare multiple treatments using direct and indirect evidence. Statistical heterogeneity is a characteristic describing the variability in the intervention effects being evaluated in the different studies in network meta-analysis. One approach to dealing with statistical heterogeneity is to perform a random effects network meta-analysis that incorporates a between-study variance into the statistical model. A common assumption in the random effects model for network meta-analysis is the homogeneity of between-study variance across all interventions. However, there are applications of NMA where the single between-study assumption is potentially incorrect and instead the model should incorporate more than one between-study variances. Methods: In this paper, we develop an approach to testing the homogeneity of between-study variance assumption based on a likelihood ratio test. A simulation study was conducted to assess the type I error and power of the proposed test. This method is then applied to a network meta-analysis of antibiotic treatments for Bovine respiratory disease (BRD). Results: The type I error rate was well controlled in the Monte Carlo simulation. The homogeneous between-study variance assumption is unrealistic both statistically and practically in the network meta-analysis BRD. The point estimate and conffdence interval of relative effect sizes are strongly inuenced by this assumption. Conclusions: Since homogeneous between-study variance assumption is a strong assumption, it is crucial to test the validity of this assumption before conducting a network meta-analysis. Here we propose and validate a method for testing this single between-study variance assumption which is widely used for many NMA.

Location Tests for Biomarker Studies: A Comparison Using Simulations for the Two-sample Case

2013 ◽  
Vol 52 (04) ◽  
pp. 351-359 ◽  
Author(s):  
M. O. Scheinhardt ◽  
A. Ziegler
Keyword(s):  
Heavy Tails ◽  
Type I Error ◽  
Adaptive Methods ◽  
Type I ◽  
Skewed Data ◽  
Error Level ◽  
Adaptive Tests ◽  
Wide Range ◽  
Heavy Tailed ◽  
Non Parametric

Summary Background: Gene, protein, or metabolite expression levels are often non-normally distributed, heavy tailed and contain outliers. Standard statistical approaches may fail as location tests in this situation. Objectives: In three Monte-Carlo simulation studies, we aimed at comparing the type I error levels and empirical power of standard location tests and three adaptive tests [O’Gorman, Can J Stat 1997; 25: 269 –279; Keselman et al., Brit J Math Stat Psychol 2007; 60: 267– 293; Szymczak et al., Stat Med 2013; 32: 524 – 537] for a wide range of distributions. Methods: We simulated two-sample scena -rios using the g-and-k-distribution family to systematically vary tail length and skewness with identical and varying variability between groups. Results: All tests kept the type I error level when groups did not vary in their variability. The standard non-parametric U-test per -formed well in all simulated scenarios. It was outperformed by the two non-parametric adaptive methods in case of heavy tails or large skewness. Most tests did not keep the type I error level for skewed data in the case of heterogeneous variances. Conclusions: The standard U-test was a powerful and robust location test for most of the simulated scenarios except for very heavy tailed or heavy skewed data, and it is thus to be recommended except for these cases. The non-parametric adaptive tests were powerful for both normal and non-normal distributions under sample variance homogeneity. But when sample variances differed, they did not keep the type I error level. The parametric adaptive test lacks power for skewed and heavy tailed distributions.

Type I error and statistical power of the Mantel-Haenszel procedure for detecting DIF: A meta-analysis.

2013 ◽  
Vol 18 (4) ◽  
pp. 553-571 ◽  
Author(s):  
Georgina Guilera ◽  
Juana Gómez-Benito ◽  
Maria Dolores Hidalgo ◽  
Julio Sánchez-Meca
Keyword(s):  
Statistical Power ◽  
Type I Error ◽  
Meta Analysis ◽  
Type I

scholarly journals Combining the strengths of inverse-variance weighting and Egger regression in Mendelian randomization using a mixture of regressions model

PLoS Genetics ◽  
2021 ◽  
Vol 17 (11) ◽  
pp. e1009922
Author(s):  
Zhaotong Lin ◽  
Yangqing Deng ◽  
Wei Pan
Keyword(s):  
Large Scale ◽  
Type I Error ◽  
Mendelian Randomization ◽  
Meta Analysis ◽  
Type I ◽  
Perturbation Scheme ◽  
Analysis Model ◽  
Component Mixture ◽  
Inverse Variance ◽  
Summary Data

With the increasing availability of large-scale GWAS summary data on various traits, Mendelian randomization (MR) has become commonly used to infer causality between a pair of traits, an exposure and an outcome. It depends on using genetic variants, typically SNPs, as instrumental variables (IVs). The inverse-variance weighted (IVW) method (with a fixed-effect meta-analysis model) is most powerful when all IVs are valid; however, when horizontal pleiotropy is present, it may lead to biased inference. On the other hand, Egger regression is one of the most widely used methods robust to (uncorrelated) pleiotropy, but it suffers from loss of power. We propose a two-component mixture of regressions to combine and thus take advantage of both IVW and Egger regression; it is often both more efficient (i.e. higher powered) and more robust to pleiotropy (i.e. controlling type I error) than either IVW or Egger regression alone by accounting for both valid and invalid IVs respectively. We propose a model averaging approach and a novel data perturbation scheme to account for uncertainties in model/IV selection, leading to more robust statistical inference for finite samples. Through extensive simulations and applications to the GWAS summary data of 48 risk factor-disease pairs and 63 genetically uncorrelated trait pairs, we showcase that our proposed methods could often control type I error better while achieving much higher power than IVW and Egger regression (and sometimes than several other new/popular MR methods). We expect that our proposed methods will be a useful addition to the toolbox of Mendelian randomization for causal inference.

scholarly journals On the Conditional and Unconditional Type I Error Rates and Power of Tests in Linear Models with Heteroscedastic Errors

2019 ◽  
Vol 17 (2) ◽  
Author(s):  
Patrick J. Rosopa ◽  
Alice M. Brawley ◽  
Theresa P. Atkinson ◽  
Stephen A. Robertson
Keyword(s):  
Linear Models ◽  
Type I Error ◽  
Weighted Least Squares ◽  
Error Rates ◽  
Type I ◽  
Least Squares Regression ◽  
Type I Error Rates ◽  
Power Of Tests ◽  
Wide Range ◽  
Heteroscedastic Errors

Preliminary tests for homoscedasticity may be unnecessary in general linear models. Based on Monte Carlo simulations, results suggest that when testing for differences between independent slopes, the unconditional use of weighted least squares regression and HC4 regression performed the best across a wide range of conditions.

Inconsistencies in Reported p-Values in Spanish Journals of Psychology

Methodology ◽  
2016 ◽  
Vol 12 (2) ◽  
pp. 44-51 ◽  
Author(s):  
José Manuel Caperos ◽  
Ricardo Olmos ◽  
Antonio Pardo
Keyword(s):  
Type I Error ◽  
Meta Analysis ◽  
P Value ◽  
Type I ◽  
Simultaneous Inference ◽  
Test Statistics ◽  
Type I Errors ◽  
P Values ◽  
Editorial Boards ◽  
Correlation Tests

Abstract. Correlation analysis is one of the most widely used methods to test hypotheses in social and health sciences; however, its use is not completely error free. We have explored the frequency of inconsistencies between reported p-values and the associated test statistics in 186 papers published in four Spanish journals of psychology (1,950 correlation tests); we have also collected information about the use of one- versus two-tailed tests in the presence of directional hypotheses, and about the use of some kind of adjustment to control Type I errors due to simultaneous inference. Reported correlation tests (83.8%) are incomplete and 92.5% include an inexact p-value. Gross inconsistencies, which are liable to alter the statistical conclusions, appear in 4% of the reviewed tests, and 26.9% of the inconsistencies found were large enough to bias the results of a meta-analysis. The election of one-tailed tests and the use of adjustments to control the Type I error rate are negligible. We therefore urge authors, reviewers, and editorial boards to pay particular attention to this in order to prevent inconsistencies in statistical reports.

Sign in / Sign up

Export Citation Format

Share Document