A Practical Method for Analyzing Factorial Designs with Heteroscedastic Data

2008 ◽  
Vol 102 (3) ◽  
pp. 643-656
Author(s):  
Guiillermo Vallejo ◽  
M. Paula Fernández ◽  
Manuel Ato ◽  
Pablo E. Livacic-Rojas
Keyword(s):  
Mixed Model ◽  
Type I Error ◽  
Practical Method ◽  
Error Rates ◽  
Type I ◽  
Factorial Designs ◽  
Symmetric Distributions ◽  
Higher Power ◽  
Asymmetric Distributions ◽  
Heteroscedastic Data

The Type I error rates and powers of three recent tests for analyzing nonorthogonal factorial designs under departures from the assumptions of homogeneity and normality were evaluated using Monte Carlo simulation. Specifically, this work compared the performance of the modified Brown-Forsythe procedure, the generalization of Box's method proposed by Brunner, Dette, and Munk, and the mixed-model procedure adjusted by the Kenward-Roger solution available in the SAS statistical package. With regard to robustness, the three approaches adequately controlled Type I error when the data were generated from symmetric distributions; however, this study's results indicate that, when the data were extracted from asymmetric distributions, the modified Brown-Forsythe approach controlled the Type I error slightly better than the other procedures. With regard to sensitivity, the higher power rates were obtained when the analyses were done with the MIXED procedure of the SAS program. Furthermore, results also identified that, when the data were generated from symmetric distributions, little power was sacrificed by using the generalization of Box's method in place of the modified Brown-Forsythe procedure.

scholarly journals A comparison of empirical BLUP with different considerations of residual error variance for genotype evaluation of multi-location trials

2019 ◽  
Vol 17 (1) ◽  
pp. e0701
Author(s):  
Renhe Zhang ◽  
Xiyuan Hu
Keyword(s):  
Mixed Model ◽  
Type I Error ◽  
Block Design ◽  
Model Fitting ◽  
Covariance Structure ◽  
Error Variance ◽  
Error Rates ◽  
Residual Error ◽  
Type I ◽  
The North

AbstractThe empirical best linear unbiased prediction (eBLUP) is usually based on the assumption that the residual error variance (REV) is homogenous. This may be unrealistic, and therefore limits the accuracy of genotype evaluations for multi-location trials, where the REV often varies across locations. The objective of this contribution was to investigate the direct implications of the eBLUP with different considerations about REV based on the mixed model for evaluation of genotype simple effects (i.e. genotype effects at individual locations). A series of 14 multi-location trials from a rape-breeding program in the north of China were simultaneously analyzed from 2012 to 2014 using a randomized complete block design at each location. The results showed that the model with heterogeneous REV was more appropriate than the one with homogeneous REV in all of the trials according to model fitting statistics. Whether the REV differences across locations were accounted for in the analysis procedure influenced the variance estimate of related random effects and testing of the variance of genotype-location (G-L) interactions. Ignoring REV differences by use of the eBLUP could result not only in an inflation or deflation of statistical Type I error rates for pair-wise testing but also in an inaccurate ranking of genotype simple effects for these trials. Therefore, it is suggested that in application of the eBLUP for evaluation of genotype simple effects in multi-location trials, the heterogeneity of REV should be accounted for based on mixed model approaches with appropriate variance-covariance structure.

Combined 5 × 2 cv F Test for Comparing Supervised Classification Learning Algorithms

1999 ◽  
Vol 11 (8) ◽  
pp. 1885-1892 ◽  
Author(s):  
Ethem Alpaydm
Keyword(s):  
Type I Error ◽  
Statistical Tests ◽  
Error Rates ◽  
Type I ◽  
Classification Learning ◽  
F Test ◽  
Robust Test ◽  
Significant Difference ◽  
Higher Power ◽  
Test Result

Dietterich (1998) reviews five statistical tests and proposes the 5 × 2 cvt test for determining whether there is a significant difference between the error rates of two classifiers. In our experiments, we noticed that the 5 × 2 cvt test result may vary depending on factors that should not affect the test, and we propose a variant, the combined 5 × 2 cv F test, that combines multiple statistics to get a more robust test. Simulation results show that this combined version of the test has lower type I error and higher power than 5 × 2 cv proper.

scholarly journals Efficient mixed model approach for large-scale genome-wide association studies of ordinal categorical phenotypes

2020 ◽  
Author(s):  
Wenjian Bi ◽  
Wei Zhou ◽  
Rounak Dey ◽  
Bhramar Mukherjee ◽  
Joshua N Sampson ◽  
...  
Keyword(s):  
Mixed Model ◽  
Type I Error ◽  
Association Studies ◽  
Error Rates ◽  
Genome Wide Association ◽  
Alternative Methods ◽  
Type I ◽  
Genome Wide Association Studies ◽  
Type I Error Rates ◽  
Genome Wide

AbstractIn genome-wide association studies (GWAS), ordinal categorical phenotypes are widely used to measure human behaviors, satisfaction, and preferences. However, due to the lack of analysis tools, methods designed for binary and quantitative traits have often been used inappropriately to analyze categorical phenotypes, which produces inflated type I error rates or is less powerful. To accurately model the dependence of an ordinal categorical phenotype on covariates, we propose an efficient mixed model association test, Proportional Odds Logistic Mixed Model (POLMM). POLMM is demonstrated to be computationally efficient to analyze large datasets with hundreds of thousands of genetic related samples, can control type I error rates at a stringent significance level regardless of the phenotypic distribution, and is more powerful than other alternative methods. We applied POLMM to 258 ordinal categorical phenotypes on array-genotypes and imputed samples from 408,961 individuals in UK Biobank. In total, we identified 5,885 genome-wide significant variants, of which 424 variants (7.2%) are rare variants with MAF < 0.01.

scholarly journals Robustness of T-test Based on Skewness and Kurtosis

2021 ◽  
pp. 102-110
Author(s):  
Steven T. Garren ◽  
Kate McGann Osborne
Keyword(s):  
Degrees Of Freedom ◽  
Type I Error ◽  
Error Rates ◽  
T Test ◽  
Type I ◽  
Skewed Distributions ◽  
Symmetric Distributions ◽  
Type I Error Rates ◽  
Coverage Probabilities ◽  
Skewness And Kurtosis

Coverage probabilities of the two-sided one-sample t-test are simulated for some symmetric and right-skewed distributions. The symmetric distributions analyzed are Normal, Uniform, Laplace, and student-t with 5, 7, and 10 degrees of freedom. The right-skewed distributions analyzed are Exponential and Chi-square with 1, 2, and 3 degrees of freedom. Left-skewed distributions were not analyzed without loss of generality. The coverage probabilities for the symmetric distributions tend to achieve or just barely exceed the nominal values. The coverage probabilities for the skewed distributions tend to be too low, indicating high Type I error rates. Percentiles for the skewness and kurtosis statistics are simulated using Normal data. For sample sizes of 5, 10, 15 and 20 the skewness statistic does an excellent job of detecting non-Normal data, except for Uniform data. The kurtosis statistic also does an excellent job of detecting non-Normal data, including Uniform data. Examined herein are Type I error rates, but not power calculations. We nd that sample skewness is unhelpful when determining whether or not the t-test should be used, but low sample kurtosis is reason to avoid using the t-test.

Correction: “Influence of Selection Bias on the Test Decision – A Simulation Study”

2014 ◽  
Vol 53 (05) ◽  
pp. 343-343
Keyword(s):  
Selection Bias ◽  
Simulation Study ◽  
Error Rate ◽  
Type I Error ◽  
Block Size ◽  
Error Rates ◽  
Type I ◽  
Type I Error Rate ◽  
Representation Error ◽  
Numeric Representation

We have to report marginal changes in the empirical type I error rates for the cut-offs 2/3 and 4/7 of Table 4, Table 5 and Table 6 of the paper “Influence of Selection Bias on the Test Decision – A Simulation Study” by M. Tamm, E. Cramer, L. N. Kennes, N. Heussen (Methods Inf Med 2012; 51: 138 –143). In a small number of cases the kind of representation of numeric values in SAS has resulted in wrong categorization due to a numeric representation error of differences. We corrected the simulation by using the round function of SAS in the calculation process with the same seeds as before. For Table 4 the value for the cut-off 2/3 changes from 0.180323 to 0.153494. For Table 5 the value for the cut-off 4/7 changes from 0.144729 to 0.139626 and the value for the cut-off 2/3 changes from 0.114885 to 0.101773. For Table 6 the value for the cut-off 4/7 changes from 0.125528 to 0.122144 and the value for the cut-off 2/3 changes from 0.099488 to 0.090828. The sentence on p. 141 “E.g. for block size 4 and q = 2/3 the type I error rate is 18% (Table 4).” has to be replaced by “E.g. for block size 4 and q = 2/3 the type I error rate is 15.3% (Table 4).”. There were only minor changes smaller than 0.03. These changes do not affect the interpretation of the results or our recommendations.

The Use of Theory of Linear Mixed-Effects Models to Detect Fraudulent Erasures at an Aggregate Level

2021 ◽  
pp. 001316442199489
Author(s):  
Luyao Peng ◽  
Sandip Sinharay
Keyword(s):  
Type I Error ◽  
Real Data ◽  
Mixed Effects ◽  
Error Rates ◽  
Mixed Effects Models ◽  
Type I ◽  
Aggregate Level ◽  
Linear Mixed Effects Models ◽  
Linear Mixed Effects ◽  
Best Linear Unbiased

Wollack et al. (2015) suggested the erasure detection index (EDI) for detecting fraudulent erasures for individual examinees. Wollack and Eckerly (2017) and Sinharay (2018) extended the index of Wollack et al. (2015) to suggest three EDIs for detecting fraudulent erasures at the aggregate or group level. This article follows up on the research of Wollack and Eckerly (2017) and Sinharay (2018) and suggests a new aggregate-level EDI by incorporating the empirical best linear unbiased predictor from the literature of linear mixed-effects models (e.g., McCulloch et al., 2008). A simulation study shows that the new EDI has larger power than the indices of Wollack and Eckerly (2017) and Sinharay (2018). In addition, the new index has satisfactory Type I error rates. A real data example is also included.

scholarly journals Testing for equality of distributions using the concept of (niche) overlap

2021 ◽  
Author(s):  
Judith H. Parkinson-Schwarz ◽  
Arne C. Bathke
Keyword(s):  
Basic Assumption ◽  
Type I Error ◽  
Test Procedure ◽  
Niche Overlap ◽  
Relative Effect ◽  
Type I ◽  
Underlying Distribution ◽  
Parametric Test ◽  
Higher Power ◽  
Non Parametric

AbstractIn this paper, we propose a new non-parametric test for equality of distributions. The test is based on the recently introduced measure of (niche) overlap and its rank-based estimator. As the estimator makes only one basic assumption on the underlying distribution, namely continuity, the test is universal applicable in contrast to many tests that are restricted to only specific scenarios. By construction, the new test is capable of detecting differences in location and scale. It thus complements the large class of rank-based tests that are constructed based on the non-parametric relative effect. In simulations this new test procedure obtained higher power and lower type I error compared to two common tests in several settings. The new procedure shows overall good performance. Together with its simplicity, this test can be used broadly.

The Robustness of the Likelihood Ratio Chi-Square Test for Structural Equation Models: A Meta-Analysis

2001 ◽  
Vol 26 (1) ◽  
pp. 105-132 ◽  
Author(s):  
Douglas A. Powell ◽  
William D. Schafer
Keyword(s):  
Structural Equation ◽  
Structural Equation Models ◽  
Type I Error ◽  
Meta Analysis ◽  
Generalized Least Squares ◽  
Error Rates ◽  
Type I ◽  
Chi Square ◽  
Distribution Free ◽  
Projection Techniques

The robustness literature for the structural equation model was synthesized following the method of Harwell which employs meta-analysis as developed by Hedges and Vevea. The study focused on the explanation of empirical Type I error rates for six principal classes of estimators: two that assume multivariate normality (maximum likelihood and generalized least squares), elliptical estimators, two distribution-free estimators (asymptotic and others), and latent projection. Generally, the chi-square tests for overall model fit were found to be sensitive to non-normality and the size of the model for all estimators (with the possible exception of the elliptical estimators with respect to model size and the latent projection techniques with respect to non-normality). The asymptotic distribution-free (ADF) and latent projection techniques were also found to be sensitive to sample sizes. Distribution-free methods other than ADF showed, in general, much less sensitivity to all factors considered.

Sign in / Sign up

Export Citation Format

Share Document