Type I Error Rates and Power Estimates for Selected Two-Sample Tests of Scale

1989 ◽  
Vol 14 (4) ◽  
pp. 373-384 ◽  
Author(s):  
James Algina ◽  
Stephen Olejnik ◽  
Romer Ocanto
Keyword(s):  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Sample Sizes ◽  
Nominal Level ◽  
Nonnormal Distributions ◽  
Group Design ◽  
Type I Error Rates

Estimated Type I error rates and power are reported for a modified Fligner-Killeen test, O’Brien’s test, O’Brien’s test using Welch’s modified ANOVA, the Brown-Forsythe test, and two tests developed by Tiku. Normal and nonnormal distributions and a two-group design were investigated. O’Brien’s test and the Brown-Forsythe test had estimated Type I error rates near the nominal level for all conditions investigated. To maximize power when the sample sizes are equal, O’Brien’s test should be used with platykurtic distributions and the Brown-Forsythe test with leptokurtic distributions. Either test can be used when the kurtosis is zero. When the sample sizes are unequal, O’Brien’s test should be used with platykurtic distributions and the Brown-Forsythe with symmetric-leptokurtic distributions. With other distributions, the tests have similar power.

Type I Error Rates for Welch’s Test and James’s Second-Order Test Under Nonnormality and Inequality of Variance When There Are Two Groups

1994 ◽  
Vol 19 (3) ◽  
pp. 275-291 ◽  
Author(s):  
James Algina ◽  
T. C. Oshima ◽  
Wen-Ying Lin
Keyword(s):  
Degrees Of Freedom ◽  
Type I Error ◽  
Total Sample ◽  
Error Rates ◽  
Second Order ◽  
T Test ◽  
Type I ◽  
Sample Sizes ◽  
Unequal Variances ◽  
Type I Error Rates

Type I error rates were estimated for three tests that compare means by using data from two independent samples: the independent samples t test, Welch’s approximate degrees of freedom test, and James’s second-order test. Type I error rates were estimated for skewed distributions, equal and unequal variances, equal and unequal sample sizes, and a range of total sample sizes. Welch’s test and James’s test have very similar Type I error rates and tend to control the Type I error rate as well or better than the independent samples t test does. The results provide guidance about the total sample sizes required for controlling Type I error rates.

Performance of Monte Carlo Permutation and Approximate Tests for Multivariate Means Comparisons With Small Sample Sizes When Parametric Assumptions are Violated

Methodology ◽  
2009 ◽  
Vol 5 (2) ◽  
pp. 60-70 ◽  
Author(s):  
W. Holmes Finch ◽  
Teresa Davenport
Keyword(s):  
Monte Carlo ◽  
Type I Error ◽  
Permutation Tests ◽  
Error Rates ◽  
Covariance Matrices ◽  
Small Sample ◽  
Type I ◽  
Permutation Testing ◽  
Sample Sizes ◽  
Type I Error Rates

Permutation testing has been suggested as an alternative to the standard F approximate tests used in multivariate analysis of variance (MANOVA). These approximate tests, such as Wilks’ Lambda and Pillai’s Trace, have been shown to perform poorly when assumptions of normally distributed dependent variables and homogeneity of group covariance matrices were violated. Because Monte Carlo permutation tests do not rely on distributional assumptions, they may be expected to work better than their approximate cousins when the data do not conform to the assumptions described above. The current simulation study compared the performance of four standard MANOVA test statistics with their Monte Carlo permutation-based counterparts under a variety of conditions with small samples, including conditions when the assumptions were met and when they were not. Results suggest that for sample sizes of 50 subjects, power is very low for all the statistics. In addition, Type I error rates for both the approximate F and Monte Carlo tests were inflated under the condition of nonnormal data and unequal covariance matrices. In general, the performance of the Monte Carlo permutation tests was slightly better in terms of Type I error rates and power when both assumptions of normality and homogeneous covariance matrices were not met. It should be noted that these simulations were based upon the case with three groups only, and as such results presented in this study can only be generalized to similar situations.

scholarly journals Differences of Type I error rates for ANOVA and Multilevel-Linear-Models using SAS and SPSS for repeated measures designs

2019 ◽  
Vol 3 ◽  
Author(s):  
Nicolas Haverkamp ◽  
André Beauducel
Keyword(s):  
Repeated Measures ◽  
Linear Models ◽  
Type I Error ◽  
Error Rates ◽  
Small Sample ◽  
Small Samples ◽  
Type I ◽  
Sample Sizes ◽  
Type I Error Rates ◽  
Multilevel Linear Models

  To derive recommendations on how to analyze longitudinal data, we examined Type I error rates of Multilevel Linear Models (MLM) and repeated measures Analysis of Variance (rANOVA) using SAS and SPSS. We performed a simulation with the following specifications: To explore the effects of high numbers of measurement occasions and small sample sizes on Type I error, measurement occasions of m = 9 and 12 were investigated as well as sample sizes of n = 15, 20, 25 and 30. Effects of non-sphericity in the population on Type I error were also inspected: 5,000 random samples were drawn from two populations containing neither a within-subject nor a between-group effect. They were analyzed including the most common options to correct rANOVA and MLM-results: The Huynh-Feldt-correction for rANOVA (rANOVA-HF) and the Kenward-Roger-correction for MLM (MLM-KR), which could help to correct progressive bias of MLM with an unstructured covariance matrix (MLM-UN). Moreover, uncorrected rANOVA and MLM assuming a compound symmetry covariance structure (MLM-CS) were also taken into account. The results showed a progressive bias for MLM-UN for small samples which was stronger in SPSS than in SAS. Moreover, an appropriate bias correction for Type I error via rANOVA-HF and an insufficient correction by MLM-UN-KR for n < 30 were found. These findings suggest MLM-CS or rANOVA if sphericity holds and a correction of a violation via rANOVA-HF. If an analysis requires MLM, SPSS yields more accurate Type I error rates for MLM-CS and SAS yields more accurate Type I error rates for MLM-UN.

A Monte Carlo Comparison of Seven ε-Adjustment Procedures in Repeated Measures Designs With Small Sample Sizes

1994 ◽  
Vol 19 (1) ◽  
pp. 57-71 ◽  
Author(s):  
Stephen M. Quintana ◽  
Scott E. Maxwell
Keyword(s):  
Repeated Measures ◽  
Type I Error ◽  
Error Rates ◽  
Small Sample ◽  
Small Samples ◽  
Type I ◽  
Sample Sizes ◽  
Type I Error Rates ◽  
Repeated Measures Designs ◽  
Small Sample Sizes

The purpose of this study was to evaluate seven univariate procedures for testing omnibus null hypotheses for data gathered from repeated measures designs. Five alternate approaches are compared to the two more traditional adjustment procedures (Geisser and Greenhouse’s ε̂ and Huynh and Feldt’s ε̃), neither of which may be entirely adequate when sample sizes are small and the number of levels of the repeated factors is large. Empirical Type I error rates and power levels were obtained by simulation for conditions where small samples occur in combination with many levels of the repeated factor. Results suggested that alternate univariate approaches were improvements to the traditional approaches. One alternate approach in particular was found to be most effective in controlling Type I error rates without unduly sacrificing power.

Detecting Fraudulent Erasures at an Aggregate Level

2017 ◽  
Vol 43 (3) ◽  
pp. 286-315 ◽  
Author(s):  
Sandip Sinharay
Keyword(s):  
Type I Error ◽  
Null Distribution ◽  
Real Data ◽  
Error Rates ◽  
Standard Normal Distribution ◽  
Type I ◽  
Group Level ◽  
Nominal Level ◽  
Type I Error Rates ◽  
Standard Normal

Wollack, Cohen, and Eckerly suggested the “erasure detection index” (EDI) to detect fraudulent erasures for individual examinees. Wollack and Eckerly extended the EDI to detect fraudulent erasures at the group level. The EDI at the group level was found to be slightly conservative. This article suggests two modifications of the EDI for the group level. The asymptotic null distribution of the two modified indices is proved to be the standard normal distribution. In a simulation study, the modified indices are shown to have Type I error rates close to the nominal level and larger power than the index of Wollack and Eckerly. A real data example is also included.

A Maximum Test for Scale: Type I Error Rates and Power

1995 ◽  
Vol 20 (1) ◽  
pp. 27-39 ◽  
Author(s):  
James Algina ◽  
R. Clifford Blair ◽  
William T. Coombs
Keyword(s):  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Test Statistics ◽  
Test Statistic ◽  
Nominal Level ◽  
Scale Type ◽  
Type I Error Rates ◽  
Maximum Test ◽  
Study Type

A maximum test in which the test statistic is the more extreme of the Brown-Forsythe and O’Brien’s test statistics is developed. Estimated Type I error rates and power are presented for the Brown-Forsythe test, O’Brien’s test, and the maximum test. For the conditions included in the study, Type I error rates for the maximum test are near the nominal level. In all conditions, the power of the maximum test tended to be equal to or greater than that of the test—O’Brien or Brown-Forsythe—that had the larger power.

Sign in / Sign up

Export Citation Format

Share Document