Permutation tests for univariate or multivariate analysis of variance and regression

The most appropriate strategy to be used to create a permutation distribution for tests of individual terms in complex experimental designs is currently unclear. There are often many possibilities, including restricted permutation or permutation of some form of residuals. This paper provides a summary of recent empirical and theoretical results concerning available methods and gives recommendations for their use in univariate and multivariate applications. The focus of the paper is on complex designs in analysis of variance and multiple regression (i.e., linear models). The assumption of exchangeability required for a permutation test is assured by random allocation of treatments to units in experimental work. For observational data, exchangeability is tantamount to the assumption of independent and identically distributed errors under a null hypothesis. For partial regression, the method of permutation of residuals under a reduced model has been shown to provide the best test. For analysis of variance, one must first identify exchangeable units by considering expected mean squares. Then, one may generally produce either (i) an exact test by restricting permutations or (ii) an approximate test by permuting raw data or some form of residuals. The latter can provide a more powerful test in many situations.

Nonuniform Bounds for Nonparametric t-Tests

This paper gives simple nonuniform bounds on the tail areas of the permutation distribution of the usual Student's t-statistic when the observations are independent with symmetric distributions. As opposed to uniform bounds, nonuniform bounds depend on the observed sample. It is shown that the nonuniform bounds proposed are always tighter than uniform exponential bounds previously suggested. The use of the bounds to perform nonparametric t-tests is discussed and numerical examples are presented. Further, the bounds are extended to t-tests in the context of a simple linear regression.