Fitting a Serial Correlation Pattern to Repeated Observations

1991 ◽  
Vol 16 (1) ◽  
pp. 53-76
Author(s):  
Lynne K. Edwards
Keyword(s):  
Correction Factor ◽  
Serial Correlation ◽  
Degrees Of Freedom ◽  
Type I Error ◽  
Error Rates ◽  
Stationary Time Series ◽  
Type I ◽  
Correlation Pattern ◽  
Time Effects ◽  
Repeated Observations

When repeated observations are taken at equal time intervals, a simple form of a stationary time series structure may be fitted to the observations. Wallenstein and Fleiss (1979) have shown that the degrees-of-freedom correction factor for time effects has a higher lowerbound for data with a serial correlation pattern (or a simplex pattern) than for data without such a structure. The reanalysis of the example data found in Hearne, Clark, and Hatch (1983) indicated that the correction factor from a patterned matrix could be smaller than the counterpart without fitting a simplex pattern. First, an example from education was used to illustrate the computational steps in obtaining these two correction factors. Second, a simulation study was conducted to determine the conditions under which fitting a simplex pattern would be advantageous over not assuming such a pattern. Fitting a serial correlation pattern did not always produce more powerful tests of time effects than not assuming such a pattern. This was particularly true when correlations were high (ρ > .50). Furthermore, it inflated Type I error rates when the simplex shypothesis was not warranted. Indiscriminately fitting a serial correlation pattern should be discouraged.

scholarly journals Statistical model specification and power: recommendations on the use of test-qualified pooling in analysis of experimental data

2017 ◽  
Vol 284 (1851) ◽  
pp. 20161850 ◽  
Author(s):  
Nick Colegrave ◽  
Graeme D. Ruxton
Keyword(s):  
Experimental Data ◽  
Statistical Model ◽  
Statistical Power ◽  
Error Term ◽  
Degrees Of Freedom ◽  
Type I Error ◽  
Error Rates ◽  
Statistical Testing ◽  
Model Specification ◽  
Type I

A common approach to the analysis of experimental data across much of the biological sciences is test-qualified pooling. Here non-significant terms are dropped from a statistical model, effectively pooling the variation associated with each removed term with the error term used to test hypotheses (or estimate effect sizes). This pooling is only carried out if statistical testing on the basis of applying that data to a previous more complicated model provides motivation for this model simplification; hence the pooling is test-qualified. In pooling, the researcher increases the degrees of freedom of the error term with the aim of increasing statistical power to test their hypotheses of interest. Despite this approach being widely adopted and explicitly recommended by some of the most widely cited statistical textbooks aimed at biologists, here we argue that (except in highly specialized circumstances that we can identify) the hoped-for improvement in statistical power will be small or non-existent, and there is likely to be much reduced reliability of the statistical procedures through deviation of type I error rates from nominal levels. We thus call for greatly reduced use of test-qualified pooling across experimental biology, more careful justification of any use that continues, and a different philosophy for initial selection of statistical models in the light of this change in procedure.

Type I Error Rates for Welch’s Test and James’s Second-Order Test Under Nonnormality and Inequality of Variance When There Are Two Groups

1994 ◽  
Vol 19 (3) ◽  
pp. 275-291 ◽  
Author(s):  
James Algina ◽  
T. C. Oshima ◽  
Wen-Ying Lin
Keyword(s):  
Degrees Of Freedom ◽  
Type I Error ◽  
Total Sample ◽  
Error Rates ◽  
Second Order ◽  
T Test ◽  
Type I ◽  
Sample Sizes ◽  
Unequal Variances ◽  
Type I Error Rates

Type I error rates were estimated for three tests that compare means by using data from two independent samples: the independent samples t test, Welch’s approximate degrees of freedom test, and James’s second-order test. Type I error rates were estimated for skewed distributions, equal and unequal variances, equal and unequal sample sizes, and a range of total sample sizes. Welch’s test and James’s test have very similar Type I error rates and tend to control the Type I error rate as well or better than the independent samples t test does. The results provide guidance about the total sample sizes required for controlling Type I error rates.

scholarly journals PSII-7 Type I error rates of two strategies to analyze a randomized complete block design within multiple sites

2019 ◽  
Vol 97 (Supplement_2) ◽  
pp. 235-236
Author(s):  
Hilda Calderon Cartagena ◽  
Christopher I Vahl ◽  
Steve S Dritz
Keyword(s):  
Degrees Of Freedom ◽  
Type I Error ◽  
Block Design ◽  
Error Rates ◽  
Type I ◽  
Block Designs ◽  
Significant Treatment ◽  
Type I Error Rates ◽  
Treatment Interaction ◽  
To Come

Abstract It is not unusual to come across randomized complete block designs (RCBD) replicated over a small number of sites in swine nutrition trials. For example, pens could be blocked by location or by initial body weight within three rooms or barns. One possibility is to analyze this design with the assumption of no treatment by site interaction which implies treatment differences are similar across all sites. This assumption might not always seem reasonable and site by treatment interaction could be included in the analysis to account for these differences should they exist. However, the site by treatment mean square becomes the error term for evaluating treatment. The objective of this study was to provide a recommendation of a practical strategy based on Type I error rates estimated from a simulation study. Scenarios with and without site by treatment interaction were considered with three sites and equal means across four treatments. The variance component for the error was set to 1 and the rest were either selected to be equal (σ2s = σ2b = σ2s*t =1) or one of them was set to 10. For the scenarios with no site by treatment interaction, σ2s*t = 0, for a total of 7 scenarios. Each scenario was simulated 10,000 times. For each simulation, both strategies were applied. The Kenward-Rodger approximation (KR) to the denominator degrees of freedom was also considered. Type I errors were estimated as the proportion of simulations with a significant treatment effect with α = 0.05. Overall, there was no evidence Type I error rates were inflated when the site by treatment interaction was omitted, even when σ2s*t = 10. The KR had no effect. In contrast, including the interaction term leads to a highly conservative Type I error rate far below the 5% level which results in a reduction of power; however, using KR mitigated the conservativeness.

The Runs Test for Autocorrelated Errors: Unacceptable Properties

1996 ◽  
Vol 21 (4) ◽  
pp. 390-404 ◽  
Author(s):  
Bradley E. Huitema ◽  
Joseph W. McKean ◽  
Jinsheng Zhao
Keyword(s):  
Time Series ◽  
Regression Models ◽  
Degrees Of Freedom ◽  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Time Series Regression ◽  
Monte Carlo Investigation ◽  
Autocorrelated Errors ◽  
Asymmetrical Error

The runs test is frequently recommended as a method of testing for nonindependent errors in time-series regression models. A Monte Carlo investigation was carried out to evaluate the empirical properties of this test using (a) several intervention and nonintervention regression models, (b) sample sizes ranging from 12 to 100, (c) three levels of α, (d) directional and nondirectional tests, and (e) 19 levels of autocorrelation among the errors. The results indicate that the runs test yields markedly asymmetrical error rates in the two tails and that neither directional nor nondirectional tests are satisfactory with respect to Type I error, even when the ratio of degrees of freedom to sample size is as high as .98. It is recommended that the test generally not be employed in evaluating the independence of the errors in time-series regression models.

scholarly journals Robustness of T-test Based on Skewness and Kurtosis

2021 ◽  
pp. 102-110
Author(s):  
Steven T. Garren ◽  
Kate McGann Osborne
Keyword(s):  
Degrees Of Freedom ◽  
Type I Error ◽  
Error Rates ◽  
T Test ◽  
Type I ◽  
Skewed Distributions ◽  
Symmetric Distributions ◽  
Type I Error Rates ◽  
Coverage Probabilities ◽  
Skewness And Kurtosis

Coverage probabilities of the two-sided one-sample t-test are simulated for some symmetric and right-skewed distributions. The symmetric distributions analyzed are Normal, Uniform, Laplace, and student-t with 5, 7, and 10 degrees of freedom. The right-skewed distributions analyzed are Exponential and Chi-square with 1, 2, and 3 degrees of freedom. Left-skewed distributions were not analyzed without loss of generality. The coverage probabilities for the symmetric distributions tend to achieve or just barely exceed the nominal values. The coverage probabilities for the skewed distributions tend to be too low, indicating high Type I error rates. Percentiles for the skewness and kurtosis statistics are simulated using Normal data. For sample sizes of 5, 10, 15 and 20 the skewness statistic does an excellent job of detecting non-Normal data, except for Uniform data. The kurtosis statistic also does an excellent job of detecting non-Normal data, including Uniform data. Examined herein are Type I error rates, but not power calculations. We nd that sample skewness is unhelpful when determining whether or not the t-test should be used, but low sample kurtosis is reason to avoid using the t-test.

Correction: “Influence of Selection Bias on the Test Decision – A Simulation Study”

2014 ◽  
Vol 53 (05) ◽  
pp. 343-343
Keyword(s):  
Selection Bias ◽  
Simulation Study ◽  
Error Rate ◽  
Type I Error ◽  
Block Size ◽  
Error Rates ◽  
Type I ◽  
Type I Error Rate ◽  
Representation Error ◽  
Numeric Representation

We have to report marginal changes in the empirical type I error rates for the cut-offs 2/3 and 4/7 of Table 4, Table 5 and Table 6 of the paper “Influence of Selection Bias on the Test Decision – A Simulation Study” by M. Tamm, E. Cramer, L. N. Kennes, N. Heussen (Methods Inf Med 2012; 51: 138 –143). In a small number of cases the kind of representation of numeric values in SAS has resulted in wrong categorization due to a numeric representation error of differences. We corrected the simulation by using the round function of SAS in the calculation process with the same seeds as before. For Table 4 the value for the cut-off 2/3 changes from 0.180323 to 0.153494. For Table 5 the value for the cut-off 4/7 changes from 0.144729 to 0.139626 and the value for the cut-off 2/3 changes from 0.114885 to 0.101773. For Table 6 the value for the cut-off 4/7 changes from 0.125528 to 0.122144 and the value for the cut-off 2/3 changes from 0.099488 to 0.090828. The sentence on p. 141 “E.g. for block size 4 and q = 2/3 the type I error rate is 18% (Table 4).” has to be replaced by “E.g. for block size 4 and q = 2/3 the type I error rate is 15.3% (Table 4).”. There were only minor changes smaller than 0.03. These changes do not affect the interpretation of the results or our recommendations.

The Use of Theory of Linear Mixed-Effects Models to Detect Fraudulent Erasures at an Aggregate Level

2021 ◽  
pp. 001316442199489
Author(s):  
Luyao Peng ◽  
Sandip Sinharay
Keyword(s):  
Type I Error ◽  
Real Data ◽  
Mixed Effects ◽  
Error Rates ◽  
Mixed Effects Models ◽  
Type I ◽  
Aggregate Level ◽  
Linear Mixed Effects Models ◽  
Linear Mixed Effects ◽  
Best Linear Unbiased

Wollack et al. (2015) suggested the erasure detection index (EDI) for detecting fraudulent erasures for individual examinees. Wollack and Eckerly (2017) and Sinharay (2018) extended the index of Wollack et al. (2015) to suggest three EDIs for detecting fraudulent erasures at the aggregate or group level. This article follows up on the research of Wollack and Eckerly (2017) and Sinharay (2018) and suggests a new aggregate-level EDI by incorporating the empirical best linear unbiased predictor from the literature of linear mixed-effects models (e.g., McCulloch et al., 2008). A simulation study shows that the new EDI has larger power than the indices of Wollack and Eckerly (2017) and Sinharay (2018). In addition, the new index has satisfactory Type I error rates. A real data example is also included.

Sign in / Sign up

Export Citation Format

Share Document