Statistical procedures for testing hypotheses of equivalence in the safety evaluation of a genetically modified crop

2016 ◽  
Vol 154 (8) ◽  
pp. 1392-1412 ◽  
Author(s):  
Q. KANG ◽  
C. I. VAHL
Keyword(s):  
Type I Error ◽  
Genetically Modified ◽  
Safety Evaluation ◽  
Type I ◽  
Genetically Modified Crop ◽  
Balanced Designs ◽  
Pros And Cons ◽  
Modified Large Sample ◽  
User Friendly ◽  
Generalized Pivotal Quantities

SUMMARYSafety evaluation of a genetically modified crop entails assessing its equivalence to conventional crops under multi-site randomized block field designs. Despite mounting petitions for regulatory approval, there lack a scientifically sound and powerful statistical method for establishing equivalence. The current paper develops and validates two procedures for testing a recently identified class of equivalence uniquely suited to crop safety. One procedure employs the modified large sample (MLS) method; the other is based on generalized pivotal quantities (GPQs). Because both methods were originally created under balanced designs, common issues associated with incomplete and unbalanced field designs were addressed by first identifying unfulfilled theoretical assumptions and then replacing them with user-friendly approximations. Simulation indicated that the MLS procedure could be very conservative in many occasions irrespective of the balance of the design; the GPQ procedure was mildly liberal with its type I error rate near the nominal level when the design is balanced. Additional pros and cons of these two procedures are also discussed. Their utility is demonstrated in a case study using summary statistics derived from a real-world dataset.

Testing Repeated Measures Hypotheses When Covariance Matrices are Heterogeneous

1993 ◽  
Vol 18 (4) ◽  
pp. 305-319 ◽  
Author(s):  
H. J. Keselman ◽  
Keumhee Chough Carriere ◽  
Lisa M. Lix
Keyword(s):  
Interaction Effect ◽  
Repeated Measures ◽  
Degrees Of Freedom ◽  
Type I Error ◽  
Covariance Matrices ◽  
Repeated Measurements ◽  
Skewed Distribution ◽  
Type I ◽  
Robust Test ◽  
Balanced Designs

For balanced designs, degrees of freedom-adjusted univariate F tests or multivariate test statistics can be used to obtain a robust test of repeated measures main and interaction effect hypotheses even when the assumption of equality of the covariance matrices is not satisfied. For unbalanced designs, however, covariance heterogeneity can seriously distort the rates of Type I error of either of these approaches. This article shows how a multivariate approximate degrees of freedom procedure based on Welch (1947 , 1951 )- James (1951 , 1954) , as simplified by Johansen (1980) , can be applied to the analysis of unbalanced repeated measures designs without assuming covariance homogeneity. Through Monte Carlo methods, we demonstrate that this approach provides a robust test of the repeated measures main effect hypothesis even when the data are obtained from a skewed distribution. The Welch-James approach also provides a robust test of the interaction effect, provided that the smallest of the unequal group sizes is five to six times the number of repeated measurements minus one or provided that a reduced level of significance is employed.

scholarly journals Violating the normality assumption may be the lesser of two evils

2021 ◽  
Author(s):  
Ulrich Knief ◽  
Wolfgang Forstmeier
Keyword(s):  
Mixed Models ◽  
Type I Error ◽  
Parameter Estimates ◽  
Type I ◽  
Gaussian Models ◽  
Normality Assumption ◽  
Wide Range ◽  
Pros And Cons ◽  
Non Gaussian ◽  
Specific Error

AbstractWhen data are not normally distributed, researchers are often uncertain whether it is legitimate to use tests that assume Gaussian errors, or whether one has to either model a more specific error structure or use randomization techniques. Here we use Monte Carlo simulations to explore the pros and cons of fitting Gaussian models to non-normal data in terms of risk of type I error, power and utility for parameter estimation. We find that Gaussian models are robust to non-normality over a wide range of conditions, meaning that p values remain fairly reliable except for data with influential outliers judged at strict alpha levels. Gaussian models also performed well in terms of power across all simulated scenarios. Parameter estimates were mostly unbiased and precise except if sample sizes were small or the distribution of the predictor was highly skewed. Transformation of data before analysis is often advisable and visual inspection for outliers and heteroscedasticity is important for assessment. In strong contrast, some non-Gaussian models and randomization techniques bear a range of risks that are often insufficiently known. High rates of false-positive conclusions can arise for instance when overdispersion in count data is not controlled appropriately or when randomization procedures ignore existing non-independencies in the data. Hence, newly developed statistical methods not only bring new opportunities, but they can also pose new threats to reliability. We argue that violating the normality assumption bears risks that are limited and manageable, while several more sophisticated approaches are relatively error prone and particularly difficult to check during peer review. Scientists and reviewers who are not fully aware of the risks might benefit from preferentially trusting Gaussian mixed models in which random effects account for non-independencies in the data.

scholarly journals Inference for One-Way ANOVA with Equicorrelation Error Structure

2014 ◽  
Vol 2014 ◽  
pp. 1-6 ◽  
Author(s):  
Weiyan Mu ◽  
Xiaojing Wang
Keyword(s):  
Type I Error ◽  
Bootstrap Method ◽  
Parametric Bootstrap ◽  
Type I ◽  
Anova Model ◽  
Coverage Probabilities ◽  
Empirical Performance ◽  
Simulation Results ◽  
Generalized Pivotal Quantities ◽  
One Way Anova

We consider inferences in a one-way ANOVA model with equicorrelation error structures. Hypotheses of the equality of the means are discussed. A generalizedF-test has been proposed by in the literature to compare the means of all populations. However, they did not discuss the performance of that test. We propose two methods, a generalized pivotal quantities-based method and a parametric bootstrap method, to test the hypotheses of equality of the means. We compare the empirical performance of the proposed tests with the generalizedF-test. It can be seen from the simulation results that the generalizedF-test does not perform well in terms of Type I error rate, and the proposed tests perform much better. We also provide corresponding simultaneous confidence intervals for all pair-wise differences of the means, whose coverage probabilities are close to the confidence level.

Smoothing Facilitates the Detection of Coupled Responses in Psychophysiological Time Series

2000 ◽  
Vol 14 (1) ◽  
pp. 1-10 ◽  
Author(s):  
Joni Kettunen ◽  
Niklas Ravaja ◽  
Liisa Keltikangas-Järvinen
Keyword(s):  
Time Series ◽  
Type I Error ◽  
Electrodermal Activity ◽  
Moving Average ◽  
Type I ◽  
Facial Electromyography ◽  
Response Characteristics ◽  
Smoothing Methods ◽  
Response Systems ◽  
Different Response

Abstract We examined the use of smoothing to enhance the detection of response coupling from the activity of different response systems. Three different types of moving average smoothers were applied to both simulated interbeat interval (IBI) and electrodermal activity (EDA) time series and to empirical IBI, EDA, and facial electromyography time series. The results indicated that progressive smoothing increased the efficiency of the detection of response coupling but did not increase the probability of Type I error. The power of the smoothing methods depended on the response characteristics. The benefits and use of the smoothing methods to extract information from psychophysiological time series are discussed.

One Size Fits All?

Methodology ◽  
2012 ◽  
Vol 8 (1) ◽  
pp. 23-38 ◽  
Author(s):  
Manuel C. Voelkle ◽  
Patrick E. McKnight
Keyword(s):  
Repeated Measures ◽  
Structural Equation ◽  
Type I Error ◽  
Equation Modeling ◽  
Type I ◽  
Finite Sample ◽  
Traditional Methods ◽  
Repeated Measures Anova ◽  
Special Cases ◽  
Latent Curve

The use of latent curve models (LCMs) has increased almost exponentially during the last decade. Oftentimes, researchers regard LCM as a “new” method to analyze change with little attention paid to the fact that the technique was originally introduced as an “alternative to standard repeated measures ANOVA and first-order auto-regressive methods” (Meredith & Tisak, 1990, p. 107). In the first part of the paper, this close relationship is reviewed, and it is demonstrated how “traditional” methods, such as the repeated measures ANOVA, and MANOVA, can be formulated as LCMs. Given that latent curve modeling is essentially a large-sample technique, compared to “traditional” finite-sample approaches, the second part of the paper addresses the question to what degree the more flexible LCMs can actually replace some of the older tests by means of a Monte-Carlo simulation. In addition, a structural equation modeling alternative to Mauchly’s (1940) test of sphericity is explored. Although “traditional” methods may be expressed as special cases of more general LCMs, we found the equivalence holds only asymptotically. For practical purposes, however, no approach always outperformed the other alternatives in terms of power and type I error, so the best method to be used depends on the situation. We provide detailed recommendations of when to use which method.

Assessing Person Fit With the Information Matrix Test

Methodology ◽  
2015 ◽  
Vol 11 (1) ◽  
pp. 3-12 ◽  
Author(s):  
Jochen Ranger ◽  
Jörg-Tobias Kuhn
Keyword(s):  
Simulation Study ◽  
Type I Error ◽  
Information Matrix ◽  
Small Samples ◽  
Type I ◽  
Person Fit ◽  
Power Of The Test ◽  
Order Expansion ◽  
Trait Stability ◽  
Information Matrix Test

In this manuscript, a new approach to the analysis of person fit is presented that is based on the information matrix test of White (1982) . This test can be interpreted as a test of trait stability during the measurement situation. The test follows approximately a χ2-distribution. In small samples, the approximation can be improved by a higher-order expansion. The performance of the test is explored in a simulation study. This simulation study suggests that the test adheres to the nominal Type-I error rate well, although it tends to be conservative in very short scales. The power of the test is compared to the power of four alternative tests of person fit. This comparison corroborates that the power of the information matrix test is similar to the power of the alternative tests. Advantages and areas of application of the information matrix test are discussed.

How to Detect Publication Bias in Psychological Research

2019 ◽  
Vol 227 (4) ◽  
pp. 261-279 ◽  
Author(s):  
Frank Renkewitz ◽  
Melanie Keiner
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Type I Error ◽  
Psychological Research ◽  
Type I ◽  
True Effect Size ◽  
Questionable Research Practices ◽  
True Effect ◽  
Meta Analyses

Abstract. Publication biases and questionable research practices are assumed to be two of the main causes of low replication rates. Both of these problems lead to severely inflated effect size estimates in meta-analyses. Methodologists have proposed a number of statistical tools to detect such bias in meta-analytic results. We present an evaluation of the performance of six of these tools. To assess the Type I error rate and the statistical power of these methods, we simulated a large variety of literatures that differed with regard to true effect size, heterogeneity, number of available primary studies, and sample sizes of these primary studies; furthermore, simulated studies were subjected to different degrees of publication bias. Our results show that across all simulated conditions, no method consistently outperformed the others. Additionally, all methods performed poorly when true effect sizes were heterogeneous or primary studies had a small chance of being published, irrespective of their results. This suggests that in many actual meta-analyses in psychology, bias will remain undiscovered no matter which detection method is used.

Sign in / Sign up

Export Citation Format

Share Document