scholarly journals A Monte Carlo-Based Outlier Diagnosis Method for Sensitivity Analysis

2020 ◽  
Vol 12 (5) ◽  
pp. 860 ◽  
Author(s):  
Vinicius Francisco Rofatto ◽  
Marcelo Tomio Matsuoka ◽  
Ivandro Klein ◽  
Maurício Roberto Veronez ◽  
Luiz Gonzaga da Silveira
Keyword(s):  
Monte Carlo ◽  
Outlier Detection ◽  
Type I Error ◽  
Critical Value ◽  
Correct Identification ◽  
Type I ◽  
Test Statistic ◽  
Decision Error ◽  
Detection And Identification ◽  
First Results

An iterative outlier elimination procedure based on hypothesis testing, commonly known as Iterative Data Snooping (IDS) among geodesists, is often used for the quality control of modern measurement systems in geodesy and surveying. The test statistic associated with IDS is the extreme normalised least-squares residual. It is well-known in the literature that critical values (quantile values) of such a test statistic cannot be derived from well-known test distributions but must be computed numerically by means of Monte Carlo. This paper provides the first results on the Monte Carlo-based critical value inserted into different scenarios of correlation between outlier statistics. From the Monte Carlo evaluation, we compute the probabilities of correct identification, missed detection, wrong exclusion, over-identifications and statistical overlap associated with IDS in the presence of a single outlier. On the basis of such probability levels, we obtain the Minimal Detectable Bias (MDB) and Minimal Identifiable Bias (MIB) for cases in which IDS is in play. The MDB and MIB are sensitivity indicators for outlier detection and identification, respectively. The results show that there are circumstances in which the larger the Type I decision error (smaller critical value), the higher the rates of outlier detection but the lower the rates of outlier identification. In such a case, the larger the Type I Error, the larger the ratio between the MIB and MDB. We also highlight that an outlier becomes identifiable when the contributions of the measures to the wrong exclusion rate decline simultaneously. In this case, we verify that the effect of the correlation between outlier statistics on the wrong exclusion rate becomes insignificant for a certain outlier magnitude, which increases the probability of identification.

scholarly journals A Monte Carlo-Based Outlier Diagnosis Method for Sensitivity Analysis

2020 ◽  
Author(s):  
Vinicius Francisco Rofatto ◽  
Marcelo Tomio Matsuoka ◽  
Ivandro Klein ◽  
Mauricio Roberto Veronez ◽  
Luiz Gonzaga da Silveira Jr.
Keyword(s):  
Monte Carlo ◽  
Outlier Detection ◽  
Type I Error ◽  
Critical Value ◽  
Correct Identification ◽  
Type I ◽  
Test Statistic ◽  
Decision Error ◽  
Detection And Identification ◽  
First Results

An iterative outlier elimination procedure based on hypothesis testing, commonly known as Iterative Data Snooping (IDS) among geodesists, is often used for the quality control of the modern measurement systems in geodesy and surveying. The test statistic associated with IDS is the extreme normalised least-squares residual. It is well-known in the literature that critical values (quantile values) of such a test statistic cannot be derived from well-known test distributions, but must be computed numerically by means of Monte Carlo. This paper provides the first results about Monte Carlo-based critical value inserted to different scenarios of correlation between the outlier statistics. From the Monte Carlo evaluation, we compute the probabilities of correct identification, missed detection, wrong exclusion, overidentifications and statistical overlap associated with IDS in the presence of a single outlier. Based on such probability levels we obtain the Minimal Detectable Bias (MDB) and Minimal Identifiable Bias (MIB) for the case where IDS is in play. MDB and MIB are sensitivity indicators for outlier detection and identification, respectively. The results show that there are circumstances that the larger the Type I decision error (smaller critical value), the higher the rates of outlier detection, but the lower the rates of outlier identification. For that case, the larger the Type I Error, the larger the ratio between MIB and MDB. We also highlight that an outlier becomes identifiable when the contribution of the measures to the wrong exclusion rate decline simultaneously. In that case, we verify that the effect of the correlation between the outlier statistics on the wrong exclusion rates becomes insignificant from a certain outlier magnitude, which increases the probability of identification.

Parametric Alternatives to the Analysis of Variance

1982 ◽  
Vol 7 (3) ◽  
pp. 207-214 ◽  
Author(s):  
Jennifer J. Clinch ◽  
H. J. Keselman
Keyword(s):  
Monte Carlo ◽  
Analysis Of Variance ◽  
Monte Carlo Methods ◽  
Type I Error ◽  
Type I ◽  
Test Statistic ◽  
F Test ◽  
Assumption Violations ◽  
Mean Equality

The ANOVA, Welch, and Brown and Forsyth tests for mean equality were compared using Monte Carlo methods. The tests’ rates of Type I error and power were examined when populations were non-normal, variances were heterogeneous, and group sizes were unequal. The ANOVA F test was most affected by the assumption violations. The test proposed by Brown and Forsyth appeared, on the average, to be the “best” test statistic for testing an omnibus hypothesis of mean equality.

scholarly journals More on the Supremum Statistic to Test Multivariate Skew-Normality

Computation ◽  
2021 ◽  
Vol 9 (12) ◽  
pp. 126
Author(s):  
Timothy Opheim ◽  
Anuradha Roy
Keyword(s):  
Monte Carlo Simulation ◽  
Monte Carlo ◽  
Type I Error ◽  
Relevant Information ◽  
Type I ◽  
Simulation Studies ◽  
Test Statistic ◽  
Nominal Significance ◽  
Significance Levels ◽  
Empirical Size

This review is about verifying and generalizing the supremum test statistic developed by Balakrishnan et al. Exhaustive simulation studies are conducted for various dimensions to determine the effect, in terms of empirical size, of the supremum test statistic developed by Balakrishnan et al. to test multivariate skew-normality. Monte Carlo simulation studies indicate that the Type-I error of the supremum test can be controlled reasonably well for various dimensions for given nominal significance levels 0.05 and 0.01. Cut-off values are provided for the number of samples required to attain the nominal significance levels 0.05 and 0.01. Some new and relevant information of the supremum test statistic are reported here.

scholarly journals Testing for treatment effect in covariate-adaptive randomized trials with generalized linear models and omitted covariates

2021 ◽  
pp. 096228022110082
Author(s):  
Yang Li ◽  
Wei Ma ◽  
Yichen Qin ◽  
Feifang Hu
Keyword(s):  
Generalized Linear Models ◽  
Treatment Effect ◽  
Linear Models ◽  
Type I Error ◽  
Type I ◽  
Test Statistic ◽  
Asymptotic Results ◽  
Adaptive Randomization ◽  
Adjustment Method ◽  
Inflated Type

Concerns have been expressed over the validity of statistical inference under covariate-adaptive randomization despite the extensive use in clinical trials. In the literature, the inferential properties under covariate-adaptive randomization have been mainly studied for continuous responses; in particular, it is well known that the usual two-sample t-test for treatment effect is typically conservative. This phenomenon of invalid tests has also been found for generalized linear models without adjusting for the covariates and are sometimes more worrisome due to inflated Type I error. The purpose of this study is to examine the unadjusted test for treatment effect under generalized linear models and covariate-adaptive randomization. For a large class of covariate-adaptive randomization methods, we obtain the asymptotic distribution of the test statistic under the null hypothesis and derive the conditions under which the test is conservative, valid, or anti-conservative. Several commonly used generalized linear models, such as logistic regression and Poisson regression, are discussed in detail. An adjustment method is also proposed to achieve a valid size based on the asymptotic results. Numerical studies confirm the theoretical findings and demonstrate the effectiveness of the proposed adjustment method.

scholarly journals REMAXINT: a two-mode clustering-based method for statistical inference on two-way interaction

2021 ◽  
Author(s):  
Zaheer Ahmed ◽  
Alberto Cassese ◽  
Gerard van Breukelen ◽  
Jan Schepers
Keyword(s):  
Null Hypothesis ◽  
Type I Error ◽  
Type I ◽  
The Novel ◽  
Simulation Studies ◽  
Test Statistic ◽  
Clustering Model ◽  
Novel Method ◽  
Main Effects ◽  
Empirical Case Studies

AbstractWe present a novel method, REMAXINT, that captures the gist of two-way interaction in row by column (i.e., two-mode) data, with one observation per cell. REMAXINT is a probabilistic two-mode clustering model that yields two-mode partitions with maximal interaction between row and column clusters. For estimation of the parameters of REMAXINT, we maximize a conditional classification likelihood in which the random row (or column) main effects are conditioned out. For testing the null hypothesis of no interaction between row and column clusters, we propose a $$max-F$$ m a x - F test statistic and discuss its properties. We develop a Monte Carlo approach to obtain its sampling distribution under the null hypothesis. We evaluate the performance of the method through simulation studies. Specifically, for selected values of data size and (true) numbers of clusters, we obtain critical values of the $$max-F$$ m a x - F statistic, determine empirical Type I error rate of the proposed inferential procedure and study its power to reject the null hypothesis. Next, we show that the novel method is useful in a variety of applications by presenting two empirical case studies and end with some concluding remarks.

Comparison of Means: An F Test Followed by a Modified Multiple Range Procedure

1979 ◽  
Vol 4 (1) ◽  
pp. 14-23 ◽  
Author(s):  
Juliet Popper Shaffer
Keyword(s):  
Error Control ◽  
Type I Error ◽  
Critical Value ◽  
The Other ◽  
Type I ◽  
F Test ◽  
Range Test

If used only when a preliminary F test yields significance, the usual multiple range procedures can be modified to increase the probability of detecting differences without changing the control of Type I error. The modification consists of a reduction in the critical value when comparing the largest and smallest means. Equivalence of modified and unmodified procedures in error control is demonstrated. The modified procedure is also compared with the alternative of using the unmodified range test without a preliminary F test, and it is shown that each has advantages over the other under some circumstances.

Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA Cases

1992 ◽  
Vol 17 (4) ◽  
pp. 315-339 ◽  
Author(s):  
Michael R. Harwell ◽  
Elaine N. Rubinstein ◽  
William S. Hayes ◽  
Corley C. Olds
Keyword(s):  
Monte Carlo ◽  
Error Rate ◽  
Fixed Effects ◽  
Type I Error ◽  
Type I ◽  
F Test ◽  
Unequal Variances ◽  
Type I Error Rate ◽  
Monte Carlo Studies ◽  
The One

Meta-analytic methods were used to integrate the findings of a sample of Monte Carlo studies of the robustness of the F test in the one- and two-factor fixed effects ANOVA models. Monte Carlo results for the Welch (1947) and Kruskal-Wallis (Kruskal & Wallis, 1952) tests were also analyzed. The meta-analytic results provided strong support for the robustness of the Type I error rate of the F test when certain assumptions were violated. The F test also showed excellent power properties. However, the Type I error rate of the F test was sensitive to unequal variances, even when sample sizes were equal. The error rate of the Welch test was insensitive to unequal variances when the population distribution was normal, but nonnormal distributions tended to inflate its error rate and to depress its power. Meta-analytic and exact statistical theory results were used to summarize the effects of assumption violations for the tests.

scholarly journals A novel gene-set association test based on variance-gamma distribution

2018 ◽  
Vol 28 (9) ◽  
pp. 2868-2875
Author(s):  
Zhongxue Chen ◽  
Qingzhong Liu ◽  
Kai Wang
Keyword(s):  
Gamma Distribution ◽  
Type I Error ◽  
Null Distribution ◽  
Real Data ◽  
Association Test ◽  
P Value ◽  
Type I ◽  
Test Statistic ◽  
Data Set ◽  
Variance Gamma

Several gene- or set-based association tests have been proposed recently in the literature. Powerful statistical approaches are still highly desirable in this area. In this paper we propose a novel statistical association test, which uses information of the burden component and its complement from the genotypes. This new test statistic has a simple null distribution, which is a special and simplified variance-gamma distribution, and its p-value can be easily calculated. Through a comprehensive simulation study, we show that the new test can control type I error rate and has superior detecting power compared with some popular existing methods. We also apply the new approach to a real data set; the results demonstrate that this test is promising.

Comparison of methods to account for autocorrelation in correlation analyses of fish data

1998 ◽  
Vol 55 (9) ◽  
pp. 2127-2140 ◽  
Author(s):  
Brian J Pyper ◽  
Randall M Peterman
Keyword(s):  
Monte Carlo ◽  
Hypothesis Testing ◽  
Type I Error ◽  
Low Frequency ◽  
Error Rates ◽  
Type I ◽  
Testing Procedures ◽  
Type I Error Rates ◽  
Fish Recruitment ◽  
Correlation Analyses

Autocorrelation in fish recruitment and environmental data can complicate statistical inference in correlation analyses. To address this problem, researchers often either adjust hypothesis testing procedures (e.g., adjust degrees of freedom) to account for autocorrelation or remove the autocorrelation using prewhitening or first-differencing before analysis. However, the effectiveness of methods that adjust hypothesis testing procedures has not yet been fully explored quantitatively. We therefore compared several adjustment methods via Monte Carlo simulation and found that a modified version of these methods kept Type I error rates near . In contrast, methods that remove autocorrelation control Type I error rates well but may in some circumstances increase Type II error rates (probability of failing to detect some environmental effect) and hence reduce statistical power, in comparison with adjusting the test procedure. Specifically, our Monte Carlo simulations show that prewhitening and especially first-differencing decrease power in the common situations where low-frequency (slowly changing) processes are important sources of covariation in fish recruitment or in environmental variables. Conversely, removing autocorrelation can increase power when low-frequency processes account for only some of the covariation. We therefore recommend that researchers carefully consider the importance of different time scales of variability when analyzing autocorrelated data.

Sign in / Sign up

Export Citation Format

Share Document