A Poisson Method of Controlling the Maximum Tolerable Number of Type I Errors

1978 ◽  
Vol 46 (1) ◽  
pp. 211-218
Author(s):  
Louis M. Hsu
Keyword(s):  
Type I Error ◽  
Statistical Tests ◽  
Poisson Approximation ◽  
Type I ◽  
Type I Errors ◽  
Significance Level ◽  
Orthogonal Contrasts ◽  
Multiple Tests ◽  
Significance Levels ◽  
Tests Of Hypotheses

The problem of controlling the risk of occurrence of at least one Type I Error in a family of n statistical tests has been discussed extensively in psychological literature. However, the more general problem of controlling the probability of occurrence of more than some maximum (not necessarily zero) tolerable number ( xm) of Type I Errors in such a family appears to have received little attention. The present paper presents a simple Poisson approximation to the significance level P( EI) which should be used per test, to achieve this goal, in a family of n independent tests. The cases of equal and unequal significance levels for the n tests are discussed. Relative merits and limitations of the Poisson and Bonferroni methods of controlling the number of Type I Errors are examined, and application of the Poisson method to tests of orthogonal contrasts in analysis of variance, multiple tests of hypotheses in single studies, and multiple tests of hypotheses in literature reviews, are discussed.

Multiplicity adjustments in trials with two correlated comparisons of interest

2010 ◽  
Vol 20 (6) ◽  
pp. 579-594 ◽  
Author(s):  
Nikki Fernandes ◽  
Andrew Stone
Keyword(s):  
Clinical Trials ◽  
Type I Error ◽  
Experimental Treatment ◽  
Type I ◽  
Significance Level ◽  
Practical Guide ◽  
Significance Levels ◽  
Graphical Presentation

Clinical trials investigating the efficacy of two or more doses of an experimental treatment compared to a single reference arm are not uncommon. In such situations, if each dose is compared to the reference arm using an un-adjusted significance level, consideration of the Type I familywise error is likely to be required. Furthermore, in trials where two or more comparisons are performed using the same reference arm, the comparisons are inherently correlated. The correlation between comparisons can be utilised to remove some of the conservativeness of some commonly used procedures. This article is intended as a practical guide that should enable calculation of significance levels that fully conserve Type I error and provides graphical presentation that could facilitate their description to non-statisticians.

On Use of Multiple Tests of Significance in Psychological Research

1974 ◽  
Vol 35 (1) ◽  
pp. 427-431 ◽  
Author(s):  
Hubert S. Feild ◽  
Achilles A. Armenakis
Keyword(s):  
Null Hypothesis ◽  
Type I Error ◽  
Statistical Tests ◽  
Psychological Research ◽  
Type I ◽  
Multiple Tests ◽  
Tests Of Significance

The evaluation of a series of statistical tests in psychological research is a common problem faced by many investigators. As the number of statistical tests increases, the probability of making a Type I error, i.e., of rejecting the null hypothesis when in fact it is true, increases as well. To help researchers evaluate their results, tables have been constructed which show the probability of obtaining k or more significant results due to chance in a series of K independent statistical tests. Recommendations are also given in order to avoid the problems of a Type I error.

scholarly journals Controlling the rate of Type I error over a large set of statistical tests

2002 ◽  
Vol 55 (1) ◽  
pp. 27-39 ◽  
Author(s):  
H.J. Keselman ◽  
Robert Cribbie ◽  
Burt Holland
Keyword(s):  
Type I Error ◽  
Statistical Tests ◽  
Type I ◽  
Large Set

scholarly journals An Evaluation of Four Solutions to the Forking Paths Problem: Adjusted Alpha, Preregistration, Sensitivity Analyses, and Abandoning the Neyman-Pearson Approach

2017 ◽  
Vol 21 (4) ◽  
pp. 321-329 ◽  
Author(s):  
Mark Rubin
Keyword(s):  
Hypothesis Testing ◽  
Present Article ◽  
Type I Error ◽  
Statistical Analyses ◽  
Nonlinear Transformation ◽  
Sensitivity Analyses ◽  
Type I ◽  
Alternative Analysis ◽  
Multiple Tests ◽  
Alpha Level

Gelman and Loken (2013 , 2014 ) proposed that when researchers base their statistical analyses on the idiosyncratic characteristics of a specific sample (e.g., a nonlinear transformation of a variable because it is skewed), they open up alternative analysis paths in potential replications of their study that are based on different samples (i.e., no transformation of the variable because it is not skewed). These alternative analysis paths count as additional (multiple) tests and, consequently, they increase the probability of making a Type I error during hypothesis testing. The present article considers this forking paths problem and evaluates four potential solutions that might be used in psychology and other fields: (a) adjusting the prespecified alpha level, (b) preregistration, (c) sensitivity analyses, and (d) abandoning the Neyman-Pearson approach. It is concluded that although preregistration and sensitivity analyses are effective solutions to p-hacking, they are ineffective against result-neutral forking paths, such as those caused by transforming data. Conversely, although adjusting the alpha level cannot address p-hacking, it can be effective for result-neutral forking paths. Finally, abandoning the Neyman-Pearson approach represents a further solution to the forking paths problem.

scholarly journals Efficient model-based bioequivalence testing

Biostatistics ◽  
2020 ◽  
Author(s):  
Kathrin Möllenhoff ◽  
Florence Loingeville ◽  
Julie Bertrand ◽  
Thu Thuy Nguyen ◽  
Satish Sharan ◽  
...  
Keyword(s):  
Geometric Mean ◽  
Area Under The Curve ◽  
European Medicines Agency ◽  
Type I ◽  
Nonlinear Mixed Effects Models ◽  
Type I Errors ◽  
Significance Level ◽  
Treatment Groups ◽  
Model Based ◽  
Bioequivalence Testing

Summary The classical approach to analyze pharmacokinetic (PK) data in bioequivalence studies aiming to compare two different formulations is to perform noncompartmental analysis (NCA) followed by two one-sided tests (TOST). In this regard, the PK parameters area under the curve (AUC) and $C_{\max}$ are obtained for both treatment groups and their geometric mean ratios are considered. According to current guidelines by the U.S. Food and Drug Administration and the European Medicines Agency, the formulations are declared to be sufficiently similar if the $90\%$ confidence interval for these ratios falls between $0.8$ and $1.25 $. As NCA is not a reliable approach in case of sparse designs, a model-based alternative has already been proposed for the estimation of $\rm AUC$ and $C_{\max}$ using nonlinear mixed effects models. Here we propose another, more powerful test than the TOST and demonstrate its superiority through a simulation study both for NCA and model-based approaches. For products with high variability on PK parameters, this method appears to have closer type I errors to the conventionally accepted significance level of $0.05$, suggesting its potential use in situations where conventional bioequivalence analysis is not applicable.

scholarly journals Statistical Tests for the Reciprocal of a Normal Mean with a Known Coefficient of Variation

2015 ◽  
Vol 2015 ◽  
pp. 1-5
Author(s):  
Wararit Panichkitkosolkul
Keyword(s):  
Monte Carlo Simulation ◽  
Coefficient Of Variation ◽  
Statistical Tests ◽  
Taylor Series Expansion ◽  
Type I ◽  
Type I Errors ◽  
Monte Carlo Simulation Study ◽  
Asymptotic Test ◽  
Normal Mean ◽  
Simulation Results

An asymptotic test and an approximate test for the reciprocal of a normal mean with a known coefficient of variation were proposed in this paper. The asymptotic test was based on the expectation and variance of the estimator of the reciprocal of a normal mean. The approximate test used the approximate expectation and variance of the estimator by Taylor series expansion. A Monte Carlo simulation study was conducted to compare the performance of the two statistical tests. Simulation results showed that the two proposed tests performed well in terms of empirical type I errors and power. Nevertheless, the approximate test was easier to compute than the asymptotic test.

Repeated Measures Multiple Comparison Procedures: Effects of Violating Multisample Sphericity in Unbalanced Designs

1988 ◽  
Vol 13 (3) ◽  
pp. 215-226 ◽  
Author(s):  
H. J. Keselman ◽  
Joanne C. Keselman
Keyword(s):  
Repeated Measures ◽  
Type I Error ◽  
Multiple Comparison ◽  
Type I ◽  
Type I Errors ◽  
Unbalanced Designs ◽  
Bonferroni Procedure ◽  
Weighted Means ◽  
Multiple Comparison Procedures ◽  
Multisample Sphericity

Two Tukey multiple comparison procedures as well as a Bonferroni and multivariate approach were compared for their rates of Type I error and any-pairs power when multisample sphericity was not satisfied and the design was unbalanced. Pairwise comparisons of unweighted and weighted repeated measures means were computed. Results indicated that heterogenous covariance matrices in combination with unequal group sizes resulted in substantially inflated rates of Type I error for all MCPs involving comparisons of unweighted means. For tests of weighted means, both the Bonferroni and a multivariate critical value limited the number of Type I errors; however, the Bonferroni procedure provided a more powerful test, particularly when the number of repeated measures treatment levels was large.

scholarly journals STATISTIC TESTS AIDED MULTI-SOURCE DEM FUSION

2016 ◽  
Vol XLI-B6 ◽  
pp. 227-233
Author(s):  
C. Y. Fu ◽  
J. R. Tsay
Keyword(s):  
Land Surface ◽  
Type I Error ◽  
Low Cost ◽  
Type I ◽  
Fusion Algorithm ◽  
Type I Errors ◽  
Production Methods ◽  
Grid Points ◽  
The Cost ◽  
Proper Setting

Since the land surface has been changing naturally or manually, DEMs have to be updated continually to satisfy applications using the latest DEM at present. However, the cost of wide-area DEM production is too high. DEMs, which cover the same area but have different quality, grid sizes, generation time or production methods, are called as multi-source DEMs. It provides a solution to fuse multi-source DEMs for low cost DEM updating. The coverage of DEM has to be classified according to slope and visibility in advance, because the precisions of DEM grid points in different areas with different slopes and visibilities are not the same. Next, difference DEM (dDEM) is computed by subtracting two DEMs. It is assumed that dDEM, which only contains random error, obeys normal distribution. Therefore, student test is implemented for blunder detection and three kinds of rejected grid points are generated. First kind of rejected grid points is blunder points and has to be eliminated. Another one is the ones in change areas, where the latest data are regarded as their fusion result. Moreover, the DEM grid points of type I error are correct data and have to be reserved for fusion. The experiment result shows that using DEMs with terrain classification can obtain better blunder detection result. A proper setting of significant levels (α) can detect real blunders without creating too many type I errors. Weighting averaging is chosen as DEM fusion algorithm. The priori precisions estimated by our national DEM production guideline are applied to define weights. Fisher’s test is implemented to prove that the priori precisions correspond to the RMSEs of blunder detection result.

scholarly journals THE MISUSE OF SYNTAX ALTERATION FOR GEE AND ANOVA POST-HOC ANALYSES IN SPSS

2020 ◽  
Vol 7 (1) ◽  
pp. 1-6
Author(s):  
João Pedro Nunes ◽  
Giovanna F. Frigoli
Keyword(s):  
Repeated Measures ◽  
Type I Error ◽  
Statistical Tests ◽  
Data Interpretation ◽  
Online Support ◽  
Type I ◽  
K Value ◽  
Repeated Measures Anova ◽  
Post Hoc ◽  
Statistical Results

The online support of IBM SPSS proposes that users alter the syntax when performing post-hoc analyses for interaction effects of ANOVA tests. Other authors also suggest altering the syntax when performing GEE analyses. This being done, the number of possible comparisons (k value) is also altered, therefore influencing the results from statistical tests that k is a component of the formula, such as repeated measures-ANOVA and Bonferroni post-hoc of ANOVA and GEE. This alteration also exacerbates type I error, producing erroneous results and conferring potential misinterpretations of data. Reasoning from this, the purpose of this paper is to report the misuse and improper handling of syntax for ANOVAs and GEE post-hoc analyses in SPSS and to illustrate its consequences on statistical results and data interpretation.

Sign in / Sign up

Export Citation Format

Share Document