Non-inferiority test for a continuous variable with a flexible margin in an active controlled trial : An application to the "Stratall ANRS 12110 / ESTHER" trial

Abstract Background: The non-inferiority trials are becoming increasingly popular in public health and clinical research. The choice of the non-inferiority margin is the cornerstone of the non-inferiority trial. When the effect of active control intervention is unknown, it can be interesting to choose the non-inferiority margin as a function of the active control intervention effect. In this case, the uncertainty surrounding non-inferiority margin should be taken into account in statistical tests.Methods: It was proposed in this study two procedures for the non-inferiority test with a flexible margin for the continuous endpoint. The proposed test procedures are based on the asymptotic test and confidence interval approach. Simulations have been used to assess the performances and properties of the proposed test procedures. An application was done on clinical real data. The purpose of this study was to assess the effectiveness and safety of clinical monitoring alone versus laboratory and clinical monitoring in HIV infected adults patients.Results: The two proposed test procedures have good properties for large sample sizes. It has been found that the confidence interval level determines approximately the level of significance. The The 80%, 90% and 95% two-sided confidence interval levels led approximately to a type I error of 5%, 2:5% and 1%respectively. For small sample sizes, the test with 80% confidence interval level was the most powerful.Conclusions: This study highlights the choice of non-inferiority margin as flexible and depending on the reference treatment in an active trial of non-inferiority. The results of the study show that the proposed methods are applicable in the practice.Trial registration : The trial data used in this study was from the "Stratall ANRS 12110 / ESTHER", registered with ClinicalTrials.gov, number NCT00301561. Date : March 13, 2006, url : https://clinicaltrials.gov/ct2/show/NCT00301561.

Download Full-text

Non-inferiority test for a continuous variable with a flexible margin in an active controlled trial : An application to the "Stratall ANRS 12110 / ESTHER" trial

10.21203/rs.3.rs-21325/v2 ◽

2020 ◽

Author(s):

Arsene Sandie ◽

Nicholas Molinari ◽

Anthony Wanjoya ◽

Charles Kouanfack ◽

Christian Laurent ◽

...

Keyword(s):

Confidence Interval ◽

Active Control ◽

Error Rate ◽

Type I Error ◽

Clinical Monitoring ◽

Small Scale ◽

Type I ◽

Test Procedures ◽

Power Estimate ◽

Control Intervention

Abstract Background: The non-inferiority trials are becoming increasingly popular in public health and clinical research. The choice of the non-inferiority margin is the cornerstone of the non-inferiority trial. When the eﬀect of active control intervention is unknown, it can be interesting to choose the non-inferiority margin as a function of the active control intervention eﬀect. In this case, the uncertainty surrounding the non-inferiority margin should be accounted for in statistical tests. In this work, we explored how to perform the non-inferiority test with a ﬂexible margin for continuous endpoint.Methods: It was proposed in this study two procedures for the non-inferiority test with a ﬂexible margin for the continuous endpoint. The proposed test procedures are based on test statistic and conﬁdence interval approach. Simulations have been used to assess the performances and properties of the proposed test procedures. An application was done on clinical real data, which the purpose was to assess the eﬃcacy of clinical monitoring alone versus laboratory and clinical monitoring in HIV-infected adult patients.Results: Basically, the two proposed test procedures have good properties. In the test based on a statistic, the actual type 1 error rate estimate is approximatively equal to the nominal value. It has been found that the conﬁdence interval level determines approximately the level of signiﬁcance. The 80%, 90%, and 95%one-sided conﬁdence interval levels led approximately to a type I error of 10%, 5% and 2.5% respectively. The power estimate was almost 100% for two proposed tests, except for the small scale values of the reference treatment where the power was relatively low when the sample sizes were small.Conclusions: Based on type I error rate and power estimates, the proposed non-inferiority hypothesis test procedures have good performance and are applicable in practice.Trial registration: The trial data used in this study was from the ”Stratall ANRS 12110 / ESTHER”, registered with ClinicalTrials.gov, number NCT00301561. Date : March 13, 2006, url : https://clinicaltrials.gov/ct2/show/NCT00301561.

Download Full-text

Non-Inferiority Test for a Continuous Variable with a Flexible Margin in an Active Controlled Trial : An Application to the "Stratall ANRS 12110 / ESTHER" Trial

10.21203/rs.3.rs-21325/v3 ◽

2021 ◽

Author(s):

Arsene Sandie ◽

Nicholas Molinari ◽

Anthony Wanjoya ◽

Charles Kouanfack ◽

Christian Laurent ◽

...

Keyword(s):

Confidence Interval ◽

Active Control ◽

Error Rate ◽

Type I Error ◽

Clinical Monitoring ◽

Small Scale ◽

Type I ◽

Test Procedures ◽

Power Estimate ◽

Control Intervention

Abstract Background : The non-inferiority trials are becoming increasingly popular in public health and clinical research. The choice of the non-inferiority margin is the cornerstone of the non-inferiority trial. When the effect of active control intervention is unknown, it can be interesting to choose the non-inferiority margin as a function of the active control intervention effect. In this case, the uncertainty surrounding the non-inferiority margin should be accounted for in statistical tests. In this work, we explored how to perform the non-inferiority test with a flexible margin for continuous endpoint.Methods: It was proposed in this study two procedures for the non-inferiority test with a flexible margin for the continuous endpoint. The proposed test procedures are based on test statistic and confidence interval approach. Simulations have been used to assess the performances and properties of the proposed test procedures. An application was done on clinical real data, which the purpose was to assess the efficacy of clinical monitoring alone versus laboratory and clinical monitoring in HIV-infected adult patients.Results : Basically, the two proposed test procedures have good properties. In the test based on a statistic, the actual type 1 error rate estimate is approximatively equal to the nominal value. It has been found that the confidence interval level determines approximately the level of significance. The $80\%$ , $90\%$ , and $95\%$ one-sided confidence interval levels led approximately to a type I error of $10\%$ , $5\%$ and $2.5\%$ respectively. The power estimate was almost $100\%$ for two proposed tests, except for the small scale values of the reference treatment where the power was relatively low when the sample sizes were small.Conclusions : Based on type I error rate and power estimates, the proposed non-inferiority hypothesis test procedures have good performance and are applicable in practice.Trial registration : The trial data used in this study was from the "Stratall ANRS 12110 / ESTHER", registered with ClinicalTrials.gov, number NCT00301561. Date : March 13, 2006, url : https://clinicaltrials.gov/ct2/show/NCT00301561.

Download Full-text

On the use of chi-square analyses in studies of resource utilization

Canadian Journal of Forest Research ◽

10.1139/x91-009 ◽

1991 ◽

Vol 21 (1) ◽

pp. 58-65 ◽

Cited By ~ 10

Author(s):

Dennis E. Jelinski

Keyword(s):

Resource Utilization ◽

Goodness Of Fit ◽

Small Sample ◽

Homogeneity Test ◽

Type I ◽

Sample Sizes ◽

Goodness Of Fit Test ◽

Chi Square ◽

Test Of Homogeneity ◽

Small Sample Sizes

Chi-square (χ2) tests are analytic procedures that are often used to test the hypothesis that animals use a particular food item or habitat in proportion to its availability. Unfortunately, several sources of error are common to the use of χ2 analysis in studies of resource utilization. Both the goodness-of-fit and homogeneity tests have been incorrectly used interchangeably when resource availabilities are estimated or known apriori. An empirical comparison of the two methods demonstrates that the χ2 test of homogeneity may generate results contrary to the χ2 goodness-of-fit test. Failure to recognize the conservative nature of the χ2 homogeneity test, when "expected" values are known apriori, may lead to erroneous conclusions owing to the increased possibility of committing a type II error. Conversely, proper use of the goodness-of-fit method is predicated on the availability of accurate maps of resource abundance, or on estimates of resource availability based on very large sample sizes. Where resource availabilities have been estimated from small sample sizes, the use of the χ2 goodness-of-fit test may lead to type I errors beyond the nominal level of α. Both tests require adherence to specific critical assumptions that often have been violated, and accordingly, these assumptions are reviewed here. Alternatives to the Pearson χ2 statistic are also discussed.

Download Full-text

Comparative evaluation of goodness of fit tests for normal distribution using simulation and empirical data

Biometrical Letters ◽

10.2478/bile-2020-0015 ◽

2020 ◽

Vol 57 (2) ◽

pp. 237-251

Author(s):

Achilleas Anastasiou ◽

Alex Karagrigoriou ◽

Anastasios Katsileros

Keyword(s):

Sample Size ◽

Normal Distribution ◽

Empirical Data ◽

Goodness Of Fit ◽

Small Sample Size ◽

Small Sample ◽

Type I ◽

Sample Sizes ◽

Omnibus Test ◽

Wide Range

SummaryThe normal distribution is considered to be one of the most important distributions, with numerous applications in various fields, including the field of agricultural sciences. The purpose of this study is to evaluate the most popular normality tests, comparing the performance in terms of the size (type I error) and the power against a large spectrum of distributions with simulations for various sample sizes and significance levels, as well as through empirical data from agricultural experiments. The simulation results show that the power of all normality tests is low for small sample size, but as the sample size increases, the power increases as well. Also, the results show that the Shapiro–Wilk test is powerful over a wide range of alternative distributions and sample sizes and especially in asymmetric distributions. Moreover the D’Agostino–Pearson Omnibus test is powerful for small sample sizes against symmetric alternative distributions, while the same is true for the Kurtosis test for moderate and large sample sizes.

Download Full-text

Performance of Monte Carlo Permutation and Approximate Tests for Multivariate Means Comparisons With Small Sample Sizes When Parametric Assumptions are Violated

Methodology ◽

10.1027/1614-2241.5.2.60 ◽

2009 ◽

Vol 5 (2) ◽

pp. 60-70 ◽

Cited By ~ 6

Author(s):

W. Holmes Finch ◽

Teresa Davenport

Keyword(s):

Monte Carlo ◽

Type I Error ◽

Permutation Tests ◽

Error Rates ◽

Covariance Matrices ◽

Small Sample ◽

Type I ◽

Permutation Testing ◽

Sample Sizes ◽

Type I Error Rates

Permutation testing has been suggested as an alternative to the standard F approximate tests used in multivariate analysis of variance (MANOVA). These approximate tests, such as Wilks’ Lambda and Pillai’s Trace, have been shown to perform poorly when assumptions of normally distributed dependent variables and homogeneity of group covariance matrices were violated. Because Monte Carlo permutation tests do not rely on distributional assumptions, they may be expected to work better than their approximate cousins when the data do not conform to the assumptions described above. The current simulation study compared the performance of four standard MANOVA test statistics with their Monte Carlo permutation-based counterparts under a variety of conditions with small samples, including conditions when the assumptions were met and when they were not. Results suggest that for sample sizes of 50 subjects, power is very low for all the statistics. In addition, Type I error rates for both the approximate F and Monte Carlo tests were inflated under the condition of nonnormal data and unequal covariance matrices. In general, the performance of the Monte Carlo permutation tests was slightly better in terms of Type I error rates and power when both assumptions of normality and homogeneous covariance matrices were not met. It should be noted that these simulations were based upon the case with three groups only, and as such results presented in this study can only be generalized to similar situations.

Download Full-text

How Low Can You Go?

Methodology ◽

10.1027/1614-2241/a000062 ◽

2014 ◽

Vol 10 (1) ◽

pp. 1-11 ◽

Cited By ~ 57

Author(s):

Bethany A. Bell ◽

Grant B. Morgan ◽

Jason A. Schoeneberger ◽

Jeffrey D. Kromrey ◽

John M. Ferron

Keyword(s):

Confidence Interval ◽

Sample Size ◽

Fixed Effects ◽

Statistical Power ◽

Multilevel Models ◽

Type I Error ◽

Model Complexity ◽

Type I ◽

Sample Sizes ◽

Level 1

Whereas general sample size guidelines have been suggested when estimating multilevel models, they are only generalizable to a relatively limited number of data conditions and model structures, both of which are not very feasible for the applied researcher. In an effort to expand our understanding of two-level multilevel models under less than ideal conditions, Monte Carlo methods, through SAS/IML, were used to examine model convergence rates, parameter point estimates (statistical bias), parameter interval estimates (confidence interval accuracy and precision), and both Type I error control and statistical power of tests associated with the fixed effects from linear two-level models estimated with PROC MIXED. These outcomes were analyzed as a function of: (a) level-1 sample size, (b) level-2 sample size, (c) intercept variance, (d) slope variance, (e) collinearity, and (f) model complexity. Bias was minimal across nearly all conditions simulated. The 95% confidence interval coverage and Type I error rate tended to be slightly conservative. The degree of statistical power was related to sample sizes and level of fixed effects; higher power was observed with larger sample sizes and level-1 fixed effects.

Download Full-text

A novel confidence interval for a single proportion in the presence of clustered binary outcome data

Statistical Methods in Medical Research ◽

10.1177/0962280218823231 ◽

2019 ◽

Vol 29 (1) ◽

pp. 111-121

Author(s):

Meghan I Short ◽

Howard J Cabral ◽

Janice M Weinberg ◽

Michael P LaValley ◽

Joseph M Massaro

Keyword(s):

Confidence Interval ◽

Clustered Data ◽

Epidemiological Studies ◽

Outcome Data ◽

Small Sample ◽

Interval Methods ◽

Sample Sizes ◽

Wide Range ◽

Small Sample Sizes ◽

Over Dispersion

Estimating the precision of a single proportion via a 100(1−α)% confidence interval in the presence of clustered data is an important statistical problem. It is necessary to account for possible over-dispersion, for instance, in animal-based teratology studies with within-litter correlation, epidemiological studies that involve clustered sampling, and clinical trial designs with multiple measurements per subject. Several asymptotic confidence interval methods have been developed, which have been found to have inadequate coverage of the true proportion for small-to-moderate sample sizes. In addition, many of the best-performing of these intervals have not been directly compared with regard to the operational characteristics of coverage probability and empirical length. This study uses Monte Carlo simulations to calculate coverage probabilities and empirical lengths of five existing confidence intervals for clustered data across various true correlations, true probabilities of interest, and sample sizes. In addition, we introduce a new score-based confidence interval method, which we find to have better coverage than existing intervals for small sample sizes under a wide range of scenarios.

Download Full-text

Statistical Inference for a Ratio of Dispersions Using Paired Samples

Journal of Educational and Behavioral Statistics ◽

10.3102/10769986028001021 ◽

2003 ◽

Vol 28 (1) ◽

pp. 21-30 ◽

Cited By ~ 7

Author(s):

Douglas G. Bonett ◽

Edith Seier

Keyword(s):

Confidence Interval ◽

Error Rates ◽

Small Sample ◽

Type I ◽

Type Ii Error ◽

Paired Data ◽

Nonnormal Distributions ◽

Paired Samples ◽

Robust Tests ◽

Small Sample Sizes

Wilcox (1990) examined the Type I and Type II error rates for several robust tests of H0: σ12/σ22 = 1 in paired-data designs and concluded that a satisfactory solution does not yet exist. A confidence interval for a ratio of correlated mean absolute deviations is derived and performs well in small sample sizes across realistically nonnormal distributions. When used to test a hypothesis, the proposed confidence interval is almost as powerful as the most powerful test examined by Wilcox.

Download Full-text

Differences of Type I error rates for ANOVA and Multilevel-Linear-Models using SAS and SPSS for repeated measures designs

Meta-Psychology ◽

10.15626/mp.2018.898 ◽

2019 ◽

Vol 3 ◽

Author(s):

Nicolas Haverkamp ◽

André Beauducel

Keyword(s):

Repeated Measures ◽

Linear Models ◽

Type I Error ◽

Error Rates ◽

Small Sample ◽

Small Samples ◽

Type I ◽

Sample Sizes ◽

Type I Error Rates ◽

Multilevel Linear Models

To derive recommendations on how to analyze longitudinal data, we examined Type I error rates of Multilevel Linear Models (MLM) and repeated measures Analysis of Variance (rANOVA) using SAS and SPSS. We performed a simulation with the following specifications: To explore the effects of high numbers of measurement occasions and small sample sizes on Type I error, measurement occasions of m = 9 and 12 were investigated as well as sample sizes of n = 15, 20, 25 and 30. Effects of non-sphericity in the population on Type I error were also inspected: 5,000 random samples were drawn from two populations containing neither a within-subject nor a between-group effect. They were analyzed including the most common options to correct rANOVA and MLM-results: The Huynh-Feldt-correction for rANOVA (rANOVA-HF) and the Kenward-Roger-correction for MLM (MLM-KR), which could help to correct progressive bias of MLM with an unstructured covariance matrix (MLM-UN). Moreover, uncorrected rANOVA and MLM assuming a compound symmetry covariance structure (MLM-CS) were also taken into account. The results showed a progressive bias for MLM-UN for small samples which was stronger in SPSS than in SAS. Moreover, an appropriate bias correction for Type I error via rANOVA-HF and an insufficient correction by MLM-UN-KR for n < 30 were found. These findings suggest MLM-CS or rANOVA if sphericity holds and a correction of a violation via rANOVA-HF. If an analysis requires MLM, SPSS yields more accurate Type I error rates for MLM-CS and SAS yields more accurate Type I error rates for MLM-UN.

Download Full-text

A Note on Inferences About the Probability of Success

Journal of Modern Applied Statistical Methods ◽

10.22237/jmasm/1556670420 ◽

2020 ◽

Vol 18 (1) ◽

Cited By ~ 1

Author(s):

Rand Wilcox

Keyword(s):

Confidence Interval ◽

Sample Size ◽

Small Sample ◽

Extensive Literature ◽

Sample Sizes ◽

Probability Of Success ◽

Small Sample Sizes ◽

A Minor ◽

Sample Case

There is an extensive literature dealing with inferences about the probability of success. A minor goal in this note is to point out when certain recommended methods can be unsatisfactory when the sample size is small. The main goal is to report results on the two-sample case. Extant results suggest using one of four methods. The results indicate when computing a 0.95 confidence interval, two of these methods can be more satisfactory when dealing with small sample sizes.

Download Full-text