scholarly journals Integrating the evidence from evidence factors in observational studies

Biometrika ◽  
2019 ◽  
Vol 106 (2) ◽  
pp. 353-367 ◽  
Author(s):  
B Karmakar ◽  
B French ◽  
D S Small
Keyword(s):  
Sensitivity Analysis ◽  
Observational Study ◽  
Error Rate ◽  
Type I Error ◽  
R Package ◽  
Sensitivity Analyses ◽  
Bahadur Efficiency ◽  
Type I ◽  
Familywise Error Rate ◽  
Risk Of Cancer

Summary A sensitivity analysis for an observational study assesses how much bias, due to nonrandom assignment of treatment, would be necessary to change the conclusions of an analysis that assumes treatment assignment was effectively random. The evidence for a treatment effect can be strengthened if two different analyses, which could be affected by different types of biases, are both somewhat insensitive to bias. The finding from the observational study is then said to be replicated. Evidence factors allow for two independent analyses to be constructed from the same dataset. When combining the evidence factors, the Type I error rate must be controlled to obtain valid inference. A powerful method is developed for controlling the familywise error rate for sensitivity analyses with evidence factors. It is shown that the Bahadur efficiency of sensitivity analysis for the combined evidence is greater than for either evidence factor alone. The proposed methods are illustrated through a study of the effect of radiation exposure on the risk of cancer. An R package, evidenceFactors, is available from CRAN to implement the methods of the paper.

scholarly journals Pairwise Multiple Comparison Adjustment Procedure for Survival Functions with Right-Censored Data

2017 ◽  
Vol 2017 ◽  
pp. 1-8 ◽  
Author(s):  
Ertugrul Colak ◽  
Hulya Ozen ◽  
Busra Emir ◽  
Setenay Oner
Keyword(s):  
Error Rate ◽  
Survival Data ◽  
Type I Error ◽  
Multiple Comparison ◽  
Type I ◽  
Familywise Error Rate ◽  
Data Set ◽  
Right Censored Data ◽  
Adjustment Procedure ◽  
Liver Transplant Waiting List

The aim of this study is to propose a new pairwise multiple comparison adjustment procedure based on Genz’s numerical computation of probabilities from a multivariate normal distribution. This method is applied to the results of two-sample log-rank and weighted log-rank statistics where the survival data contained right-censored observations. We conducted Monte Carlo simulation studies not only to evaluate the familywise error rate and power of the proposed procedure but also to compare the procedure with conventional methods. The proposed method is also applied to the data set consisting of 815 patients on a liver transplant waiting list from 1990 to 1999. It was found that the proposed method can control the type I error rate, and it yielded similar power as Tukey’s and high power with respect to the other adjustment procedures. In addition to having a straightforward formula, it is easy to implement.

Finite Intersection Tests: A Paradigm for Optimizing Simultaneous and Sequential Inference

1997 ◽  
Vol 22 (3) ◽  
pp. 291-307 ◽  
Author(s):  
Gregory R. Hancock ◽  
Alan J. Klockars
Keyword(s):  
Error Rate ◽  
Type I Error ◽  
Reference Group ◽  
Type I ◽  
Familywise Error Rate ◽  
Finite Intersection ◽  
Simple Simulation ◽  
Simulation Strategy ◽  
Sequential Hypothesis Testing ◽  
Multiple Comparison Procedures

When testing a family of comparisons or contrasts across k treatment groups, researchers are often encouraged to maintain control over the familywise Type I error rate. For common families such as comparisons against a reference group, sets of orthogonal and/or nonorthogonal contrasts, and all possible pairwise comparisons, numerous simultaneous (and more recently sequential) testing methods have been proposed. Many of the simultaneous methods can be shown to be a form of Krishnaiah’s (e.g., 1979) finite intersection test (FIT) for simultaneous multiple comparisons, which controls the familywise error rate to precisely a under conditions assumed in standard ANOVA scenarios. Other methods, however, merely represent conservative approximations to a FIT procedure, yielding suboptimal power for conducting simultaneous testing. The purpose of the current article is threefold. First, we discuss how FIT methodology represents a paradigm that unifies many existing methods for simultaneous inference, as well as how it suggests an improved method for testing nonorthogonal contrasts. Second, we illustrate more powerful multiple comparison procedures that combine FIT methodology with sequential hypothesis testing strategies. Third, we present a simple simulation strategy for generating critical values necessary to conduct these more powerful FIT-based methods. Examples of these methods are given.

Correction: “Influence of Selection Bias on the Test Decision – A Simulation Study”

2014 ◽  
Vol 53 (05) ◽  
pp. 343-343
Keyword(s):  
Selection Bias ◽  
Simulation Study ◽  
Error Rate ◽  
Type I Error ◽  
Block Size ◽  
Error Rates ◽  
Type I ◽  
Type I Error Rate ◽  
Representation Error ◽  
Numeric Representation

We have to report marginal changes in the empirical type I error rates for the cut-offs 2/3 and 4/7 of Table 4, Table 5 and Table 6 of the paper “Influence of Selection Bias on the Test Decision – A Simulation Study” by M. Tamm, E. Cramer, L. N. Kennes, N. Heussen (Methods Inf Med 2012; 51: 138 –143). In a small number of cases the kind of representation of numeric values in SAS has resulted in wrong categorization due to a numeric representation error of differences. We corrected the simulation by using the round function of SAS in the calculation process with the same seeds as before. For Table 4 the value for the cut-off 2/3 changes from 0.180323 to 0.153494. For Table 5 the value for the cut-off 4/7 changes from 0.144729 to 0.139626 and the value for the cut-off 2/3 changes from 0.114885 to 0.101773. For Table 6 the value for the cut-off 4/7 changes from 0.125528 to 0.122144 and the value for the cut-off 2/3 changes from 0.099488 to 0.090828. The sentence on p. 141 “E.g. for block size 4 and q = 2/3 the type I error rate is 18% (Table 4).” has to be replaced by “E.g. for block size 4 and q = 2/3 the type I error rate is 15.3% (Table 4).”. There were only minor changes smaller than 0.03. These changes do not affect the interpretation of the results or our recommendations.

Controlling type I error rate for fast track drug development programmes

2003 ◽  
Vol 22 (5) ◽  
pp. 665-675 ◽  
Author(s):  
Weichung J. Shih ◽  
Peter Ouyang ◽  
Hui Quan ◽  
Yong Lin ◽  
Bart Michiels ◽  
...  
Keyword(s):  
Drug Development ◽  
Error Rate ◽  
Fast Track ◽  
Type I Error ◽  
Type I ◽  
Type I Error Rate

Alternative models and randomization techniques for Bayesian response-adaptive randomization with binary outcomes

2021 ◽  
pp. 174077452110101
Author(s):  
Jennifer Proper ◽  
John Connett ◽  
Thomas Murray
Keyword(s):  
Logistic Regression ◽  
Sample Size ◽  
Error Rate ◽  
Adaptive Design ◽  
Type I Error ◽  
Probability Model ◽  
Binary Outcomes ◽  
Type I ◽  
Operating Characteristics ◽  
Type I Error Rate

Background: Bayesian response-adaptive designs, which data adaptively alter the allocation ratio in favor of the better performing treatment, are often criticized for engendering a non-trivial probability of a subject imbalance in favor of the inferior treatment, inflating type I error rate, and increasing sample size requirements. The implementation of these designs using the Thompson sampling methods has generally assumed a simple beta-binomial probability model in the literature; however, the effect of these choices on the resulting design operating characteristics relative to other reasonable alternatives has not been fully examined. Motivated by the Advanced R2 Eperfusion STrategies for Refractory Cardiac Arrest trial, we posit that a logistic probability model coupled with an urn or permuted block randomization method will alleviate some of the practical limitations engendered by the conventional implementation of a two-arm Bayesian response-adaptive design with binary outcomes. In this article, we discuss up to what extent this solution works and when it does not. Methods: A computer simulation study was performed to evaluate the relative merits of a Bayesian response-adaptive design for the Advanced R2 Eperfusion STrategies for Refractory Cardiac Arrest trial using the Thompson sampling methods based on a logistic regression probability model coupled with either an urn or permuted block randomization method that limits deviations from the evolving target allocation ratio. The different implementations of the response-adaptive design were evaluated for type I error rate control across various null response rates and power, among other performance metrics. Results: The logistic regression probability model engenders smaller average sample sizes with similar power, better control over type I error rate, and more favorable treatment arm sample size distributions than the conventional beta-binomial probability model, and designs using the alternative randomization methods have a negligible chance of a sample size imbalance in the wrong direction. Conclusion: Pairing the logistic regression probability model with either of the alternative randomization methods results in a much improved response-adaptive design in regard to important operating characteristics, including type I error rate control and the risk of a sample size imbalance in favor of the inferior treatment.

Anova Tests for Homogeneity of Variance: Nonnormality and Unequal Samples

1977 ◽  
Vol 2 (3) ◽  
pp. 187-206 ◽  
Author(s):  
Charles G. Martin ◽  
Paul A. Games
Keyword(s):  
Error Rate ◽  
Type I Error ◽  
Type I ◽  
Empirical Comparison ◽  
Jackknife Test ◽  
Type I Error Rate ◽  
Power And Control ◽  
Homogeneity Of Variance ◽  
Test Use ◽  
And Control

This paper presents an exposition and an empirical comparison of two potentially useful tests for homogeneity of variance. Control of Type I error rate, P(EI), and power are investigated for three forms of the Box test and for two forms of the jackknife test with equal and unequal n's under conditions of normality and nonnormality. The Box test is shown to be robust to violations of the assumption of normality. The jackknife test is shown not to be robust. When n's are unequal, the problem of heterogeneous within-cell variances of the transformed values and unequal n's affects the jackknife and Box tests. Previously reported suggestions for selecting subsample sizes for the Box test are shown to be inappropriate, producing an inflated P(EI). Two procedures which alleviate this problem are presented for the Box test. Use of the jack-knife test with a reduced alpha is shown to provide power and control of P(EI) at approximately the same level as the Box test. Recommendations for the use of these techniques and computational examples of each are provided.

scholarly journals Non-inferiority test for a continuous variable with a flexible margin in an active controlled trial : An application to the "Stratall ANRS 12110 / ESTHER" trial

2020 ◽  
Author(s):  
Arsene Sandie ◽  
Nicholas Molinari ◽  
Anthony Wanjoya ◽  
Charles Kouanfack ◽  
Christian Laurent ◽  
...  
Keyword(s):  
Confidence Interval ◽  
Active Control ◽  
Error Rate ◽  
Type I Error ◽  
Clinical Monitoring ◽  
Small Scale ◽  
Type I ◽  
Test Procedures ◽  
Power Estimate ◽  
Control Intervention

Abstract Background: The non-inferiority trials are becoming increasingly popular in public health and clinical research. The choice of the non-inferiority margin is the cornerstone of the non-inferiority trial. When the effect of active control intervention is unknown, it can be interesting to choose the non-inferiority margin as a function of the active control intervention effect. In this case, the uncertainty surrounding the non-inferiority margin should be accounted for in statistical tests. In this work, we explored how to perform the non-inferiority test with a flexible margin for continuous endpoint.Methods: It was proposed in this study two procedures for the non-inferiority test with a flexible margin for the continuous endpoint. The proposed test procedures are based on test statistic and confidence interval approach. Simulations have been used to assess the performances and properties of the proposed test procedures. An application was done on clinical real data, which the purpose was to assess the efficacy of clinical monitoring alone versus laboratory and clinical monitoring in HIV-infected adult patients.Results: Basically, the two proposed test procedures have good properties. In the test based on a statistic, the actual type 1 error rate estimate is approximatively equal to the nominal value. It has been found that the confidence interval level determines approximately the level of significance. The 80%, 90%, and 95%one-sided confidence interval levels led approximately to a type I error of 10%, 5% and 2.5% respectively. The power estimate was almost 100% for two proposed tests, except for the small scale values of the reference treatment where the power was relatively low when the sample sizes were small.Conclusions: Based on type I error rate and power estimates, the proposed non-inferiority hypothesis test procedures have good performance and are applicable in practice.Trial registration: The trial data used in this study was from the ”Stratall ANRS 12110 / ESTHER”, registered with ClinicalTrials.gov, number NCT00301561. Date : March 13, 2006, url : https://clinicaltrials.gov/ct2/show/NCT00301561.

scholarly journals Implementation of the Omega (ω) Index to detect large-scale systematic cheating

2019 ◽  
Author(s):  
Alvin Vista
Keyword(s):  
Standardized Testing ◽  
Large Scale ◽  
Type I Error ◽  
R Package ◽  
Statistical Testing ◽  
System Level ◽  
Control Group ◽  
Type I ◽  
Data Contamination ◽  
Cheating Detection

Cheating detection is an important issue in standardized testing, especially in large-scale settings. Statistical approaches are often computationally intensive and require specialised software to conduct. We present a two-stage approach that quickly filters suspected groups using statistical testing on an IRT-based answer-copying index. We also present an approach to mitigate data contamination and improve the performance of the index. The computation of the index was implemented through a modified version of an open source R package, thus enabling wider access to the method. Using data from PIRLS 2011 (N=64,232) we conduct a simulation to demonstrate our approach. Type I error was well-controlled and no control group was falsely flagged for cheating, while 16 (combined n=12,569) of the 18 (combined n=14,149) simulated groups were detected. Implications for system-level cheating detection and further improvements of the approach were discussed.

Group sequential designs with robust semiparametric recurrent event models

2018 ◽  
Vol 28 (8) ◽  
pp. 2385-2403 ◽  
Author(s):  
Tobias Mütze ◽  
Ekkehard Glimm ◽  
Heinz Schmidli ◽  
Tim Friede
Keyword(s):  
Error Rate ◽  
Joint Distribution ◽  
Recurrent Events ◽  
Type I Error ◽  
Type I ◽  
Sequential Designs ◽  
Group Sequential ◽  
Type I Error Rate ◽  
Sequential Procedures ◽  
Group Sequential Designs

Robust semiparametric models for recurrent events have received increasing attention in the analysis of clinical trials in a variety of diseases including chronic heart failure. In comparison to parametric recurrent event models, robust semiparametric models are more flexible in that neither the baseline event rate nor the process inducing between-patient heterogeneity needs to be specified in terms of a specific parametric statistical model. However, implementing group sequential designs in the robust semiparametric model is complicated by the fact that the sequence of Wald statistics does not follow asymptotically the canonical joint distribution. In this manuscript, we propose two types of group sequential procedures for a robust semiparametric analysis of recurrent events. The first group sequential procedure is based on the asymptotic covariance of the sequence of Wald statistics and it guarantees asymptotic control of the type I error rate. The second procedure is based on the canonical joint distribution and does not guarantee asymptotic type I error rate control but is easy to implement and corresponds to the well-known standard approach for group sequential designs. Moreover, we describe how to determine the maximum information when planning a clinical trial with a group sequential design and a robust semiparametric analysis of recurrent events. We contrast the operating characteristics of the proposed group sequential procedures in a simulation study motivated by the ongoing phase 3 PARAGON-HF trial (ClinicalTrials.gov identifier: NCT01920711) in more than 4600 patients with chronic heart failure and a preserved ejection fraction. We found that both group sequential procedures have similar operating characteristics and that for some practically relevant scenarios, the group sequential procedure based on the canonical joint distribution has advantages with respect to the control of the type I error rate. The proposed method for calculating the maximum information results in appropriately powered trials for both procedures.

Sign in / Sign up

Export Citation Format

Share Document