scholarly journals An Improved Two Independent-Samples Randomization Test for Single-Case AB-Type Intervention Designs: A 20-Year Journey

2020 ◽  
Vol 18 (1) ◽  
pp. 2-20
Author(s):  
Joel R. Levin ◽  
John M. Ferron ◽  
Boris S. Gafurov
Keyword(s):  
Error Control ◽  
Type I Error ◽  
Single Case ◽  
Test Procedure ◽  
Randomization Test ◽  
Multiple Baseline ◽  
Type I ◽  
Two Phase ◽  
Sample Test ◽  
Adequate Power

Detailed is a 20-year arduous journey to develop a statistically viable two-phase (AB) single-case two independent-samples randomization test procedure. The test is designed to compare the effectiveness of two different interventions that are randomly assigned to cases. In contrast to the unsatisfactory simulation results produced by an earlier proposed randomization test, the present test consistently exhibited acceptable Type I error control under various design and effect-type configurations, while at the same time possessing adequate power to detect moderately sized intervention-difference effects. Selected issues, applications, and a multiple-baseline extension of the two-sample test are discussed.

scholarly journals Random Assignment of Intervention Points in Two-Phase Single-Case Designs: Data-Division-Specific Distributions

2008 ◽  
Vol 103 (2) ◽  
pp. 499-515 ◽  
Author(s):  
Antonio Solanas ◽  
Vicenta Sierra ◽  
Vicenç Quera ◽  
Rumen Manolov
Keyword(s):  
Single Case ◽  
Randomization Test ◽  
Error Rates ◽  
Data Series ◽  
Type I ◽  
Random Assignment ◽  
Test Statistic ◽  
Two Phase ◽  
Intervention Point ◽  
Data Division

The present study explored the statistical properties of a randomization test based on the random assignment of the intervention point in a two-phase (AB) single-case design. The focus is on randomization distributions constructed with the values of the test statistic for all possible random assignments and used to obtain p values. The shape of those distributions is investigated for each specific data division defined by the moment in which the intervention is introduced. Another aim of the study consisted in testing the detection of inexistent effects (i.e., production of false alarms) in autocorrelated data series, in which the assumption of exchangeability between observations may be untenable. In this way, it was possible to compare nominal and empirical Type I error rates to obtain evidence on the statistical validity of the randomization test for each individual data division. The results suggest that, when either of the two phases has considerably fewer measurement times, Type I errors may be too probable and, hence, the decision-making process to be carried out by applied researchers may be jeopardized.

scholarly journals Testing for equality of distributions using the concept of (niche) overlap

2021 ◽  
Author(s):  
Judith H. Parkinson-Schwarz ◽  
Arne C. Bathke
Keyword(s):  
Basic Assumption ◽  
Type I Error ◽  
Test Procedure ◽  
Niche Overlap ◽  
Relative Effect ◽  
Type I ◽  
Underlying Distribution ◽  
Parametric Test ◽  
Higher Power ◽  
Non Parametric

AbstractIn this paper, we propose a new non-parametric test for equality of distributions. The test is based on the recently introduced measure of (niche) overlap and its rank-based estimator. As the estimator makes only one basic assumption on the underlying distribution, namely continuity, the test is universal applicable in contrast to many tests that are restricted to only specific scenarios. By construction, the new test is capable of detecting differences in location and scale. It thus complements the large class of rank-based tests that are constructed based on the non-parametric relative effect. In simulations this new test procedure obtained higher power and lower type I error compared to two common tests in several settings. The new procedure shows overall good performance. Together with its simplicity, this test can be used broadly.

Comparison of Means: An F Test Followed by a Modified Multiple Range Procedure

1979 ◽  
Vol 4 (1) ◽  
pp. 14-23 ◽  
Author(s):  
Juliet Popper Shaffer
Keyword(s):  
Error Control ◽  
Type I Error ◽  
Critical Value ◽  
The Other ◽  
Type I ◽  
F Test ◽  
Range Test

If used only when a preliminary F test yields significance, the usual multiple range procedures can be modified to increase the probability of detecting differences without changing the control of Type I error. The modification consists of a reduction in the critical value when comparing the largest and smallest means. Equivalence of modified and unmodified procedures in error control is demonstrated. The modified procedure is also compared with the alternative of using the unmodified range test without a preliminary F test, and it is shown that each has advantages over the other under some circumstances.

scholarly journals Efficient Noninferiority Testing Procedures for Simultaneously Assessing Sensitivity and Specificity of Two Diagnostic Tests

2015 ◽  
Vol 2015 ◽  
pp. 1-7 ◽  
Author(s):  
Guogen Shan ◽  
Amei Amei ◽  
Daniel Young
Keyword(s):  
Sensitivity And Specificity ◽  
Diagnostic Tests ◽  
Error Control ◽  
Type I Error ◽  
Binary Outcomes ◽  
Type I ◽  
Test Statistics ◽  
Asymptotic Approach ◽  
Testing Procedures ◽  
Assessing Sensitivity

Sensitivity and specificity are often used to assess the performance of a diagnostic test with binary outcomes. Wald-type test statistics have been proposed for testing sensitivity and specificity individually. In the presence of a gold standard, simultaneous comparison between two diagnostic tests for noninferiority of sensitivity and specificity based on an asymptotic approach has been studied by Chen et al. (2003). However, the asymptotic approach may suffer from unsatisfactory type I error control as observed from many studies, especially in small to medium sample settings. In this paper, we compare three unconditional approaches for simultaneously testing sensitivity and specificity. They are approaches based on estimation, maximization, and a combination of estimation and maximization. Although the estimation approach does not guarantee type I error, it has satisfactory performance with regard to type I error control. The other two unconditional approaches are exact. The approach based on estimation and maximization is generally more powerful than the approach based on maximization.

scholarly journals Statistical inference of genetic pathway analysis in high dimensions

Biometrika ◽  
2019 ◽  
Vol 106 (3) ◽  
pp. 651-651
Author(s):  
Yang Liu ◽  
Wei Sun ◽  
Alexander P Reiner ◽  
Charles Kooperberg ◽  
Qianchuan He
Keyword(s):  
Statistical Inference ◽  
Pathway Analysis ◽  
Genetic Variants ◽  
Error Control ◽  
Genome Wide Association Study ◽  
Type I Error ◽  
High Density Lipoproteins ◽  
Type I ◽  
Genetic Pathway ◽  
A Genome

Summary Genetic pathway analysis has become an important tool for investigating the association between a group of genetic variants and traits. With dense genotyping and extensive imputation, the number of genetic variants in biological pathways has increased considerably and sometimes exceeds the sample size $n$. Conducting genetic pathway analysis and statistical inference in such settings is challenging. We introduce an approach that can handle pathways whose dimension $p$ could be greater than $n$. Our method can be used to detect pathways that have nonsparse weak signals, as well as pathways that have sparse but stronger signals. We establish the asymptotic distribution for the proposed statistic and conduct theoretical analysis on its power. Simulation studies show that our test has correct Type I error control and is more powerful than existing approaches. An application to a genome-wide association study of high-density lipoproteins demonstrates the proposed approach.

scholarly journals Type I error control in biomarker-stratified clinical trials

Trials ◽  
2015 ◽  
Vol 16 (S2) ◽  
Author(s):  
Deepak Parashar ◽  
Jack Bowden ◽  
Colin Starr ◽  
Lorenz Wernisch ◽  
Adrian Mander
Keyword(s):  
Clinical Trials ◽  
Error Control ◽  
Type I Error ◽  
Type I

No counts, no variance: allowing for loss of degrees of freedom when assessing biological variability from RNA-seq data

2017 ◽  
Vol 16 (2) ◽  
Author(s):  
Aaron T. L. Lun ◽  
Gordon K. Smyth
Keyword(s):  
Software Package ◽  
Error Control ◽  
Degrees Of Freedom ◽  
Linear Models ◽  
Type I Error ◽  
Real Data ◽  
Type I ◽  
Rna Seq ◽  
Study Gene Expression ◽  
Complex Models

AbstractRNA sequencing (RNA-seq) is widely used to study gene expression changes associated with treatments or biological conditions. Many popular methods for detecting differential expression (DE) from RNA-seq data use generalized linear models (GLMs) fitted to the read counts across independent replicate samples for each gene. This article shows that the standard formula for the residual degrees of freedom (d.f.) in a linear model is overstated when the model contains fitted values that are exactly zero. Such fitted values occur whenever all the counts in a treatment group are zero as well as in more complex models such as those involving paired comparisons. This misspecification results in underestimation of the genewise variances and loss of type I error control. This article proposes a formula for the reduced residual d.f. that restores error control in simulated RNA-seq data and improves detection of DE genes in a real data analysis. The new approach is implemented in the quasi-likelihood framework of the edgeR software package. The results of this article also apply to RNA-seq analyses that apply linear models to log-transformed counts, such as those in the limma software package, and more generally to any count-based GLM where exactly zero fitted values are possible.

Association of human endogenous retrovirus (hERV) expression with clinical efficacy of PD-1 blockade in metastatic clear cell renal cell carcinoma (mccRCC).

2019 ◽  
Vol 37 (15_suppl) ◽  
pp. 4568-4568 ◽  
Author(s):  
Jean-Christophe Pignon ◽  
Opeyemi Jegede ◽  
Sachet A Shukla ◽  
David A. Braun ◽  
Christine Horak ◽  
...  
Keyword(s):  
Error Control ◽  
Type I Error ◽  
Endogenous Retrovirus ◽  
Phase Iii ◽  
Human Endogenous Retrovirus ◽  
Rank Test ◽  
Type I ◽  
Limit Of Quantification ◽  
Cell Renal Cell Carcinoma ◽  
Exact Test

4568 Background: hERV levels positively correlate with tumor immune infiltrate and were recently shown to be associated with clinical benefit to PD-1/PD-L1 blockade in two small cohorts of patients (pts) with mccRCC (Smith C.C. et al and Panda A. et al; 2018). We tested whether hERV levels correlate with efficacy of nivolumab in a prospective phase II study of pts with mccRCC (Checkmate 010). Methods: Reverse transcribed RNA extracted from 99 FFPE pretreatment tumors were analyzed by RT-qPCR to assess levels of pan- ERVE4, pan- ERV3.2, hERV4700 GAG or ENV, and the reference genes 18S and HPRT1. Normalized hERV levels were transformed as categorical value (high or low) using population quartiles as cutoffs. For each cutoff, samples with non-quantifiable hERV levels for which the limit of quantification was above the tested cutoff could not be categorized and were excluded from analysis. Log rank test was used to test the association of hERV levels with PFS/irPFS (RECISTv1.1/irRECIST) at each cutoff using Holm-Bonferroni correction for Type I error control; adjusted P-values are reported. Fisher’s exact test was then used to explore the association with ORR/irORR (RECISTv1.1/irRECIST). Results: Among the hERV studied, only hERV4700 ENV was significantly associated with PFS/irPFS. At the 25th percentile cutoff, 45 pts had high levels of hERV4700 ENV and 24 pts had low levels of hERV4700 ENV. Median PFS and irPFS were significantly longer in the high- hERV4700 ENV group [7.0 (95% CI: 2.2 - 10.2) and 8.5 (95% CI: 4.2 - 14.1) months, respectively] versus the low- hERV4700 ENV group [2.6 (95% CI: 1.4 - 5.4) and 2.9 (95% CI: 1.4 - 5.7) months, respectively] ( P = 0.010 for PFS and P = 0.028 for irPFS). At the same cutoff, ORR and irORR rates were significantly higher in the high- hERV4700 ENV group [35.6 (95% CI: 21.9 - 51.2) % for both ORR/irORR] versus the low- hERV4700 ENV group [12.5 (95% CI: 2.7 - 32.4) and 8.3 (95% CI: 1.0 - 27.0) %, respectively] ( P = 0.036 for ORR and P = 0.012 for irORR). Conclusions: hERV4700 ENV levels may predict outcome on nivolumab in mccRCC. Validation of our results and correlation of hERV levels with immune markers in a controlled phase III trial (CheckMate 025) is ongoing.

scholarly journals Comparison of randomization-test procedures for single-case multiple-baseline designs

2016 ◽  
Vol 21 (5) ◽  
pp. 290-311 ◽  
Author(s):  
Joel R. Levin ◽  
John M. Ferron ◽  
Boris S. Gafurov
Keyword(s):  
Single Case ◽  
Randomization Test ◽  
Multiple Baseline ◽  
Test Procedures

scholarly journals Optimal testing of statistical hypotheses and multiple familywise error rates

Filomat ◽  
2016 ◽  
Vol 30 (3) ◽  
pp. 681-688
Author(s):  
Farshin Hormozinejad
Keyword(s):  
Error Control ◽  
Hypothesis Test ◽  
Probability Distributions ◽  
Test Procedure ◽  
Functional Relation ◽  
Error Rates ◽  
Type I ◽  
Hypotheses Testing ◽  
Statistical Hypotheses ◽  
Statistical Hypotheses Testing

In this article the author considers the statistical hypotheses testing to make decision among hypotheses concerning many families of probability distributions. The statistician would like to control the overall error rate relative to draw statistically valid conclusions from each test, while being as efficient as possible. The familywise error (FWE) rate metric and the hypothesis test procedure while controlling both the type I and II FWEs are generalized. The proposed procedure shows simultaneous more reliability and less conservative error control relative to fixed sample and other recently proposed sequential procedures. Also, the characteristics of logarithmically asymptotically optimal (LAO) hypotheses testing are studied. The purpose of research is to express the optimal functional relation among the reliabilities of LAO hypotheses testing and to judge with FWE metric.

Sign in / Sign up

Export Citation Format

Share Document