Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings

Abstract In Barber & Candès (2015, Ann. Statist., 43, 2055–2085), the authors introduced a new variable selection procedure called the knockoff filter to control the false discovery rate (FDR) and proved that this method achieves exact FDR control. Inspired by the work by Barber & Candès (2015, Ann. Statist., 43, 2055–2085), we propose a pseudo knockoff filter that inherits some advantages of the original knockoff filter and has more flexibility in constructing its knockoff matrix. Moreover, we perform a number of numerical experiments that seem to suggest that the pseudo knockoff filter with the half Lasso statistic has FDR control and offers more power than the original knockoff filter with the Lasso Path or the half Lasso statistic for the numerical examples that we consider in this paper. Although we cannot establish rigourous FDR control for the pseudo knockoff filter, we provide some partial analysis of the pseudo knockoff filter with the half Lasso statistic and establish a uniform false discovery proportion bound and an expectation inequality.

Download Full-text

Control of the False Discovery Proportion for Independently Tested Null Hypotheses

Journal of Probability and Statistics ◽

10.1155/2012/320425 ◽

2012 ◽

Vol 2012 ◽

pp. 1-19 ◽

Cited By ~ 2

Author(s):

Yongchao Ge ◽

Xiaochun Li

Keyword(s):

Multiple Testing ◽

Asymptotic Theory ◽

Random Variable ◽

Convergence Problem ◽

Finite Samples ◽

False Discovery ◽

Classical Procedure ◽

Multiple Testing Problem ◽

Upper Confidence Bound ◽

False Discovery Proportion

Consider the multiple testing problem of testingmnull hypothesesH1,…,Hm, among whichm0hypotheses are truly null. Given theP-values for each hypothesis, the question of interest is how to combine theP-values to find out which hypotheses are false nulls and possibly to make a statistical inference onm0. Benjamini and Hochberg proposed a classical procedure that can control the false discovery rate (FDR). The FDR control is a little bit unsatisfactory in that it only concerns the expectation of the false discovery proportion (FDP). The control of the actual random variable FDP has recently drawn much attention. For any level1−α, this paper proposes a procedure to construct an upper prediction bound (UPB) for the FDP for a fixed rejection region. When1−α=50%, our procedure is very close to the classical Benjamini and Hochberg procedure. Simultaneous UPBs for all rejection regions' FDPs and the upper confidence bound for the unknownm0are presented consequently. This new proposed procedure works for finite samples and hence avoids the slow convergence problem of the asymptotic theory.

Download Full-text

Sample size calculations for controlling the distribution of false discovery proportion in microarray experiments

Biostatistics ◽

10.1093/biostatistics/kxp024 ◽

2009 ◽

Vol 10 (4) ◽

pp. 694-705 ◽

Cited By ~ 8

Author(s):

T. Oura ◽

S. Matsui ◽

K. Kawakami

Keyword(s):

Sample Size ◽

Microarray Experiments ◽

False Discovery ◽

Sample Size Calculations ◽

False Discovery Proportion

Download Full-text

Exact calculations for false discovery proportion with application to least favorable configurations

The Annals of Statistics ◽

10.1214/10-aos847 ◽

2011 ◽

Vol 39 (1) ◽

pp. 584-612 ◽

Cited By ~ 13

Author(s):

Etienne Roquain ◽

Fanny Villers

Keyword(s):

False Discovery ◽

Exact Calculations ◽

False Discovery Proportion

Download Full-text

Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing

Biometrika ◽

10.1093/biomet/asz041 ◽

2019 ◽

Vol 106 (4) ◽

pp. 841-856 ◽

Cited By ~ 4

Author(s):

Jelle J Goeman ◽

Rosa J Meijer ◽

Thijmen J P Krebs ◽

Aldo Solari

Keyword(s):

Large Scale ◽

Average Power ◽

Multiple Hypothesis Testing ◽

Familywise Error Rate ◽

Confidence Bounds ◽

Testing Procedures ◽

False Discovery ◽

Simultaneous Control ◽

Simultaneous Confidence Bounds ◽

False Discovery Proportion

Summary Closed testing procedures are classically used for familywise error rate control, but they can also be used to obtain simultaneous confidence bounds for the false discovery proportion in all subsets of the hypotheses, allowing for inference robust to post hoc selection of subsets. In this paper we investigate the special case of closed testing with Simes local tests. We construct a novel fast and exact shortcut and use it to investigate the power of this approach when the number of hypotheses goes to infinity. We show that if a minimal level of signal is present, the average power to detect false hypotheses at any desired false discovery proportion does not vanish. Additionally, we show that the confidence bounds for false discovery proportion are consistent estimators for the true false discovery proportion for every nonvanishing subset. We also show close connections between Simes-based closed testing and the procedure of Benjamini and Hochberg.

Download Full-text