scholarly journals Estimation of statistical power and false discovery rate of QTL mapping methods through computer simulation

2012 ◽  
Vol 57 (21) ◽  
pp. 2701-2710 ◽  
Author(s):  
HuiHui Li ◽  
LuYan Zhang ◽  
JianKang Wang
Keyword(s):  
Computer Simulation ◽  
Qtl Mapping ◽  
False Discovery Rate ◽  
Statistical Power ◽  
False Discovery

On “Field Significance” and the False Discovery Rate

2006 ◽  
Vol 45 (9) ◽  
pp. 1181-1189 ◽  
Author(s):  
D. S. Wilks
Keyword(s):  
False Discovery Rate ◽  
Statistical Power ◽  
Statistical Significance ◽  
Significance Test ◽  
P Value ◽  
Test Statistic ◽  
Global Test ◽  
Additional Advantage ◽  
Counting Procedure ◽  
False Discovery

Abstract The conventional approach to evaluating the joint statistical significance of multiple hypothesis tests (i.e., “field,” or “global,” significance) in meteorology and climatology is to count the number of individual (or “local”) tests yielding nominally significant results and then to judge the unusualness of this integer value in the context of the distribution of such counts that would occur if all local null hypotheses were true. The sensitivity (i.e., statistical power) of this approach is potentially compromised both by the discrete nature of the test statistic and by the fact that the approach ignores the confidence with which locally significant tests reject their null hypotheses. An alternative global test statistic that has neither of these problems is the minimum p value among all of the local tests. Evaluation of field significance using the minimum local p value as the global test statistic, which is also known as the Walker test, has strong connections to the joint evaluation of multiple tests in a way that controls the “false discovery rate” (FDR, or the expected fraction of local null hypothesis rejections that are incorrect). In particular, using the minimum local p value to evaluate field significance at a level αglobal is nearly equivalent to the slightly more powerful global test based on the FDR criterion. An additional advantage shared by Walker’s test and the FDR approach is that both are robust to spatial dependence within the field of tests. The FDR method not only provides a more broadly applicable and generally more powerful field significance test than the conventional counting procedure but also allows better identification of locations with significant differences, because fewer than αglobal × 100% (on average) of apparently significant local tests will have resulted from local null hypotheses that are true.

scholarly journals An averaging strategy to reduce variability in target-decoy estimates of false discovery rate

2018 ◽  
Author(s):  
Uri Keich ◽  
Kaipo Tamura ◽  
William Stafford Noble
Keyword(s):  
False Discovery Rate ◽  
Statistical Power ◽  
Shotgun Proteomics ◽  
Database Search ◽  
Proteomics Data ◽  
Decoy Database ◽  
Software Toolkit ◽  
True Proportion ◽  
False Discovery ◽  
False Discoveries

AbstractDecoy database search with target-decoy competition (TDC) provides an intuitive, easy-to-implement method for estimating the false discovery rate (FDR) associated with spectrum identifications from shotgun proteomics data. However, the procedure can yield different results for a fixed dataset analyzed with different decoy databases, and this decoy-induced variability is particularly problematic for smaller FDR thresholds, datasets or databases. In such cases, the nominal FDR might be 1% but the true proportion of false discoveries might be 10%. The averaged TDC protocol combats this problem by exploiting multiple independently shuffled decoy databases to provide an FDR estimate with reduced variability. We provide a tutorial introduction to aTDC, describe an improved variant of the protocol that offers increased statistical power, and discuss how to deploy aTDC in practice using the Crux software toolkit.

scholarly journals Discrete False-Discovery Rate Improves Identification of Differentially Abundant Microbes

mSystems ◽  
2017 ◽  
Vol 2 (6) ◽  
Author(s):  
Lingjing Jiang ◽  
Amnon Amir ◽  
James T. Morton ◽  
Ruth Heller ◽  
Ery Arias-Castro ◽  
...  
Keyword(s):  
Gene Expression ◽  
False Discovery Rate ◽  
Real World ◽  
Statistical Power ◽  
Gene Expression Analysis ◽  
New Method ◽  
Data Sets ◽  
Differential Abundance ◽  
False Discovery ◽  
Microbiome Data

ABSTRACT DS-FDR can achieve higher statistical power to detect significant findings in sparse and noisy microbiome data compared to the commonly used Benjamini-Hochberg procedure and other FDR-controlling procedures. Differential abundance testing is a critical task in microbiome studies that is complicated by the sparsity of data matrices. Here we adapt for microbiome studies a solution from the field of gene expression analysis to produce a new method, discrete false-discovery rate (DS-FDR), that greatly improves the power to detect differential taxa by exploiting the discreteness of the data. Additionally, DS-FDR is relatively robust to the number of noninformative features, and thus removes the problem of filtering taxonomy tables by an arbitrary abundance threshold. We show by using a combination of simulations and reanalysis of nine real-world microbiome data sets that this new method outperforms existing methods at the differential abundance testing task, producing a false-discovery rate that is up to threefold more accurate, and halves the number of samples required to find a given difference (thus increasing the efficiency of microbiome experiments considerably). We therefore expect DS-FDR to be widely applied in microbiome studies. IMPORTANCE DS-FDR can achieve higher statistical power to detect significant findings in sparse and noisy microbiome data compared to the commonly used Benjamini-Hochberg procedure and other FDR-controlling procedures.

Brain networks construction using Bayes FDR and average power function

2019 ◽  
Vol 29 (3) ◽  
pp. 866-878
Author(s):  
Piero Quatto ◽  
Nicolò Margaritella ◽  
Isa Costantini ◽  
Francesca Baglio ◽  
Massimo Garegnani ◽  
...  
Keyword(s):  
Time Series ◽  
False Discovery Rate ◽  
Correlation Coefficient ◽  
Power Function ◽  
Multiple Testing ◽  
Statistical Power ◽  
Brain Connectivity ◽  
Average Power ◽  
Statistical Error ◽  
False Discovery

Brain functional connectivity is a widely investigated topic in neuroscience. In recent years, the study of brain connectivity has been largely aided by graph theory. The link between time series recorded at multiple locations in the brain and the construction of a graph is usually an adjacency matrix. The latter converts a measure of the connectivity between two time series, typically a correlation coefficient, into a binary choice on whether the two brain locations are functionally connected or not. As a result, the choice of a threshold τ over the correlation coefficient is key. In the present work, we propose a multiple testing approach to the choice of τ that uses the Bayes false discovery rate and a new estimator of the statistical power called average power function to balance the two types of statistical error. We show that the proposed average power function estimator behaves well both in case of independence and weak dependence of the tests and it is reliable under several simulated dependence conditions. Moreover, we propose a robust method for the choice of τ using the 5% and 95% percentiles of the average power function and False Discovery Rate bootstrap distributions, respectively, to improve stability. We applied our approach to functional magnetic resonance imaging and high density electroencephalogram data.

scholarly journals A New Approach to the Problem of Multiple Comparisons in the Genetic Dissection of Complex Traits

Genetics ◽  
1998 ◽  
Vol 150 (4) ◽  
pp. 1699-1706 ◽  
Author(s):  
Joel Ira Weller ◽  
Jiu Zhou Song ◽  
David W Heyen ◽  
Harris A Lewin ◽  
Micha Ron
Keyword(s):  
False Discovery Rate ◽  
Complex Traits ◽  
Statistical Power ◽  
Type I Error ◽  
Type I ◽  
Genetic Dissection ◽  
Multiple Traits ◽  
Experimentwise Error Rate ◽  
False Discovery ◽  
Backcross Populations

Abstract Saturated genetic marker maps are being used to map individual genes affecting quantitative traits. Controlling the “experimentwise” type-I error severely lowers power to detect segregating loci. For preliminary genome scans, we propose controlling the “false discovery rate,” that is, the expected proportion of true null hypotheses within the class of rejected null hypotheses. Examples are given based on a granddaughter design analysis of dairy cattle and simulated backcross populations. By controlling the false discovery rate, power to detect true effects is not dependent on the number of tests performed. If no detectable genes are segregating, controlling the false discovery rate is equivalent to controlling the experimentwise error rate. If quantitative loci are segregating in the population, statistical power is increased as compared to control of the experimentwise type-I error. The difference between the two criteria increases with the increase in the number of false null hypotheses. The false discovery rate can be controlled at the same level whether the complete genome or only part of it has been analyzed. Additional levels of contrasts, such as multiple traits or pedigrees, can be handled without the necessity of a proportional decrease in the critical test probability.

scholarly journals dearseq: a variance component score test for RNA-seq differential analysis that effectively controls the false discovery rate

2020 ◽  
Vol 2 (4) ◽  
Author(s):  
Marine Gauthier ◽  
Denis Agniel ◽  
Rodolphe Thiébaut ◽  
Boris P Hejblum
Keyword(s):  
False Discovery Rate ◽  
Statistical Power ◽  
Differential Expression Analysis ◽  
Score Test ◽  
Real Data ◽  
Differential Analysis ◽  
Rna Seq ◽  
Data Set ◽  
Mathematical Proofs ◽  
False Discovery

Abstract RNA-seq studies are growing in size and popularity. We provide evidence that the most commonly used methods for differential expression analysis (DEA) may yield too many false positive results in some situations. We present dearseq, a new method for DEA that controls the false discovery rate (FDR) without making any assumption about the true distribution of RNA-seq data. We show that dearseq controls the FDR while maintaining strong statistical power compared to the most popular methods. We demonstrate this behavior with mathematical proofs, simulations and a real data set from a study of tuberculosis, where our method produces fewer apparent false positives.

scholarly journals Bayes Estimate of Primary Threshold in Cluster-wise fMRI Inferences

2020 ◽  
Author(s):  
Yunjiang Ge ◽  
Stephanie Hare ◽  
Gang Chen ◽  
James Waltz ◽  
Peter Kochunov ◽  
...  
Keyword(s):  
False Discovery Rate ◽  
Statistical Inference ◽  
Statistical Power ◽  
Empirical Bayes ◽  
Fmri Data ◽  
False Discovery ◽  
Proposed Model ◽  
The Family ◽  
Data Driven Approach ◽  
Selection Of

AbstractCluster-wise statistical inference is the most widely used technique for functional magnetic resonance imaging (fMRI) data analyses. Cluster-wise statistical inference consists of two steps: i) primary thresholding that excludes less significant voxels by a pre-specified cut-off (e.g., p < 0.001); and ii) cluster-wise thresholding that controls the family-wise error rate (FWER) caused by clusters consisting of false positive suprathreshold voxels. It has been well known that the selection of the primary threshold is critical because it determines both statistical power and false discovery rate. However, in most existing statistical packages, the primary threshold is selected based on prior knowledge (e.g., p < 0.001) without taking into account the information in the data. In this manuscript, we propose a data-driven approach to objectively select the optimal primary threshold based on an empirical Bayes framework. We evaluate the proposed model using extensive simulation studies and an fMRI data example. The results show that our method can effectively increase statistical power while effectively controlling the false discovery rate.

scholarly journals Leveraging auxiliary data from arbitrary distributions to boost GWAS discovery with Flexible cFDR

2020 ◽  
Author(s):  
Anna Hutchinson ◽  
Guillermo Reales ◽  
Chris Wallace
Keyword(s):  
False Discovery Rate ◽  
Complex Traits ◽  
Statistical Power ◽  
Association Studies ◽  
Continuous Distribution ◽  
Genome Wide Association Studies ◽  
Genetic Associations ◽  
Auxiliary Data ◽  
Significance Threshold ◽  
False Discovery

1.AbstractGenome-wide association studies (GWAS) have identified thousands of genetic variants that are associated with complex traits. However, a stringent significance threshold is required to identify robust genetic associations. Leveraging relevant auxiliary data has the potential to boost statistical power to exceed the significance threshold. Particularly, abundant pleiotropy and the non-random distribution of SNPs across various functional categories suggests that leveraging GWAS test statistics from related traits and/or functional genomic data may boost GWAS discovery. While type 1 error rate control has become standard in GWAS, control of the false discovery rate (FDR) can be a more powerful approach as sample sizes increase and many associations are expected in each study. The conditional false discovery rate (cFDR) extends the standard FDR framework by conditioning on auxiliary data to call significant associations, but current implementations are restricted to auxiliary data satisfying specific parametric distributions. We relax the distributional assumptions, enabling an extension of the cFDR framework that supports auxiliary data from any continuous distribution (“Flexible cFDR”). Our method is iterative, whereby additional layers of auxiliary data can be leveraged in turn. Through simulations we show that flexible cFDR increases sensitivity whilst controlling FDR after one or several iterations. We further demonstrate its practical potential through application to an asthma GWAS, leveraging various functional data to find additional genetic associations for asthma, which we validated in the larger UK Biobank data resource.

scholarly journals Mass spectrometrists should search for all peptides, but assess only the ones they care about

2017 ◽  
Author(s):  
Adriaan Sticker ◽  
Lennart Martens ◽  
Lieven Clement
Keyword(s):  
False Discovery Rate ◽  
Statistical Power ◽  
Mass Spectra ◽  
Shotgun Proteomics ◽  
False Discovery ◽  
Scientific Hypothesis ◽  
Calculation Results ◽  
Classical Strategy

AbstractIn shotgun proteomics identified mass spectra that are deemed irrelevant to the scientific hypothesis are often discarded. Noble (2015)1 therefore urged researchers to remove irrelevant peptides from the database prior to searching to improve statistical power. We here however, argue that both the classical as well as Noble’s revised method produce suboptimal peptide identifications and have problems in controlling the false discovery rate (FDR). Instead, we show that searching for all expected peptides, and removing irrelevant peptides prior to FDR calculation results in more reliable identifications at controlled FDR level than the classical strategy that discards irrelevant peptides post FDR calculation, or than Noble’s strategy that discards irrelevant peptides prior to searching.

Sign in / Sign up

Export Citation Format

Share Document