Multiple Testing Methods

Author(s):  
Alessio Farcomeni
2020 ◽  
Vol 10 (2) ◽  
pp. 199-248 ◽  
Author(s):  
Campbell R Harvey ◽  
Yan Liu ◽  
Alessio Saretto

Abstract In almost every area of empirical finance, researchers confront multiple tests. One high-profile example is the identification of outperforming investment managers, many of whom beat their benchmarks purely by luck. Multiple testing methods are designed to control for luck. Factor selection is another glaring case in which multiple tests are performed, but numerous other applications do not receive as much attention. One important example is a simple regression model testing five variables. In this case, because five variables are tried, a t-statistic of 2.0 is not enough to establish significance. Our paper provides a guide to various multiple testing methods and details a number of applications. We provide simulation evidence on the relative performance of different methods across a variety of testing environments. The goal of our paper is to provide a menu that researchers can choose from to improve inference in financial economics. (JEL G0, G1, G3, G5, M4, C1)


2019 ◽  
Vol 58 (3) ◽  
pp. 408-410 ◽  
Author(s):  
Paul O Lewis ◽  
Cameron G Lanier ◽  
Paras D Patel ◽  
Whitney D Krolikowski ◽  
Matthew A Krolikowski

Abstract The accuracy of the BioFire FilmArray Meningitis/Encephalitis (ME) panel for the identification of Cryptococcus has recently been called into question. The primary objective of this study was to assess the agreement between the BioFire ME polymerase chain reaction (PCR) and other markers of cryptococcal infection. This retrospective review identified five patients with cryptococcal meningoencephalitis, 4 of whom had a negative ME panel for Cryptococcus. All five cases had positive serum cryptococcal antigens, and three of five had a positive cerebrospinal fluid (CSF) culture for Cryptococcus. The BioFire ME panel does not appear to be reliable for ruling out Cryptococcus meningoencephalitis; multiple testing methods are recommended.


2021 ◽  
Vol 207 ◽  
pp. 111523
Author(s):  
Bethanie Carney Almroth ◽  
Josefin Cartine ◽  
Christina Jönander ◽  
Max Karlsson ◽  
Julie Langlois ◽  
...  

Author(s):  
Jinsong Chen ◽  
Mark J. van der Laan ◽  
Martyn T. Smith ◽  
Alan E. Hubbard

Microarray studies often need to simultaneously examine thousands of genes to determine which are differentially expressed. One main challenge in those studies is to find suitable multiple testing procedures that provide accurate control of the error rates of interest and meanwhile are most powerful, that is, they return the longest list of truly interesting genes among competitors. Many multiple testing methods have been developed recently for microarray data analysis, especially resampling based methods, such as permutation methods, the null-centered and scaled bootstrap (NCSB) method, and the quantile-transformed-bootstrap-distribution (QTBD) method. Each of these methods has its own merits and limitations. Theoretically permutation methods can fail to provide accurate control of Type I errors when the so-called subset pivotality condition is violated. The NCSB method does not suffer from that limitation, but an impractical number of bootstrap samples are often needed to get proper control of Type I errors. The newly developed QTBD method has the virtues of providing accurate control of Type I errors under few restrictions. However, the relative practical performance of the above three types of multiple testing methods remains unresolved. This paper compares the above three resampling based methods according to the control of family wise error rates (FWER) through data simulations. Results show that among the three resampling based methods, the QTBD method provides relatively accurate and powerful control in more general circumstances.


Author(s):  
Zhanglin Cui ◽  
Daniel H. Mowrey ◽  
Alan G. Zimmermann ◽  
Douglas E. Hutchens

Sign in / Sign up

Export Citation Format

Share Document