scholarly journals Dichotomous thinking and informational waste in neuroimaging

2021 ◽  
Author(s):  
Gang Chen ◽  
Paul A Taylor ◽  
Joel Stoddard ◽  
Robert W Cox ◽  
Peter A Bandettini ◽  
...  
Keyword(s):  
Multiple Testing ◽  
Predictive Accuracy ◽  
Univariate Analysis ◽  
Model Data ◽  
Modeling Framework ◽  
Nominal Level ◽  
Family Wise Error Rate ◽  
Statistical Inferences ◽  
The Family ◽  
Spatial Locations

Neuroimaging relies on separate statistical inferences at tens of thousands of spatial locations. Such massively univariate analysis typically requires adjustment for multiple testing in an attempt to maintain the family-wise error rate at a nominal level of 5%. We discuss how this approach is associated with substantial information loss because of an implicit but questionable assumption about the effect distribution across spatial units. To improve inference efficiency, predictive accuracy, and generalizability, we propose a Bayesian multilevel modeling framework. In addition, we make four actionable suggestions to alleviate information waste and to improve reproducibility: (1) abandon strict dichotomization; (2) report full results; (3) quantify effects, and (4) model data hierarchy.

scholarly journals Optimizing effective numbers of tests by vine copula modeling

2020 ◽  
Vol 8 (1) ◽  
pp. 172-185
Author(s):  
Nico Steffen ◽  
Thorsten Dickhaus
Keyword(s):  
Multiple Testing ◽  
Test Statistics ◽  
Effective Number ◽  
Significance Level ◽  
Multiple Test ◽  
Family Wise Error Rate ◽  
Copula Modeling ◽  
The Family ◽  
Vine Copula

AbstractIn the multiple testing context, we utilize vine copulae for optimizing the effective number of tests. It is well known that for the calibration of multiple tests for control of the family-wise error rate the dependencies between the marginal tests are of utmost importance. It has been shown in previous work, that positive dependencies between the marginal tests can be exploited in order to derive a relaxed Šidák-type multiplicity correction. This correction can conveniently be expressed by calculating the corresponding „effective number of tests“ for a given (global) significance level. This methodology can also be applied to blocks of test statistics so that the effective number of tests can be calculated by the sum of the effective numbers of tests for each block. In the present work, we demonstrate how the power of the multiple test can be optimized by taking blocks with high inner-block dependencies. The determination of those blocks will be performed by means of an estimated vine copula model. An algorithm is presented which uses the information of the estimated vine copula to make a data-driven choice of appropriate blocks in terms of (estimated) dependencies. Numerical experiments demonstrate the usefulness of the proposed approach.

scholarly journals Biostatistics for Multiple Testing

2020 ◽  
Vol 63 (3) ◽  
pp. 97-100
Author(s):  
Jeong-Seok Choi
Keyword(s):  
False Discovery Rate ◽  
Error Rate ◽  
Null Hypothesis ◽  
Multiple Testing ◽  
Statistical Techniques ◽  
Testing Hypotheses ◽  
Family Wise Error Rate ◽  
False Discovery ◽  
The Family ◽  
True Null Hypotheses

Multiple testings are instances that contain simultaneous tests for more than one hypothesis. When multiple testings are conducted at the same time, it is more likely that the null hypothesis is rejected, even if the null hypothesis is correct. If individual hypothesis decisions are based on unadjusted <i>p</i>-values, it is usually more likely that some of the true null hypotheses will be rejected. In order to solve the multiple testing problems, various studies have attempted to increase the power by taking into account the family-wise error rate or false discovery rate and statistics required for testing hypotheses. This article discuss methods that account for the multiplicity issue and introduces various statistical techniques.

scholarly journals Multiple Testing. Part II. Step-Down Procedures for Control of the Family-Wise Error Rate

2004 ◽  
Vol 3 (1) ◽  
pp. 1-33 ◽  
Author(s):  
Mark J. van der Laan ◽  
Sandrine Dudoit ◽  
Katherine S. Pollard
Keyword(s):  
Error Rate ◽  
Multiple Testing ◽  
Type I Error ◽  
Null Distribution ◽  
Type I ◽  
Test Statistics ◽  
P Values ◽  
Family Wise Error Rate ◽  
The Family ◽  
Step Down

The present article proposes two step-down multiple testing procedures for asymptotic control of the family-wise error rate (FWER): the first procedure is based on maxima of test statistics (step-down maxT), while the second relies on minima of unadjusted p-values (step-down minP). A key feature of our approach is the characterization and construction of a test statistics null distribution (rather than data generating null distribution) for deriving cut-offs for these test statistics (i.e., rejection regions) and the resulting adjusted p-values. For general null hypotheses, corresponding to submodels for the data generating distribution, we identify an asymptotic domination condition for a null distribution under which the step-down maxT and minP procedures asymptotically control the Type I error rate, for arbitrary data generating distributions, without the need for conditions such as subset pivotality. Inspired by this general characterization, we then propose as an explicit null distribution the asymptotic distribution of the vector of null value shifted and scaled test statistics. Step-down procedures based on consistent estimators of the null distribution are shown to also provide asymptotic control of the Type I error rate. A general bootstrap algorithm is supplied to conveniently obtain consistent estimators of the null distribution.

scholarly journals Distributions associated with simultaneous multiple hypothesis testing

2020 ◽  
Vol 7 (1) ◽  
Author(s):  
Chang Yu ◽  
Daniel Zelterman
Keyword(s):  
Latent Variable ◽  
Power Analysis ◽  
Marginal Distribution ◽  
Multiple Hypothesis Testing ◽  
Family Wise Error Rate ◽  
Uniform Distributions ◽  
Multiple Hypothesis ◽  
The Family ◽  
Alternative Hypotheses ◽  
Cancer Studies

Abstract We develop the distribution for the number of hypotheses found to be statistically significant using the rule from Simes (Biometrika 73: 751–754, 1986) for controlling the family-wise error rate (FWER). We find the distribution of the number of statistically significant p-values under the null hypothesis and show this follows a normal distribution under the alternative. We propose a parametric distribution ΨI(·) to model the marginal distribution of p-values sampled from a mixture of null uniform and non-uniform distributions under different alternative hypotheses. The ΨI distribution is useful when there are many different alternative hypotheses and these are not individually well understood. We fit ΨI to data from three cancer studies and use it to illustrate the distribution of the number of notable hypotheses observed in these examples. We model dependence in sampled p-values using a latent variable. These methods can be combined to illustrate a power analysis in planning a larger study on the basis of a smaller pilot experiment.

scholarly journals Prospective Validation of Pentraxin-3 as a Novel Serum Biomarker to Predict the Risk of Prostate Cancer in Patients Scheduled for Prostate Biopsy

Cancers ◽  
2021 ◽  
Vol 13 (7) ◽  
pp. 1611
Author(s):  
Ugo Giovanni Falagario ◽  
Gian Maria Busetto ◽  
Giuseppe Stefano Netti ◽  
Francesca Sanguedolce ◽  
Oscar Selvaggio ◽  
...  
Keyword(s):  
Prostate Biopsy ◽  
Predictive Accuracy ◽  
Characteristic Curve ◽  
Univariate Analysis ◽  
Digital Rectal Examination ◽  
Multivariable Logistic Regression Analysis ◽  
Gleason Grade ◽  
Pentraxin 3 ◽  
Net Benefit ◽  
Multivariable Models

Purpose: To test and internally validate serum Pentraxin-3 (PTX3) levels as a potential PCa biomarker to predict prostate biopsy (PBx) results. Materials and Methods: Serum PSA and serum PTX3 were prospectively assessed in patients scheduled for PBx at our Institution due to increased serum PSA levels or abnormal digital rectal examination. Uni- and multivariable logistic regression analysis, area under the receiver operating characteristic curve (AUC), and decision curve analysis (DCA), were used to test the accuracy of serum PTX3 in predicting anyPCa and clinically significant PCa (csPCa) defined as Gleason Grade (GG) ≥ 2. Results: Among the 455 eligible patients, PCa was detected in 49% and csPCa in 25%. During univariate analysis, PTX3 outperformed other variables in predicting both anyPCa and csPCa. The addition of PTX3 to multivariable models based on standard clinical variables, significantly increased each model’s predictive accuracy for anyPCa (AUC from 0.73 to 0.82; p < 0.001) and csPCa (AUC from 0.79 to 0.83; p < 0.001). At DCA, PTX3, and PTX3, density showed higher net benefit than PSA and PSA density and increased the net benefit of multivariable models in deciding when to perform PBx. Conclusions: Serum PTX3 levels might be of clinical utility in predicting prostate biopsy results. Should our findings be confirmed, this novel reflex test could be used to reduce the number and burden of unnecessary prostate biopsies.

Histopathologic-Based Prognostic Factors of Colorectal Cancers Are Associated With the State of the Local Immune Reaction

2011 ◽  
Vol 29 (6) ◽  
pp. 610-618 ◽  
Author(s):  
Bernhard Mlecnik ◽  
Marie Tosolini ◽  
Amos Kirilovsky ◽  
Anne Berger ◽  
Gabriela Bindea ◽  
...  
Keyword(s):  
Predictive Accuracy ◽  
Early Stage ◽  
Univariate Analysis ◽  
Prognostic Significance ◽  
Tnm Staging ◽  
Time Dependent ◽  
Colorectal Cancers ◽  
Tumor Extension ◽  
T Stage ◽  
Immune Score

Purpose The prognosis of patients with colorectal cancer has sometimes proved uncertain; thus, the prognostic significance of immune criteria was compared with that of the tumor extension criteria using the American Joint Committee on Cancer/International Union Against Cancer–TNM (AJCC/UICC-TNM) staging system. Patients and Methods We studied the intratumoral immune infiltrates in the center of the tumor and in the invasive margin of 599 specimens of stage I to IV colorectal cancers from two independent cohorts. We analyzed these findings in relation to the degree of tumor extension and to the frequency of recurrence. Results Growth of the primary tumor and metastatic spread were associated with decreased intratumoral immune T-cell densities. Sixty percent of patients with high densities of CD8+ cytotoxic T-lymphocyte infiltrate presented with stage Tis/T1 tumor, whereas no patients with low densities presented with such early-stage tumor. In patients who did not relapse, the density of CD8 infiltrates was inversely correlated with T stage. In contrast, in patients whose tumor recurred, the number of CD8 cells was low regardless of the T stage of the tumor. Univariate analysis showed that the immune score was significantly associated with differences in disease-free, disease-specific, and overall survival (hazard ratio [HR], 0.64, 0.60, and 0.70, respectively; P < .005). Time-dependent receiver operating characteristic curve analysis illustrated the predictive accuracy of the immune parameters (c-index = 65.3%, time-dependent c-index [Cτ] = 66.5%). A final stepwise model for Cox multivariate analysis supports the advantage of the immune score (HR, 0.64; P < .001; Cτ = 67.9%) compared with histopathologic features in predicting recurrence as well as survival. Conclusion Assessment of CD8+ cytotoxic T lymphocytes in combined tumor regions provides an indicator of tumor recurrence beyond that predicted by AJCC/UICC-TNM staging.

scholarly journals Multiplicity- and dependency-adjusted p-values for control of the family-wise error rate

2016 ◽  
Vol 111 ◽  
pp. 32-40 ◽  
Author(s):  
Jens Stange ◽  
Thorsten Dickhaus ◽  
Arcadi Navarro ◽  
Daniel Schunk
Keyword(s):  
Error Rate ◽  
P Values ◽  
Family Wise Error Rate ◽  
The Family

Uncertainty and Inference for Verification Measures

2007 ◽  
Vol 22 (3) ◽  
pp. 637-650 ◽  
Author(s):  
Ian T. Jolliffe
Keyword(s):  
Hypothesis Testing ◽  
Confidence Intervals ◽  
Multiple Testing ◽  
Prediction Intervals ◽  
Hypothesis Tests ◽  
Statistical Inferences ◽  
The Difference ◽  
Main Ideas

Abstract When a forecast is assessed, a single value for a verification measure is often quoted. This is of limited use, as it needs to be complemented by some idea of the uncertainty associated with the value. If this uncertainty can be quantified, it is then possible to make statistical inferences based on the value observed. There are two main types of inference: confidence intervals can be constructed for an underlying “population” value of the measure, or hypotheses can be tested regarding the underlying value. This paper will review the main ideas of confidence intervals and hypothesis tests, together with the less well known “prediction intervals,” concentrating on aspects that are often poorly understood. Comparisons will be made between different methods of constructing confidence intervals—exact, asymptotic, bootstrap, and Bayesian—and the difference between prediction intervals and confidence intervals will be explained. For hypothesis testing, multiple testing will be briefly discussed, together with connections between hypothesis testing, prediction intervals, and confidence intervals.

scholarly journals Public Views on Priority Groups for COVID-19 Vaccination: A Survey from Iran

2021 ◽  
Vol In Press (In Press) ◽  
Author(s):  
Najmeh Moradi ◽  
Seyyed Taghi Heydari ◽  
Leila Zarei ◽  
Jalal Arabloo ◽  
Aziz Rezapour ◽  
...  
Keyword(s):  
Online Survey ◽  
Univariate Analysis ◽  
Random Order ◽  
The Elderly ◽  
Iranian Population ◽  
Vulnerable Groups ◽  
Vaccination Programs ◽  
The Family ◽  
Key Factor ◽  
Allocation Strategies

Background: In the initial coronavirus disease 2019 (COVID-19) vaccination program, prioritizing vulnerable groups is inevitable due to limited supply. Currently, most of the allocation strategies are focused on individuals’ characteristics. Objectives: The present study aimed to assess the opinions of Iranian population in specifying high-priority individuals and groups for COVID-19 vaccination. Methods: An online survey was conducted using some popular social media in Iran. The data was collected from Iranian population (878 individuals) aged 18 years and older during the COVID-19 pandemic (2 - 20 May 2020) to investigate their opinions towards vaccine allocation strategies at the family and society levels. In vaccine prioritizing within family three option and in vaccine prioritizing within society, seven population groups were introduced by the respondents in a random order, respectively. To analyze the data, mean rank and univariate analysis was used. Results: Healthcare workers, high-risk patients, and the elderly were the first priority groups for a vaccination with a mean rank of 2.8, 2.8, and 3.8, respectively. The least priority group was policymakers and executive managers (mean rank = 5.75). At the family level, 64% of the respondents introduced one of the family members as the first priority for vaccination, followed by their children (29%) and themselves (7%). No significant relationship was observed between respondents’ characteristics and their prioritization in vaccine prioritizing within society. Conclusions: Although involving public preference in decision-making is a key factor for the success of policies, careful design and implementation of vaccination programs through considering risk-benefit assessment is strongly recommended.

Sign in / Sign up

Export Citation Format

Share Document