Extension of adaptive alpha allocation methods for strong control of the family-wise error rate

2012 ◽  
Vol 32 (2) ◽  
pp. 181-195 ◽  
Author(s):  
Haihong Li ◽  
Abdul J. Sankoh ◽  
Ralph B. D'Agostino
Keyword(s):  
Error Rate ◽  
Family Wise Error Rate ◽  
The Family ◽  
Strong Control ◽  
Allocation Methods

scholarly journals Multiplicity- and dependency-adjusted p-values for control of the family-wise error rate

2016 ◽  
Vol 111 ◽  
pp. 32-40 ◽  
Author(s):  
Jens Stange ◽  
Thorsten Dickhaus ◽  
Arcadi Navarro ◽  
Daniel Schunk
Keyword(s):  
Error Rate ◽  
P Values ◽  
Family Wise Error Rate ◽  
The Family

Optimized multi-stage designs controlling the false discovery or the family-wise error rate

2008 ◽  
Vol 27 (21) ◽  
pp. 4145-4160 ◽  
Author(s):  
Sonja Zehetmayer ◽  
Peter Bauer ◽  
Martin Posch
Keyword(s):  
Error Rate ◽  
Family Wise Error Rate ◽  
False Discovery ◽  
Multi Stage ◽  
The Family

scholarly journals Evaluation of multiple prediction models: A novel view on model selection and performance assessment

2019 ◽  
Vol 29 (6) ◽  
pp. 1728-1745 ◽  
Author(s):  
Max Westphal ◽  
Werner Brannath
Keyword(s):  
Model Selection ◽  
Performance Assessment ◽  
Error Rate ◽  
Prediction Models ◽  
Model Performance ◽  
Complex Data ◽  
Final Model ◽  
Family Wise Error Rate ◽  
The Family ◽  
And Performance

Model selection and performance assessment for prediction models are important tasks in machine learning, e.g. for the development of medical diagnosis or prognosis rules based on complex data. A common approach is to select the best model via cross-validation and to evaluate this final model on an independent dataset. In this work, we propose to instead evaluate several models simultaneously. These may result from varied hyperparameters or completely different learning algorithms. Our main goal is to increase the probability to correctly identify a model that performs sufficiently well. In this case, adjusting for multiplicity is necessary in the evaluation stage to avoid an inflation of the family wise error rate. We apply the so-called maxT-approach which is based on the joint distribution of test statistics and suitable to (approximately) control the family-wise error rate for a wide variety of performance measures. We conclude that evaluating only a single final model is suboptimal. Instead, several promising models should be evaluated simultaneously, e.g. all models within one standard error of the best validation model. This strategy has proven to increase the probability to correctly identify a good model as well as the final model performance in extensive simulation studies.

scholarly journals SDMinP: a program to control the family wise error rate using step-down minP adjusted P-values

2005 ◽  
Vol 21 (14) ◽  
pp. 3183-3184 ◽  
Author(s):  
M. Obreiter ◽  
C. Fischer ◽  
J. Chang-Claude ◽  
L. Beckmann
Keyword(s):  
Error Rate ◽  
P Values ◽  
Family Wise Error Rate ◽  
The Family ◽  
Step Down
Sign in / Sign up

Export Citation Format

Share Document