Confidence intervals for differences between volumes under receiver operating characteristic surfaces (VUS) and generalized Youden indices (GYIs)

2017 ◽  
Vol 27 (3) ◽  
pp. 675-688 ◽  
Author(s):  
Jingjing Yin ◽  
Christos T Nakas ◽  
Lili Tian ◽  
Benjamin Reiser
Keyword(s):  
Cognitive Function ◽  
Receiver Operating Characteristic ◽  
Confidence Intervals ◽  
Operating Characteristic ◽  
Youden Index ◽  
Characteristic Surface ◽  
Classification Problems ◽  
New Methods ◽  
Using Data ◽  
Receiver Operating

This article explores both existing and new methods for the construction of confidence intervals for differences of indices of diagnostic accuracy of competing pairs of biomarkers in three-class classification problems and fills the methodological gaps for both parametric and non-parametric approaches in the receiver operating characteristic surface framework. The most widely used such indices are the volume under the receiver operating characteristic surface and the generalized Youden index. We describe implementation of all methods and offer insight regarding the appropriateness of their use through a large simulation study with different distributional and sample size scenarios. Methods are illustrated using data from the Alzheimer's Disease Neuroimaging Initiative study, where assessment of cognitive function naturally results in a three-class classification setting.

Estimation of the volume under the receiver-operating characteristic surface adjusting for non-ignorable verification bias

2018 ◽  
Vol 27 (3) ◽  
pp. 715-739 ◽  
Author(s):  
Ying Zhang ◽  
Todd A Alonzo ◽  
Keyword(s):  
Receiver Operating Characteristic ◽  
Diagnostic Test ◽  
Operating Characteristic ◽  
Disease Status ◽  
Characteristic Surface ◽  
Finite Sample ◽  
Classification Problems ◽  
Verification Bias ◽  
Inverse Probability Weighted ◽  
Receiver Operating

The receiver-operating characteristic surface is frequently used for presenting the accuracy of a diagnostic test for three-category classification problems. One common problem that can complicate the estimation of the volume under receiver-operating characteristic surface is that not all subjects receive the verification of the true disease status. Estimation based only on data from subjects with verified disease status may be biased, which is referred to as verification bias. In this article, we propose new verification bias correction methods to estimate the volume under receiver-operating characteristic surface for a continuous diagnostic test. We assume the verification process is missing not at random, which means the missingness might be related to unobserved clinical characteristics. Three classes of estimators are proposed, namely, inverse probability weighted, imputation-based, and doubly robust estimators. A jackknife estimator of variance is derived for all the proposed volume under receiver-operating characteristic surface estimators. The finite sample properties of the new estimators are examined via simulation studies. We illustrate our methods with data collected from Alzheimer’s disease research.

scholarly journals Accuracy and adequacy of waist circumference cut-off points currently recommended in Brazilian adults

2013 ◽  
Vol 17 (4) ◽  
pp. 861-869 ◽  
Author(s):  
Carolina Avila Vianna ◽  
Rogério da Silva Linhares ◽  
Renata Moraes Bielemann ◽  
Eduardo Coelho Machado ◽  
David Alejandro González-Chica ◽  
...  
Keyword(s):  
Cardiovascular Risk ◽  
Waist Circumference ◽  
Receiver Operating Characteristic ◽  
Sensitivity And Specificity ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Predictive Ability ◽  
Youden Index ◽  
Southern Brazil ◽  
Receiver Operating

AbstractObjectiveTo evaluate the adequacy and accuracy of cut-off values currently recommended by the WHO for assessment of cardiovascular risk in southern Brazil.DesignPopulation-based study aimed at determining the predictive ability of waist circumference for cardiovascular risk based on the use of previous medical diagnosis for hypertension, diabetes mellitus and/or dyslipidaemia. Descriptive analysis was used for the adequacy of current cut-off values of waist circumference, receiver operating characteristic curves were constructed and the most accurate criteria according to the Youden index and points of optimal sensitivity and specificity were identified.SettingPelotas, southern Brazil.SubjectsIndividuals (n2112) aged ≥20 years living in the city were selected by multistage sampling, since these individuals did not report the presence of previous myocardial infarction, angina pectoris or stroke.ResultsThe cut-off values currently recommended by WHO were more appropriate in men than women, with overestimation of cardiovascular risk in women. The area under the receiver operating characteristic curve showed moderate predictive ability of waist circumference in men (0·74, 95 % CI 0·71, 0·76) and women (0·75, 95 % CI 0·73, 0·77). The method of optimal sensitivity and specificity showed better performance in assessing the accuracy, identifying the values of 95 cm in men and 87 cm in women as the best cut-off values of waist circumference to assess cardiovascular risk.ConclusionsThe cut-off values currently recommended for waist circumference are not suitable for women. Longitudinal studies should be conducted to evaluate the consistency of the findings.

scholarly journals Receiver operating characteristic (ROC) movies, universal ROC (UROC) curves, and coefficient of predictive ability (CPA)

2021 ◽  
Author(s):  
Tilmann Gneiting ◽  
Eva-Maria Walz
Keyword(s):  
Receiver Operating Characteristic ◽  
Roc Analysis ◽  
Operating Characteristic ◽  
Numerical Models ◽  
Binary Classification ◽  
Weather Prediction ◽  
Area Under The Curve ◽  
Predictive Ability ◽  
Classification Problems ◽  
Receiver Operating

AbstractThroughout science and technology, receiver operating characteristic (ROC) curves and associated area under the curve ($$\mathrm{AUC}$$ AUC ) measures constitute powerful tools for assessing the predictive abilities of features, markers and tests in binary classification problems. Despite its immense popularity, ROC analysis has been subject to a fundamental restriction, in that it applies to dichotomous (yes or no) outcomes only. Here we introduce ROC movies and universal ROC (UROC) curves that apply to just any linearly ordered outcome, along with an associated coefficient of predictive ability ($${\mathrm{CPA}}$$ CPA ) measure. $${\mathrm{CPA}}$$ CPA equals the area under the UROC curve, and admits appealing interpretations in terms of probabilities and rank based covariances. For binary outcomes $${\mathrm{CPA}}$$ CPA equals $$\mathrm{AUC}$$ AUC , and for pairwise distinct outcomes $${\mathrm{CPA}}$$ CPA relates linearly to Spearman’s coefficient, in the same way that the C index relates linearly to Kendall’s coefficient. ROC movies, UROC curves, and $${\mathrm{CPA}}$$ CPA nest and generalize the tools of classical ROC analysis, and are bound to supersede them in a wealth of applications. Their usage is illustrated in data examples from biomedicine and meteorology, where rank based measures yield new insights in the WeatherBench comparison of the predictive performance of convolutional neural networks and physical-numerical models for weather prediction.

Sign in / Sign up

Export Citation Format

Share Document