Moving beyond AUC: decision curve analysis for quantifying net benefit of risk prediction models

126 Background: To determine the performance characteristics of urinary PCA3 andTMPRSS2:ERG (T2:ERG) in a racially diverse group of men. Methods: Following IRB approval, from 2013-2015, post digital rectal exam (DRE) urine was prospectively collected in patients without known prostate cancer (PCa), prior to biopsy. PCA3 and T2:ERG RNA copies were quantified and normalized to PSA mRNA copies using Progensa assay (Hologic, San Diego, CA). Prediction models for PCa and high-grade PCa were created using standard of care (SOC) variables (age, race, family history of PCa, prior prostate biopsy and abnormal DRE) plus PSA. Decision Curve Analysis was performed to compare the net benefit of using SOC, plus PSA, with the addition of PCA3 and T2:ERG. Results: Of 304 patients, 182 (60%) were AA; 139(46%) were diagnosed with PCa (69% AA). PCA3 and T2:ERG scores were greater in men with PCa, ≥ 3 cores, ≥ 33.3% cores, > 50% involvement of greatest biopsy core and Epstein significant PCa (p-values < 0.04). PCA3 added to the SOC plus PSA model for the detection of any PCa in the overall cohort (0.747 vs 0.677; p < 0.0001), in AA only (0.711 vs 0.638; p = 0.0002) and non-AA (0.781 vs 0.732; p = 0.0016). PCA3 added to the model for the prediction of high-grade PCa for the overall cohort (0.804 vs 0.78; p = 0.0002) and AA only (0.759 vs 0.717; p = 0.0003) but not non-AA. Decision curve analysis demonstrated significant net benefit with the addition of PCA3 compared with SOC plus PSA. For AA, T2:ERG did not improve concordance statistics for the detection any or high-grade PCa. Conclusions: For AA, urinary PCA3 improves the ability to predict the presence of any and high-grade PCa. However for this population, T2:ERG urinary assay does not add significantly to standard detection and risk stratification tools.

Download Full-text

Impact of Tobacco Dependence in Risk Prediction Models for Lung Cancer Diagnoses and Deaths

JNCI Cancer Spectrum ◽

10.1093/jncics/pkz014 ◽

2019 ◽

Vol 3 (2) ◽

Cited By ~ 1

Author(s):

Ralph C Ward ◽

Nichole T Tanner ◽

Gerard A Silvestri ◽

Mulugeta Gebregziabher

Keyword(s):

Lung Cancer ◽

Nicotine Dependence ◽

Risk Prediction ◽

Cancer Incidence ◽

Prediction Models ◽

Tobacco Dependence ◽

Lung Cancer Incidence ◽

Net Benefit ◽

Risk Prediction Models ◽

Dependence Measures

Abstract Background Stronger nicotine dependence is associated with greater lung cancer incidence and lung cancer death. This study investigates whether including nicotine dependence in risk prediction models for lung cancer incidence and mortality provides any important clinical benefits. Methods Smoking data were used from 14 123 participants in the American College of Radiology Imaging Network arm of the National Lung Screening trial. We added nicotine dependence as the primary exposure in two published lung cancer risk prediction models (Katki-Gu or PLCO-m2012) and compared four results: with no tobacco-dependence measure, with time to first cigarette, with heaviness of smoking index, and with Fagestrom test for nicotine dependence. We used a cross-validation method based on leave-one-out and compared performance using likelihood ratio tests (LRT), area under the curve, concordance, sensitivity and specificity for 1% and 2% risk thresholds, and net benefit statistics. Statistical tests were two-sided. Results All LRT results were statistically significant (P ≤ .0001), whereas other tests were not, except that specificity statistically significantly improved (P < .0001). Because the LRT is asymptotically more powerful for testing for prediction gain, we conclude that both models were improved on a statistical level by adding dependence measures. The other performance statistics generally indicated that such gains were likely very small. Net benefit analysis confirmed there was no apparent clinical benefit for including dependence measures. Conclusions Although inclusion of dependence measures may not provide a clinical benefit when added to risk prediction models, nicotine-dependence measures should nonetheless be an integral tool for patient counseling and for encouraging tobacco cessation.

Download Full-text

Faculty Opinions recommendation of Calibration of risk prediction models: impact on decision-analytic performance.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.718545520.793499575 ◽

2014 ◽

Author(s):

Ewout Steyerberg

Keyword(s):

Risk Prediction ◽

Prediction Models ◽

Risk Prediction Models

Download Full-text

Faculty Opinions recommendation of Risk prediction models for selection of lung cancer screening candidates: A retrospective validation study.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.727476053.793541151 ◽

2018 ◽

Author(s):

Robert Keith ◽

Melissa New

Keyword(s):

Lung Cancer ◽

Cancer Screening ◽

Risk Prediction ◽

Validation Study ◽

Prediction Models ◽

Lung Cancer Screening ◽

Risk Prediction Models ◽

Selection Of

Download Full-text

Patient-Reported Symptoms Improve Performance of Risk Prediction Models for Emergency Department Visits Among Patients With Cancer: A Population-Wide Study in Ontario Using Administrative Data

Journal of Pain and Symptom Management ◽

10.1016/j.jpainsymman.2019.07.007 ◽

2019 ◽

Vol 58 (5) ◽

pp. 745-755 ◽

Cited By ~ 6

Author(s):

Rinku Sutradhar ◽

Mehdi Rostami ◽

Lisa Barbera

Keyword(s):

Emergency Department ◽

Risk Prediction ◽

Administrative Data ◽

Prediction Models ◽

Emergency Department Visits ◽

Improve Performance ◽

Risk Prediction Models ◽

Patients With Cancer ◽

Patient Reported

Download Full-text

Scoring system development and validation for prediction choledocholithiasis before open cholecystectomy

Srpski arhiv za celokupno lekarstvo ◽

10.2298/sarh1512681p ◽

2015 ◽

Vol 143 (11-12) ◽

pp. 681-687 ◽

Cited By ~ 1

Author(s):

Tomislav Pejovic ◽

Miroslav Stojadinovic

Keyword(s):

Bile Duct ◽

Clinical Utility ◽

Clinical Decision Making ◽

Prediction Models ◽

Open Cholecystectomy ◽

Univariate Analysis ◽

Curve Analysis ◽

Decision Curve Analysis ◽

Scoring Model ◽

Validation Set

Introduction. Accurate precholecystectomy detection of concurrent asymptomatic common bile duct stones (CBDS) is key in the clinical decision-making process. The standard preoperative methods used to diagnose these patients are often not accurate enough. Objective. The aim of the study was to develop a scoring model that would predict CBDS before open cholecystectomy. Methods. We retrospectively collected preoperative (demographic, biochemical, ultrasonographic) and intraoperative (intraoperative cholangiography) data for 313 patients at the department of General Surgery at Gornji Milanovac from 2004 to 2007. The patients were divided into a derivation (213) and a validation set (100). Univariate and multivariate regression analysis was used to determine independent predictors of CBDS. These predictors were used to develop scoring model. Various measures for the assessment of risk prediction models were determined, such as predictive ability, accuracy, the area under the receiver operating characteristic curve (AUC), calibration and clinical utility using decision curve analysis. Results. In a univariate analysis, seven risk factors displayed significant correlation with CBDS. Total bilirubin, alkaline phosphatase and bile duct dilation were identified as independent predictors of choledocholithiasis. The resultant total possible score in the derivation set ranged from 7.6 to 27.9. Scoring model shows good discriminatory ability in the derivation and validation set (AUC 94.3 and 89.9%, respectively), excellent accuracy (95.5%), satisfactory calibration in the derivation set, similar Brier scores and clinical utility in decision curve analysis. Conclusion. Developed scoring model might successfully estimate the presence of choledocholithiasis in patients planned for elective open cholecystectomy.

Download Full-text

Rare Germline Pathogenic Variants Identified by Multigene Panel Testing and the Risk of Aggressive Prostate Cancer

Cancers ◽

10.3390/cancers13071495 ◽

2021 ◽

Vol 13 (7) ◽

pp. 1495

Author(s):

Tú Nguyen-Dumont ◽

James G. Dowty ◽

Robert J. MacInnis ◽

Jason A. Steen ◽

Moeen Riaz ◽

...

Keyword(s):

Prostate Cancer ◽

Risk Prediction ◽

Prediction Models ◽

Risk Prediction Models ◽

Aggressive Prostate Cancer ◽

Panel Testing ◽

Pathogenic Variants ◽

Increased Risk ◽

Predisposition Genes ◽

Rare Genetic Variants

While gene panel sequencing is becoming widely used for cancer risk prediction, its clinical utility with respect to predicting aggressive prostate cancer (PrCa) is limited by our current understanding of the genetic risk factors associated with predisposition to this potentially lethal disease phenotype. This study included 837 men diagnosed with aggressive PrCa and 7261 controls (unaffected men and men who did not meet criteria for aggressive PrCa). Rare germline pathogenic variants (including likely pathogenic variants) were identified by targeted sequencing of 26 known or putative cancer predisposition genes. We found that 85 (10%) men with aggressive PrCa and 265 (4%) controls carried a pathogenic variant (p < 0.0001). Aggressive PrCa odds ratios (ORs) were estimated using unconditional logistic regression. Increased risk of aggressive PrCa (OR (95% confidence interval)) was identified for pathogenic variants in BRCA2 (5.8 (2.7–12.4)), BRCA1 (5.5 (1.8–16.6)), and ATM (3.8 (1.6–9.1)). Our study provides further evidence that rare germline pathogenic variants in these genes are associated with increased risk of this aggressive, clinically relevant subset of PrCa. These rare genetic variants could be incorporated into risk prediction models to improve their precision to identify men at highest risk of aggressive prostate cancer and be used to identify men with newly diagnosed prostate cancer who require urgent treatment.

Download Full-text

Neurocognitive measures of self-blame and risk prediction models of recurrence in major depressive disorder

Biological Psychiatry Cognitive Neuroscience and Neuroimaging ◽

10.1016/j.bpsc.2021.06.010 ◽

2021 ◽

Author(s):

Andrew J. Lawrence ◽

Daniel Stahl ◽

Suqian Duan ◽

Diede Fennema ◽

Tanja Jaeckle ◽

...

Keyword(s):

Major Depressive Disorder ◽

Depressive Disorder ◽

Risk Prediction ◽

Prediction Models ◽

Major Depressive ◽

Risk Prediction Models ◽

Self Blame ◽

Neurocognitive Measures

Download Full-text

Risk Prediction of Barrett’s Esophagus in a Taiwanese Health Examination Center Based on Regression Models

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18105332 ◽

2021 ◽

Vol 18 (10) ◽

pp. 5332

Author(s):

Po-Hsiang Lin ◽

Jer-Guang Hsieh ◽

Hsien-Chung Yu ◽

Jyh-Horng Jeng ◽

Chiao-Lin Hsu ◽

...

Keyword(s):

Barrett’S Esophagus ◽

Risk Prediction ◽

Barrett's Esophagus ◽

Gastrointestinal Endoscopy ◽

Prediction Models ◽

Upper Gastrointestinal Endoscopy ◽

Health Examination ◽

Endoscopic Screening ◽

Risk Prediction Models ◽

Upper Gastrointestinal

Determining the target population for the screening of Barrett’s esophagus (BE), a precancerous condition of esophageal adenocarcinoma, remains a challenge in Asia. The aim of our study was to develop risk prediction models for BE using logistic regression (LR) and artificial neural network (ANN) methods. Their predictive performances were compared. We retrospectively analyzed 9646 adults aged ≥20 years undergoing upper gastrointestinal endoscopy at a health examinations center in Taiwan. Evaluated by using 10-fold cross-validation, both models exhibited good discriminative power, with comparable area under curve (AUC) for the LR and ANN models (Both AUC were 0.702). Our risk prediction models for BE were developed from individuals with or without clinical indications of upper gastrointestinal endoscopy. The models have the potential to serve as a practical tool for identifying high-risk individuals of BE among the general population for endoscopic screening.

Download Full-text