Decision curve analysis to evaluate the clinical benefit of prediction models

2021 ◽  
Author(s):  
Andrew J. Vickers ◽  
Ford Holland
Keyword(s):  
Clinical Benefit ◽  
Prediction Models ◽  
Curve Analysis ◽  
Decision Curve Analysis

scholarly journals Scoring system development and validation for prediction choledocholithiasis before open cholecystectomy

2015 ◽  
Vol 143 (11-12) ◽  
pp. 681-687 ◽  
Author(s):  
Tomislav Pejovic ◽  
Miroslav Stojadinovic
Keyword(s):  
Bile Duct ◽  
Clinical Utility ◽  
Clinical Decision Making ◽  
Prediction Models ◽  
Open Cholecystectomy ◽  
Univariate Analysis ◽  
Curve Analysis ◽  
Decision Curve Analysis ◽  
Scoring Model ◽  
Validation Set

Introduction. Accurate precholecystectomy detection of concurrent asymptomatic common bile duct stones (CBDS) is key in the clinical decision-making process. The standard preoperative methods used to diagnose these patients are often not accurate enough. Objective. The aim of the study was to develop a scoring model that would predict CBDS before open cholecystectomy. Methods. We retrospectively collected preoperative (demographic, biochemical, ultrasonographic) and intraoperative (intraoperative cholangiography) data for 313 patients at the department of General Surgery at Gornji Milanovac from 2004 to 2007. The patients were divided into a derivation (213) and a validation set (100). Univariate and multivariate regression analysis was used to determine independent predictors of CBDS. These predictors were used to develop scoring model. Various measures for the assessment of risk prediction models were determined, such as predictive ability, accuracy, the area under the receiver operating characteristic curve (AUC), calibration and clinical utility using decision curve analysis. Results. In a univariate analysis, seven risk factors displayed significant correlation with CBDS. Total bilirubin, alkaline phosphatase and bile duct dilation were identified as independent predictors of choledocholithiasis. The resultant total possible score in the derivation set ranged from 7.6 to 27.9. Scoring model shows good discriminatory ability in the derivation and validation set (AUC 94.3 and 89.9%, respectively), excellent accuracy (95.5%), satisfactory calibration in the derivation set, similar Brier scores and clinical utility in decision curve analysis. Conclusion. Developed scoring model might successfully estimate the presence of choledocholithiasis in patients planned for elective open cholecystectomy.

Utility of PCA3 and TMPRSS2:ERG urinary biomarkers in African American men undergoing prostate biopsy.

2016 ◽  
Vol 34 (2_suppl) ◽  
pp. 126-126
Author(s):  
Allison H. Feibus ◽  
A. Oliver Sartor ◽  
Krishnarao Moparty ◽  
Michael W. Kattan ◽  
Kevin M. Chagin ◽  
...  
Keyword(s):  
Prostate Biopsy ◽  
Prediction Models ◽  
African American Men ◽  
Standard Of Care ◽  
Curve Analysis ◽  
High Grade ◽  
Net Benefit ◽  
Racially Diverse ◽  
Decision Curve Analysis ◽  
History Of

126 Background: To determine the performance characteristics of urinary PCA3 andTMPRSS2:ERG (T2:ERG) in a racially diverse group of men. Methods: Following IRB approval, from 2013-2015, post digital rectal exam (DRE) urine was prospectively collected in patients without known prostate cancer (PCa), prior to biopsy. PCA3 and T2:ERG RNA copies were quantified and normalized to PSA mRNA copies using Progensa assay (Hologic, San Diego, CA). Prediction models for PCa and high-grade PCa were created using standard of care (SOC) variables (age, race, family history of PCa, prior prostate biopsy and abnormal DRE) plus PSA. Decision Curve Analysis was performed to compare the net benefit of using SOC, plus PSA, with the addition of PCA3 and T2:ERG. Results: Of 304 patients, 182 (60%) were AA; 139(46%) were diagnosed with PCa (69% AA). PCA3 and T2:ERG scores were greater in men with PCa, ≥ 3 cores, ≥ 33.3% cores, > 50% involvement of greatest biopsy core and Epstein significant PCa (p-values < 0.04). PCA3 added to the SOC plus PSA model for the detection of any PCa in the overall cohort (0.747 vs 0.677; p < 0.0001), in AA only (0.711 vs 0.638; p = 0.0002) and non-AA (0.781 vs 0.732; p = 0.0016). PCA3 added to the model for the prediction of high-grade PCa for the overall cohort (0.804 vs 0.78; p = 0.0002) and AA only (0.759 vs 0.717; p = 0.0003) but not non-AA. Decision curve analysis demonstrated significant net benefit with the addition of PCA3 compared with SOC plus PSA. For AA, T2:ERG did not improve concordance statistics for the detection any or high-grade PCa. Conclusions: For AA, urinary PCA3 improves the ability to predict the presence of any and high-grade PCa. However for this population, T2:ERG urinary assay does not add significantly to standard detection and risk stratification tools.

scholarly journals Development and validation of models to predict cesarean delivery among low-risk nulliparous women at term: a retrospective study in China

2020 ◽  
Author(s):  
Fangcan Sun ◽  
Bing Han ◽  
Fangfang Wu ◽  
Qianqian Shen ◽  
Minhong Shen ◽  
...  
Keyword(s):  
Logistic Regression ◽  
Cesarean Delivery ◽  
Prediction Models ◽  
External Validation ◽  
Curve Analysis ◽  
Low Risk ◽  
Calibration Plot ◽  
Fetal Sex ◽  
Decision Curve Analysis ◽  
Nulliparous Women

Abstract Background A prediction algorithm to identify women with high risk of an emergency cesarean could help reduce morbidity and mortality associated with labor. The objective of the present study was to derive and validate a simple model to predict intrapartum cesarean delivery for low-risk nulliparous women in Chinese population.Methods We conducted a retrospective cohort study of low-risk nulliparous women with singleton, term, cephalic pregnancies. A predictive model for cesarean delivery was derived using univariate and multivariable logistic regression from the hospital of the First Affiliated Hospital of Soochow University. External validation of the prediction model was then performed using the data from Sihong county People’s Hospital. A new nomogram was established based on the development cohort to predict the cesarean. The ROC curve, calibration plot and decision curve analysis were used to assess the predictive performance.Results The intrapartum cesarean delivery rates in the development cohort and the external validation cohort were 8.79% (576/6,551) and 7.82% (599/7,657). Multivariable logistic regression analysis showed that maternal age, height, BMI, weight gained during pregnancy, gestational age, induction method, meconium-stained amniotic fluid and neonatal sex were independent factors affecting cesarean outcome. We had established two prediction models according to fetal sex was involved or not. The AUC was 0.782 and 0.774, respectively. The two prediction models were well-calibrated with Hosmer-Lemeshow test P=0.263 and P=0.817, respectively. Decision curve analysis demonstrated that two models had clinical application value, and they provided greatest net benefit between threshold probabilities of 4% to 60%. And internal validation using Bootstrap method demonstrated similar discriminatory ability. We external validated the model involving fetal sex, for which the AUC was 0.775, while the slope and intercept of the calibration plot were 0.979 and 0.004, respectively. On the external validation set, another model had an AUC of 0.775 and a calibration slope of 1.007. The online web server was constructed based on the nomogram for convenient clinical use.Conclusions Both two models established by these factors have good prediction efficiency and high accuracy, which can provide the reference for clinicians to guide pregnant women to choose an appropriate delivery mode.

Development And Validation Of Models To Predict Cesarean Delivery Among Low-Risk Nulliparous Women At Term: A Retrospective Study In China

2020 ◽  
Author(s):  
Fangcan Sun ◽  
Bing Han ◽  
Fangfang Wu ◽  
Qianqian Shen ◽  
Minhong Shen ◽  
...  
Keyword(s):  
Logistic Regression ◽  
Regression Analysis ◽  
Cesarean Delivery ◽  
Prediction Models ◽  
External Validation ◽  
Curve Analysis ◽  
Low Risk ◽  
Multivariable Logistic Regression Analysis ◽  
Decision Curve Analysis ◽  
Nulliparous Women

Abstract Background: Cesarean delivery after failure of trial of labor is associated with adverse maternal and perinatal outcomes. A prediction algorithm to identify women with high risk of an emergency cesarean could help reduce morbidity and mortality associated with labor. The objective of the present study was to derive and validate a simple model to predict cesarean delivery for low-risk nulliparous women in Chinese population.Methods: This retrospective study analyzed the low-risk nulliparous women with singleton cephalic full-term fetus delivered in two medical centers. After the clinical data of the women who delivered at the tertiary referral center (n=6 551) was collected and was used univariate and multivariable logistic regression analysis, the prediction model was fitted. We performed external validation using data from nulliparous who delivered from another hospital(secondary referral center, n=7 657). A new nomogram was established based on the development cohort to predict the cesarean. The ROC curve, calibration plot and decision curve analysis were used to assess the predictive performance. Results: The cesarean delivery rates in the development cohort and the external validation cohort were 8.79% (576/6 551) and 7.82% (599/7 657). Multivariable logistic regression analysis showed that maternal age, height, BMI, weight gained during pregnancy, gestational age, induction method, meconium-stained amniotic fluid and neonatal sex were independent factors affecting cesarean outcome. Because sex of the fetuses were unknown until they born(China's Fertility Policy), we established two prediction models according to fetal sex was involved or not. The AUC was 0.782 and 0.774, respectively. The Hosmer-Lemeshow goodness-of-fit test showed that these two models fitted well. Decision curve analysis demonstrated that the models were clinically useful. And internal validation using Bootstrap method showed that these prediction models perform well. On the external validation set, the AUC were 0.775 and 0.775, respectively. The calibration plots for the probability of cesarean showed a good correlation. The online web server was constructed based on the nomogram for convenient clinical use.Conclusions: Both two models established by these factors have good prediction efficiency and high accuracy, which can provide the reference for clinicians to guide pregnant women to choose an appropriate delivery mode.

Moving beyond AUC: decision curve analysis for quantifying net benefit of risk prediction models

2021 ◽  
pp. 2101186
Author(s):  
Mohsen Sadatsafavi ◽  
Amin Adibi ◽  
Milo Puhan ◽  
Andrea Gershon ◽  
Shawn D. Aaron ◽  
...  
Keyword(s):  
Risk Prediction ◽  
Prediction Models ◽  
Curve Analysis ◽  
Net Benefit ◽  
Decision Curve Analysis ◽  
Risk Prediction Models

scholarly journals A simple, step-by-step guide to interpreting decision curve analysis

2019 ◽  
Vol 3 (1) ◽  
Author(s):  
Andrew J. Vickers ◽  
Ben van Calster ◽  
Ewout W. Steyerberg
Keyword(s):  
Diagnostic Tests ◽  
Prediction Models ◽  
Curve Analysis ◽  
Clinical Use ◽  
Decision Curve Analysis ◽  
Simple Step ◽  
Publication Decision

Abstract Background Decision curve analysis is a method to evaluate prediction models and diagnostic tests that was introduced in a 2006 publication. Decision curves are now commonly reported in the literature, but there remains widespread misunderstanding of and confusion about what they mean. Summary of commentary In this paper, we present a didactic, step-by-step introduction to interpreting a decision curve analysis and answer some common questions about the method. We argue that many of the difficulties with interpreting decision curves can be solved by relabeling the y-axis as “benefit” and the x-axis as “preference.” A model or test can be recommended for clinical use if it has the highest level of benefit across a range of clinically reasonable preferences. Conclusion Decision curves are readily interpretable if readers and authors follow a few simple guidelines.

scholarly journals Prediction of vaginal birth after cesarean delivery in Southeast China: a retrospective cohort study

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Hua-Le Zhang ◽  
Liang-Hui Zheng ◽  
Li-Chun Cheng ◽  
Zhao-Dong Liu ◽  
Lu Yu ◽  
...  
Keyword(s):  
Cesarean Section ◽  
Pregnant Women ◽  
Success Rate ◽  
Gestational Age ◽  
Prediction Models ◽  
Curve Analysis ◽  
Vaginal Birth ◽  
Lasso Regression ◽  
Vaginal Birth After Cesarean ◽  
Decision Curve Analysis

Abstract Background We aimed to develop and validate a nomogram for effective prediction of vaginal birth after cesarean (VBAC) and guide future clinical application. Methods We retrospectively analyzed data from hospitalized pregnant women who underwent trial of labor after cesarean (TOLAC), at the Fujian Provincial Maternity and Children’s Hospital, between October 2015 and October 2017. Briefly, we included singleton pregnant women, at a gestational age above 37 weeks who underwent a primary cesarean section, in the study. We then extracted their sociodemographic data and clinical characteristics, and randomly divided the samples into training and validation sets. We employed the least absolute shrinkage and selection operator (LASSO) regression to select variables and construct VBAC success rate in the training set. Thereafter, we validated the nomogram using the concordance index (C-index), decision curve analysis (DCA), and calibration curves. Finally, we adopted the Grobman’s model to perform comparisons with published VBAC prediction models. Results Among the 708 pregnant women included according to inclusion criteria, 586 (82.77%) patients were successfully for VBAC. Multivariate logistic regression models revealed that maternal height (OR, 1.11; 95% CI, 1.04 to 1.19), maternal BMI at delivery (OR, 0.89; 95% CI, 0.79 to 1.00), fundal height (OR, 0.71; 95% CI, 0.58 to 0.88), cervix Bishop score (OR, 3.27; 95% CI, 2.49 to 4.45), maternal age at delivery (OR, 0.90; 95% CI, 0.82 to 0.98), gestational age (OR, 0.33; 95% CI, 0.17 to 0.62) and history of vaginal delivery (OR, 2.92; 95% CI, 1.42 to 6.48) were independently associated with successful VBAC. The constructed predictive model showed better discrimination than that from the Grobman’s model in the validation series (c-index 0.906 VS 0.694, respectively). On the other hand, decision curve analysis revealed that the new model had better clinical net benefits than the Grobman’s model. Conclusions VBAC will aid in reducing the rate of cesarean sections in China. In clinical practice, the TOLAC prediction model will help improve VBAC’s success rate, owing to its contribution to reducing secondary cesarean section.

Sign in / Sign up

Export Citation Format

Share Document