scholarly journals Bi-Centric Independent Validation of Outcome Prediction after Radioembolization of Primary and Secondary Liver Cancer

2021 ◽  
Vol 10 (16) ◽  
pp. 3668
Author(s):  
Matthias Philipp Fabritius ◽  
Max Seidensticker ◽  
Johannes Rueckel ◽  
Constanze Heinze ◽  
Maciej Pech ◽  
...  
Keyword(s):  
Patient Selection ◽  
Outcome Prediction ◽  
Prediction Models ◽  
External Validation ◽  
Cox Proportional Hazards ◽  
Brier Score ◽  
Survival Prediction ◽  
Liver Malignancies ◽  
Prognostic Performance ◽  
Hepatic Reserve

Background: Yttrium-90 radioembolization (RE) plays an important role in the treatment of liver malignancies. Optimal patient selection is crucial for an effective and safe treatment. In this study, we aim to validate the prognostic performance of a previously established random survival forest (RSF) with an external validation cohort from a different national center. Furthermore, we compare outcome prediction models with different established metrics. Methods: A previously established RSF model, trained on a consecutive cohort of 366 patients who had received RE due to primary or secondary liver tumor at a national center (center 1), was used to predict the outcome of an independent consecutive cohort of 202 patients from a different national center (center 2) and vice versa. Prognostic performance was evaluated using the concordance index (C-index) and the integrated Brier score (IBS). The prognostic importance of designated baseline parameters was measured with the minimal depth concept, and the influence on the predicted outcome was analyzed with accumulated local effects plots. RSF values were compared to conventional cox proportional hazards models in terms of C-index and IBS. Results: The established RSF model achieved a C-index of 0.67 for center 2, comparable to the results obtained for center 1, which it was trained on (0.66). The RSF model trained on center 2 achieved a C-index of 0.68 on center 2 data and 0.66 on center 1 data. CPH models showed comparable results on both cohorts, with C-index ranging from 0.68 to 0.72. IBS validation showed more differentiated results depending on which cohort was trained on and which cohort was predicted (range: 0.08 to 0.20). Baseline cholinesterase was the most important variable for survival prediction. Conclusion: The previously developed predictive RSF model was successfully validated with an independent external cohort. C-index and IBS are suitable metrics to compare outcome prediction models, with IBS showing more differentiated results. The findings corroborate that survival after RE is critically determined by functional hepatic reserve and thus baseline liver function should play a key role in patient selection.

scholarly journals Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques

2020 ◽  
Author(s):  
Georgios Kantidakis ◽  
Hein Putter ◽  
Carlo Lancia ◽  
Jacob de Boer ◽  
Andries E Braat ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Liver Transplantation ◽  
Prediction Models ◽  
Machine Learning Techniques ◽  
Brier Score ◽  
Survival Prediction ◽  
Cox Models ◽  
Learning Techniques ◽  
Random Survival Forest

Abstract Background: Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians.Methods: In this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques.Results: Well-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years.Conclusion: In this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables.

scholarly journals Progression-Free Survival Prediction in Patients with Nasopharyngeal Carcinoma after Intensity-Modulated Radiotherapy: Machine Learning versus Traditional Statistics

2021 ◽  
Vol 11 (8) ◽  
pp. 787
Author(s):  
Ronald Wihal Oei ◽  
Yingchen Lyu ◽  
Lulu Ye ◽  
Fangfang Kong ◽  
Chengrun Du ◽  
...  
Keyword(s):  
Machine Learning ◽  
Nasopharyngeal Carcinoma ◽  
Proportional Hazards ◽  
Progression Free Survival ◽  
Cox Proportional Hazards ◽  
Brier Score ◽  
Rank Test ◽  
Survival Prediction ◽  
Free Survival ◽  
Log Rank Test

Background: The Cox proportional hazards (CPH) model is the most commonly used statistical method for nasopharyngeal carcinoma (NPC) prognostication. Recently, machine learning (ML) models are increasingly adopted for this purpose. However, only a few studies have compared the performances between CPH and ML models. This study aimed at comparing CPH with two state-of-the-art ML algorithms, namely, conditional survival forest (CSF) and DeepSurv for disease progression prediction in NPC. Methods: From January 2010 to March 2013, 412 eligible NPC patients were reviewed. The entire dataset was split into training cohort and testing cohort in a ratio of 90%:10%. Ten features from patient-related, disease-related, and treatment-related data were used to train the models for progression-free survival (PFS) prediction. The model performance was compared using the concordance index (c-index), Brier score, and log-rank test based on the risk stratification results. Results: DeepSurv (c-index = 0.68, Brier score = 0.13, log-rank test p = 0.02) achieved the best performance compared to CSF (c-index = 0.63, Brier score = 0.14, log-rank test p = 0.38) and CPH (c-index = 0.57, Brier score = 0.15, log-rank test p = 0.81). Conclusions: Both CSF and DeepSurv outperformed CPH in our relatively small dataset. ML-based survival prediction may guide physicians in choosing the most suitable treatment strategy for NPC patients.

scholarly journals Is it time to use machine learning survival algorithms for survival and risk factors prediction instead of Cox proportional hazard regression? A comparative population-based study

2021 ◽  
Author(s):  
Sara Morsy ◽  
Truong Hong Hieu ◽  
Abdelrahman M Makram ◽  
Osama Gamal Hassan ◽  
Nguyen Tran Minh Duc ◽  
...  
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
Survival Duration ◽  
Brier Score ◽  
Survival Prediction ◽  
Proportional Hazard ◽  
Cancer Data ◽  
Cox Proportional Hazard ◽  
Short And Long Term

Purpose Applying machine learning in medical statistics offers more accurate prediction models. In this paper, we aimed to compare the performance of the Cox Proportional Hazard model (CPH), Classification and Regression Trees (CART), and Random Survival Forest (RSF) in short-, and long-term prediction in glioblastoma patients. Methods We extracted glioblastoma cancer data from the Surveillance, Epidemiology, and End Results database (SEER). We used the CPH, CART, and RSF for the prediction of 1- to 10-year survival probabilities. The Brier Score for each duration was calculated, and the model with the least score was considered the most accurate. Results The cohort included 26473 glioblastoma patients divided into two groups: training (n = 18538) and validation set (n = 7935). The average survival duration was seven months. For the short- and long-term predictions, RSF was the best algorithm followed by CPH and CART. Conclusion For big data, RSF was found to have the highest accuracy and best performance. Using an accurate statistical model for survival prediction and prognostic factors determination will help the care of cancer patients. However, more developments of the R packages are needed to allow more illustrations of the effect of each covariate on the survival probability.

Novel perioperative parameters-based nomograms for survival outcomes in upper tract urothelial carcinoma after radical nephroureterectomy.

2019 ◽  
Vol 37 (7_suppl) ◽  
pp. 414-414
Author(s):  
Ping Tan ◽  
Lu Yang ◽  
Hang Xu ◽  
Qiang Wei
Keyword(s):  
Lymph Node ◽  
Lymph Node Metastasis ◽  
Prediction Models ◽  
Validation Cohort ◽  
Fibrinogen Level ◽  
External Validation ◽  
Cox Proportional Hazards ◽  
Radical Nephroureterectomy ◽  
Cancer Specific Survival ◽  
Node Metastasis

414 Background: Recently, several postoperative nomograms for cancer-specific survival (CSS) after radical nephroureterectomy (RNU) were proposed, while they did not incorporate the same variables; meanwhile, many preoperative blood-based parameters, which were recently reported to be related to survival, were not included in their models. In addition, no nomogram for overall survival (OS) was available to date. Methods: The full data of 716 patients were available. The whole cohort was randomly divided into two cohorts: the training cohort for developing the nomograms (n = 508) and the validation cohort for validating the models (n = 208). Univariate and multivariate Cox proportional hazards regression models were used for establishing the prediction models. The discriminative accuracy of nomograms were measured by Harrell’s concordance index (C-index). The clinical usefulness and net benefit of the predictive models were estimated and visualized by using Decision curve analyses (DCA). Results: The median follow-up time was 42.0 months (IQR: 18.0-76.0). For CSS, tumor size, grade and pT stage, lymph node metastasis, NLR, PLR and fibrinogen level were identified as independent risk factors in the final model; while tumor grade and pT stage, lymph node metastasis, PLR, Cys-C and fibrinogen level were identified as independent predictors for OS model. The C-index for CSS prediction was 0.82 (95%CI: 0.79-0.85), and the OS nomogram model had an accuracy of 0.83 (95%CI: 0.80-0.86). The results of bootstrapping showed no deviation from the ideal. The calibration plots for the probability of CSS and OS at 3 or 5-year after RNU showed a favorable agreement between the prediction by the nomograms and actual observation. In the external validation cohort, the C-indexes of the nomograms for predicting CSS and OS were 0.79 (95%CI: 0.74-0.84) and 0.80 (95%CI: 0.75-0.85), respectively. As indicated by calibration plots, optimal agreement was observed between prediction and observation in the external cohort. Conclusions: The nomograms developed and validated based on preoperative blood-based parameters were superior to any single variable for predicting CSS and OS after RNU.

Predicting poor outcome in patients with suspected COVID-19 presenting to the Emergency Department (COVERED) – Development, internal and external validation of a prediction model

2021 ◽  
Vol 20 (1) ◽  
pp. 4-14
Author(s):  
K. Azijli ◽  
◽  
A.W.E. Lieveld ◽  
S.F.B. van der Horst ◽  
N. de Graaf ◽  
...  
Keyword(s):  
Emergency Department ◽  
Prediction Model ◽  
Clinical Decision Making ◽  
Prediction Models ◽  
External Validation ◽  
Model Performance ◽  
Area Under The Curve ◽  
Brier Score ◽  
Multi Centre Study ◽  
Poor Outcome

Background: A recent systematic review recommends against the use of any of the current COVID-19 prediction models in clinical practice. To enable clinicians to appropriately profile and treat suspected COVID-19 patients at the emergency department (ED), externally validated models that predict poor outcome are desperately needed. Objective: Our aims were to identify predictors of poor outcome, defined as mortality or ICU admission within 30 days, in patients presenting to the ED with a clinical suspicion of COVID-19, and to develop and externally validate a prediction model for poor outcome. Methods: In this prospective, multi-centre study, we enrolled suspected COVID-19 patients presenting at the EDs of two hospitals in the Netherlands. We used backward logistic regression to develop a prediction model. We used the area under the curve (AUC), Brier score and pseudo-R2 to assess model performance. The model was externally validated in an Italian cohort. Results: We included 1193 patients between March 12 and May 27 2020, of whom 196 (16.4%) had a poor outcome. We identified 10 predictors of poor outcome: current malignancy (OR 2.774; 95%CI 1.682-4.576), systolic blood pressure (OR 0.981; 95%CI 0.964-0.998), heart rate (OR 1.001; 95%CI 0.97-1.028), respiratory rate (OR 1.078; 95%CI 1.046-1.111), oxygen saturation (OR 0.899; 95%CI 0.850-0.952), body temperature (OR 0.505; 95%CI 0.359-0.710), serum urea (OR 1.404; 95%CI 1.198-1.645), C-reactive protein (OR 1.013; 95%CI 1.001-1.024), lactate dehydrogenase (OR 1.007; 95%CI 1.002-1.013) and SARS-CoV-2 PCR result (OR 2.456; 95%CI 1.526-3.953). The AUC was 0.86 (95%CI 0.83-0.89), with a Brier score of 0.32 and, and R2 of 0.41. The AUC in the external validation in 500 patients was 0.70 (95%CI 0.65-0.75). Conclusion: The COVERED risk score showed excellent discriminatory ability, also in external validation. It may aid clinical decision making, and improve triage at the ED in health care environments with high patient throughputs.

Predictive models for identifying risk of readmission after index hospitalization for heart failure: A systematic review

2018 ◽  
Vol 17 (8) ◽  
pp. 675-689 ◽  
Author(s):  
Satish M Mahajan ◽  
Paul Heidenreich ◽  
Bruce Abbott ◽  
Ana Newton ◽  
Deborah Ward
Keyword(s):  
Heart Failure ◽  
Predictive Models ◽  
Complex Disease ◽  
Prediction Models ◽  
Predictive Accuracy ◽  
External Validation ◽  
Cox Proportional Hazards ◽  
Adverse Outcomes ◽  
Electronic Data ◽  
The Past

Aims: Readmission rates for patients with heart failure have consistently remained high over the past two decades. As more electronic data, computing power, and newer statistical techniques become available, data-driven care could be achieved by creating predictive models for adverse outcomes such as readmissions. We therefore aimed to review models for predicting risk of readmission for patients admitted for heart failure. We also aimed to analyze and possibly group the predictors used across the models. Methods: Major electronic databases were searched to identify studies that examined correlation between readmission for heart failure and risk factors using multivariate models. We rigorously followed the review process using PRISMA methodology and other established criteria for quality assessment of the studies. Results: We did a detailed review of 334 papers and found 25 multivariate predictive models built using data from either health system or trials. A majority of models was built using multiple logistic regression followed by Cox proportional hazards regression. Some newer studies ventured into non-parametric and machine learning methods. Overall predictive accuracy with C-statistics ranged from 0.59 to 0.84. We examined significant predictors across the studies using clinical, administrative, and psychosocial groups. Conclusions: Complex disease management and correspondingly increasing costs for heart failure are driving innovations in building risk prediction models for readmission. Large volumes of diverse electronic data and new statistical methods have improved the predictive power of the models over the past two decades. More work is needed for calibration, external validation, and deployment of such models for clinical use.

scholarly journals SOURCE: Prediction Models for Overall Survival in Patients With Metastatic and Potentially Curable Esophageal and Gastric Cancer

2021 ◽  
Vol 19 (4) ◽  
pp. 403-410
Author(s):  
Héctor G. van den Boorn ◽  
Ameen Abu-Hanna ◽  
Nadia Haj Mohammad ◽  
Maarten C.C.M. Hulshof ◽  
Suzanne S. Gisbertz ◽  
...  
Keyword(s):  
Gastric Cancer ◽  
Overall Survival ◽  
Proportional Hazards ◽  
Prediction Models ◽  
Metastatic Cancer ◽  
Cox Proportional Hazards ◽  
Future Research ◽  
Survival Prediction ◽  
Netherlands Cancer Registry ◽  
Treatment Type

Background: Personalized prediction of treatment outcomes can aid patients with cancer when deciding on treatment options. Existing prediction models for esophageal and gastric cancer, however, have mostly been developed for survival prediction after surgery (ie, when treatment has already been completed). Furthermore, prediction models for patients with metastatic cancer are scarce. The aim of this study was to develop prediction models of overall survival at diagnosis for patients with potentially curable and metastatic esophageal and gastric cancer (the SOURCE study). Methods: Data from 13,080 patients with esophageal or gastric cancer diagnosed in 2015 through 2018 were retrieved from the prospective Netherlands Cancer Registry. Four Cox proportional hazards regression models were created for patients with potentially curable and metastatic esophageal or gastric cancer. Predictors, including treatment type, were selected using the Akaike information criterion. The models were validated with temporal cross-validation on their C-index and calibration. Results: The validated model’s C-index was 0.78 for potentially curable gastric cancer and 0.80 for potentially curable esophageal cancer. For the metastatic models, the c-indices were 0.72 and 0.73 for esophageal and gastric cancer, respectively. The 95% confidence interval of the calibration intercepts and slopes contain the values 0 and 1, respectively. Conclusions: The SOURCE prediction models show fair to good c-indices and an overall good calibration. The models are the first in esophageal and gastric cancer to predict survival at diagnosis for a variety of treatments. Future research is needed to demonstrate their value for shared decision-making in clinical practice.

scholarly journals Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques

2020 ◽  
Author(s):  
Georgios Kantidakis ◽  
Hein Putter ◽  
Carlo Lancia ◽  
Jacob de Boer ◽  
Andries E Braat ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Liver Transplantation ◽  
Prediction Models ◽  
Machine Learning Techniques ◽  
Brier Score ◽  
Survival Prediction ◽  
Cox Models ◽  
Learning Techniques ◽  
Random Survival Forest

Abstract Background: Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest. Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians. Methods: In this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques. Results: Well-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years. Conclusion: In this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables.

A population-based validation of a clinical age-based prognostic tool to predict survival in melanoma patients (Preprint)

2021 ◽  
Author(s):  
Yu-Jen Wang ◽  
Mingchih Chen ◽  
Yen Chun Huang ◽  
Tian-Shyug Lee
Keyword(s):  
Machine Learning ◽  
Overall Survival ◽  
Prognostic Factors ◽  
Outcome Prediction ◽  
Prediction Models ◽  
Population Based ◽  
Survival Prediction ◽  
Prediction Tool ◽  
Old Patients ◽  
Melanoma Patients

BACKGROUND Melanoma is the most serious form of skin cancer, and the treatment can be challenging if the disease progresses to the metastatic stage. Depth of invasion is a good prognostic factor for predicting outcome. However, no good outcome prediction system that combines the staging system with other chronic systemic diseases is available to date. We investigated melanoma-related data from a population-based database and developed an outcome prediction tool for melanoma patients via machine learning. OBJECTIVE Build up a prediction tool for melanoma patients METHODS The clinical data of patients with melanoma were extracted from Taiwan’s National Health Insurance Research Database between 2008 and 2015 and were analysed in this study. Clinical data including demographic, pathologic, staging, and treatment data from melanoma patients over 18 years old were abstracted and collected. Prognostic factors were analyzed. Logistic regression (LR), random forest (RF) modelling, and multivariate adaptive regression splines (MARS) were applied to calculate predicted overall survival (OS). A 5-fold cross-validation method was applied. Two age groups (≥64 years old as the older age group and <64 years old as the general population group) with different prognostic factors were identified, and prognostic models for survival outcomes were built. RESULTS A total of 3481 patients were enrolled in our study. The 1-, 3-, and 5-year overall survival rates were 92.2%, 80.1%, and 70.3%, respectively. The Cox proportional hazard model showed that older age, male sex, higher grade, higher clinical stage, larger tumour size, positive surgical margins, no surgical intervention, and a higher Charlson comorbidity index (CCI) were associated with higher hazard ratios. LR, RF, and MARS techniques were used to validate the overall survival without tracking time, the accuracy of the MARS model for the <64-year-old patients and ≥64-year-old patients was 90.4% and 80.7%, respectively, with 3-, and 5-year the accuracy of prediction models are 94% and 89.6%. CONCLUSIONS Machine learning techniques offer excellent survival prediction in melanoma patients. Age-based survival prediction models may be applied for better clinical decision making. CLINICALTRIAL N/A

Sign in / Sign up

Export Citation Format

Share Document