scholarly journals A 12-hospital prospective evaluation of a clinical decision support prognostic algorithm based on logistic regression as a form of machine learning to facilitate decision making for patients with suspected COVID-19

PLoS ONE ◽  
2022 ◽  
Vol 17 (1) ◽  
pp. e0262193
Author(s):  
Monica I. Lupei ◽  
Danni Li ◽  
Nicholas E. Ingraham ◽  
Karyn D. Baum ◽  
Bradley Benson ◽  
...  
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Decision Support ◽  
Real Time ◽  
Acute Care ◽  
Clinical Decision Support ◽  
Invasive Mechanical Ventilation ◽  
Clinical Decision ◽  
Hospital System ◽  
Temporal Validation

Objective To prospectively evaluate a logistic regression-based machine learning (ML) prognostic algorithm implemented in real-time as a clinical decision support (CDS) system for symptomatic persons under investigation (PUI) for Coronavirus disease 2019 (COVID-19) in the emergency department (ED). Methods We developed in a 12-hospital system a model using training and validation followed by a real-time assessment. The LASSO guided feature selection included demographics, comorbidities, home medications, vital signs. We constructed a logistic regression-based ML algorithm to predict “severe” COVID-19, defined as patients requiring intensive care unit (ICU) admission, invasive mechanical ventilation, or died in or out-of-hospital. Training data included 1,469 adult patients who tested positive for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) within 14 days of acute care. We performed: 1) temporal validation in 414 SARS-CoV-2 positive patients, 2) validation in a PUI set of 13,271 patients with symptomatic SARS-CoV-2 test during an acute care visit, and 3) real-time validation in 2,174 ED patients with PUI test or positive SARS-CoV-2 result. Subgroup analysis was conducted across race and gender to ensure equity in performance. Results The algorithm performed well on pre-implementation validations for predicting COVID-19 severity: 1) the temporal validation had an area under the receiver operating characteristic (AUROC) of 0.87 (95%-CI: 0.83, 0.91); 2) validation in the PUI population had an AUROC of 0.82 (95%-CI: 0.81, 0.83). The ED CDS system performed well in real-time with an AUROC of 0.85 (95%-CI, 0.83, 0.87). Zero patients in the lowest quintile developed “severe” COVID-19. Patients in the highest quintile developed “severe” COVID-19 in 33.2% of cases. The models performed without significant differences between genders and among race/ethnicities (all p-values > 0.05). Conclusion A logistic regression model-based ML-enabled CDS can be developed, validated, and implemented with high performance across multiple hospitals while being equitable and maintaining performance in real-time validation.

scholarly journals A New Real Time Clinical Decision Support System Using Machine Learning for Critical Care Units

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 185676-185687
Author(s):  
Noha Ossama El-Ganainy ◽  
Ilangko Balasingham ◽  
Per Steinar Halvorsen ◽  
Leiv Arne Rosseland
Keyword(s):  
Machine Learning ◽  
Critical Care ◽  
Decision Support ◽  
Decision Support System ◽  
Real Time ◽  
Clinical Decision Support ◽  
Support System ◽  
Clinical Decision Support System ◽  
Clinical Decision ◽  
Critical Care Units

scholarly journals Opening the Black Box of Artificial Intelligence for Clinical Decision Support: A Study Predicting Stroke Outcome

2019 ◽  
Author(s):  
Esra Zihni ◽  
Vince Istvan Madai ◽  
Michelle Livne ◽  
Ivana Galinovic ◽  
Ahmed A. Khalil ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Logistic Regression ◽  
Decision Support ◽  
Clinical Features ◽  
Clinical Decision Support ◽  
Clinical Decision ◽  
Multilayer Perceptrons ◽  
Stroke Severity ◽  
Stroke Outcome

AbstractBackgroundState-of-the-art machine learning (ML) artificial intelligence methods are increasingly leveraged in clinical predictive modeling to provide clinical decision support systems to physicians. Modern ML approaches such as artificial neural networks (ANNs) and tree boosting often perform better than more traditional methods like logistic regression. On the other hand, these modern methods yield a limited understanding of the resulting predictions. However, in the medical domain, understanding of applied models is essential, in particular, when informing clinical decision support. Thus, in recent years, interpretability methods for modern ML methods have emerged to potentially allow explainable predictions paired with high performance.MethodsTo our knowledge, we present in this work the first explainability comparison of two modern ML methods, tree boosting and multilayer perceptrons (MLPs), to traditional logistic regression methods using a stroke outcome prediction paradigm. Here, we used clinical features to predict a dichotomized 90 days post-stroke modified Rankin Scale (mRS) score. For interpretability, we evaluated clinical features’ importance with regard to predictions using deep Taylor decomposition for MLP, Shapley values for tree boosting and model coefficients for logistic regression.ResultsWith regard to performance as measured by AUC values on the test dataset, all models performed comparably: Logistic regression AUCs were 0.82, 0.82, 0.79 for three different regularization schemes; tree boosting AUC was 0.81; MLP AUC was 0.81. Importantly, the interpretability analysis demonstrated consistent results across models by rating age and stroke severity consecutively amongst the most important predictive features. For less important features, some differences were observed between the methods.ConclusionsOur analysis suggests that modern machine learning methods can provide explainability which is compatible with domain knowledge interpretation and traditional method rankings. Future work should focus on replication of these findings in other datasets and further testing of different explainability methods.

scholarly journals Modified early warning score-based clinical decision support: cost impact and clinical outcomes in sepsis

JAMIA Open ◽  
2020 ◽  
Vol 3 (2) ◽  
pp. 261-268
Author(s):  
Devin J Horton ◽  
Kencee K Graves ◽  
Polina V Kukhareva ◽  
Stacy A Johnson ◽  
Maribel Cedillo ◽  
...  
Keyword(s):  
Decision Support ◽  
Acute Care ◽  
Direct Cost ◽  
Clinical Decision Support ◽  
Early Warning ◽  
Clinical Decision ◽  
Early Warning Score ◽  
Total Direct Cost ◽  
Intervention Period ◽  
Support Cost

Abstract Objective The objective of this study was to assess the clinical and financial impact of a quality improvement project that utilized a modified Early Warning Score (mEWS)-based clinical decision support intervention targeting early recognition of sepsis decompensation. Materials and Methods We conducted a retrospective, interrupted time series study on all adult patients who received a diagnosis of sepsis and were exposed to an acute care floor with the intervention. Primary outcomes (total direct cost, length of stay [LOS], and mortality) were aggregated for each study month for the post-intervention period (March 1, 2016–February 28, 2017, n = 2118 visits) and compared to the pre-intervention period (November 1, 2014–October 31, 2015, n = 1546 visits). Results The intervention was associated with a decrease in median total direct cost and hospital LOS by 23% (P = .047) and .63 days (P = .059), respectively. There was no significant change in mortality. Discussion The implementation of an mEWS-based clinical decision support system in eight acute care floors at an academic medical center was associated with reduced total direct cost and LOS for patients hospitalized with sepsis. This was seen without an associated increase in intensive care unit utilization or broad-spectrum antibiotic use. Conclusion An automated sepsis decompensation detection system has the potential to improve clinical and financial outcomes such as LOS and total direct cost. Further evaluation is needed to validate generalizability and to understand the relative importance of individual elements of the intervention.

scholarly journals NCOG-31. DEVELOPMENT AND EVALUATION OF PROGNOSTIC MODELS TO PREDICT SURVIVAL OF PATIENTS WITH GLIOBLASTOMA MULTIFORME

2020 ◽  
Vol 22 (Supplement_2) ◽  
pp. ii135-ii136
Author(s):  
John Lin ◽  
Michelle Mai ◽  
Saba Paracha
Keyword(s):  
Machine Learning ◽  
Decision Support ◽  
Glioblastoma Multiforme ◽  
Random Forest ◽  
Regression Model ◽  
Clinical Decision Support ◽  
Multivariate Regression ◽  
Visual Analysis ◽  
Clinical Decision ◽  
Prognostic Models

Abstract Glioblastoma multiforme (GBM), the most common form of glioma, is a malignant tumor with a high risk of mortality. By providing accurate survival estimates, prognostic models have been identified as promising tools in clinical decision support. In this study, we produced and validated two machine learning-based models to predict survival time for GBM patients. Publicly available clinical and genomic data from The Cancer Genome Atlas (TCGA) and Broad Institute GDAC Firehouse were obtained through cBioPortal. Random forest and multivariate regression models were created to predict survival. Predictive accuracy was assessed and compared through mean absolute error (MAE) and root mean square error (RMSE) calculations. 619 GBM patients were included in the dataset. There were 381 (62.9%) cases of recurrence/progression and 53 (8.7%) cases of disease-free survival. The MAE and RMSE values were 0.553 and 0.887 years respectively for the random forest regression model, and they were 1.756 and 2.451 years respectively for the multivariate regression model. Both models accurately predicted overall survival. Comparison of models through MAE, RMSE, and visual analysis produced higher accuracy values for random forest than multivariate linear regression. Further investigation on feature selection and model optimization may improve predictive power. These findings suggest that using machine learning in GBM prognostic modeling will improve clinical decision support. *Co-first authors.

GUESS: projecting machine learning scores to well-calibrated probability estimates for clinical decision-making

2018 ◽  
Vol 35 (14) ◽  
pp. 2458-2465 ◽  
Author(s):  
Johanna Schwarz ◽  
Dominik Heider
Keyword(s):  
Machine Learning ◽  
Decision Support ◽  
Clinical Decision Support ◽  
State Of The Art ◽  
Clinical Decision Support Systems ◽  
Clinical Decision ◽  
Calibration Method ◽  
Probability Estimates ◽  
Calibration Methods ◽  
Novel Method

Abstract Motivation Clinical decision support systems have been applied in numerous fields, ranging from cancer survival toward drug resistance prediction. Nevertheless, clinical decision support systems typically have a caveat: many of them are perceived as black-boxes by non-experts and, unfortunately, the obtained scores cannot usually be interpreted as class probability estimates. In probability-focused medical applications, it is not sufficient to perform well with regards to discrimination and, consequently, various calibration methods have been developed to enable probabilistic interpretation. The aims of this study were (i) to develop a tool for fast and comparative analysis of different calibration methods, (ii) to demonstrate their limitations for the use on clinical data and (iii) to introduce our novel method GUESS. Results We compared the performances of two different state-of-the-art calibration methods, namely histogram binning and Bayesian Binning in Quantiles, as well as our novel method GUESS on both, simulated and real-world datasets. GUESS demonstrated calibration performance comparable to the state-of-the-art methods and always retained accurate class discrimination. GUESS showed superior calibration performance in small datasets and therefore may be an optimal calibration method for typical clinical datasets. Moreover, we provide a framework (CalibratR) for R, which can be used to identify the most suitable calibration method for novel datasets in a timely and efficient manner. Using calibrated probability estimates instead of original classifier scores will contribute to the acceptance and dissemination of machine learning based classification models in cost-sensitive applications, such as clinical research. Availability and implementation GUESS as part of CalibratR can be downloaded at CRAN.

Real-Time Clinical Decision Support at the Point of Care

2019 ◽  
pp. 125-133
Author(s):  
Ana Margarida Pereira ◽  
Cristina Jácome ◽  
Rita Amaral ◽  
Tiago Jacinto ◽  
João A Fonseca
Keyword(s):  
Decision Support ◽  
Real Time ◽  
Clinical Decision Support ◽  
Point Of Care ◽  
Clinical Decision

An Assessment of Federated Machine Learning for Translational Research

2020 ◽  
pp. 123-142
Author(s):  
Manoj A. Thomas ◽  
Diya Suzanne Abraham ◽  
Dapeng Liu
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Decision Support ◽  
Translational Research ◽  
Clinical Decision Support ◽  
Clinical Decision ◽  
High Quality ◽  
Intermediate Data ◽  
Translational Informatics ◽  
Data Analytic

Translational research (TR) is the harnessing of knowledge from basic science and clinical research to advance healthcare. As a sister discipline, translational informatics (TI) concerns the application of informatics theories, methods, and frameworks to TR. This chapter builds upon TR concepts and aims to advance the use of machine learning (ML) and data analytics for improving clinical decision support. A federated machine learning (FML) architecture is described to aggregate multiple ML endpoints, and intermediate data analytic processes and products to output high quality knowledge discovery and decision making. The proposed architecture is evaluated for its operational performance based on three propositions, and a case for clinical decision support in the prediction of adult Sepsis is presented. The chapter illustrates contributions to the advancement of FML and TI.

Sign in / Sign up

Export Citation Format

Share Document