scholarly journals Prospective Real-World Performance Evaluation of a Machine Learning Algorithm to Predict 30-Day Readmissions in Patients with Heart Failure Using Electronic Medical Record Data

Iproceedings ◽  
10.2196/11897 ◽  
2018 ◽  
Vol 4 (2) ◽  
pp. e11897
Author(s):  
Sujay S Kakarmath ◽  
Neda Derakhshani ◽  
Sara B Golas ◽  
Jennifer Felsted ◽  
Takuma Shibahara ◽  
...  
Keyword(s):  
Machine Learning ◽  
Heart Failure ◽  
Performance Evaluation ◽  
Real World ◽  
Learning Algorithm ◽  
Machine Learning Algorithm ◽  
Medical Record Data ◽  
Electronic Medical Record Data ◽  
Patients With Heart Failure ◽  
Record Data

scholarly journals Prospective Real-World Performance Evaluation of a Machine Learning Algorithm to Predict 30-Day Readmissions in Patients with Heart Failure Using Electronic Medical Record Data (Preprint)

2018 ◽  
Author(s):  
Sujay S Kakarmath ◽  
Neda Derakhshani ◽  
Sara B. Golas ◽  
Jennifer Felsted ◽  
Takuma Shibahara ◽  
...  
Keyword(s):  
Machine Learning ◽  
Heart Failure ◽  
Real World ◽  
Predictive Performance ◽  
Training Dataset ◽  
Brier Score ◽  
Medical Record Data ◽  
Electronic Medical Record Data ◽  
Patients With Heart Failure ◽  
Record Data

BACKGROUND Heart failure (HF) patients have a high readmission rate with approximately 20% of patients being readmitted within 30-days after discharge. Hospital interventions to reduce HF readmissions are resource- and effort-intensive. Widespread availability of electronic medical record data has spurred interest in using machine learning-based techniques for risk stratification of heart failure patients. The predictive performance of machine learning-based predictive models is often evaluated solely using the Area Under the Receiver Operating Characteristic (AUROC) curve. However, the AUROC is independent of prevalence therefore predictive models with the same AUROC can have differential clinical utility. Furthermore, the AUROC does not provide any insight about the presence of overfitting or decay in predictive performance of a model over time, both of which can affect its real-world performance. OBJECTIVE Our primary objective is to assess real-world performance of a 30-day readmission risk prediction model for HF patients, which had an AUROC of 0.71 in the training dataset. METHODS Predictions for risk of 30-day readmissions in HF patients in the Partners Healthcare System were prospectively obtained from the model. We assessed the positive (PPV) and negative predictive value (NPV), in addition to sensitivity, specificity, accuracy, model calibration and Brier score. RESULTS Four hundred twenty index admissions that were not part of the training dataset were included in this prospective evaluation. Readmission rate was 24% (101 30-day readmissions). The AUROC of the predictive model was 0.57. At a discrimination threshold of 0.2 for flagging high-risk index admissions, the sensitivity and specificity of the model were 53.46% and 63.32%, respectively. The PPV and NPV were 31.57% and 81.12%, respectively. The Brier score was 0.19. CONCLUSIONS Our analysis offers important insights about the real-world performance of this predictive model. The NPV suggests that the model’s prediction about patients at low risk for readmission are reliable. This insight can be useful in optimizing resource allocation for patients with heart failure.

scholarly journals Validating a Machine Learning Algorithm to Predict 30-Day Re-Admissions in Patients With Heart Failure: Protocol for a Prospective Cohort Study (Preprint)

2017 ◽  
Author(s):  
Sujay Kakarmath ◽  
Sara Golas ◽  
Jennifer Felsted ◽  
Joseph Kvedar ◽  
Kamal Jethwani ◽  
...  
Keyword(s):  
Machine Learning ◽  
Heart Failure ◽  
Health Care ◽  
Medical Record ◽  
Electronic Medical Record ◽  
Health Care System ◽  
Predictive Models ◽  
Learning Algorithm ◽  
Machine Learning Algorithm ◽  
Patients With Heart Failure

BACKGROUND Big data solutions, particularly machine learning predictive algorithms, have demonstrated the ability to unlock value from data in real time in many settings outside of health care. Rapid growth in electronic medical record adoption and the shift from a volume-based to a value-based reimbursement structure in the US health care system has spurred investments in machine learning solutions. Machine learning methods can be used to build flexible, customized, and automated predictive models to optimize resource allocation and improve the efficiency and quality of health care. However, these models are prone to the problems of overfitting, confounding, and decay in predictive performance over time. It is, therefore, necessary to evaluate machine learning–based predictive models in an independent dataset before they can be adopted in the clinical practice. In this paper, we describe the protocol for independent, prospective validation of a machine learning–based model trained to predict the risk of 30-day re-admission in patients with heart failure. OBJECTIVE This study aims to prospectively validate a machine learning–based predictive model for inpatient admissions in patients with heart failure by comparing its predictions of risk for 30-day re-admissions against outcomes observed prospectively in an independent patient cohort. METHODS All adult patients with heart failure who are discharged alive from an inpatient admission will be prospectively monitored for 30-day re-admissions through reports generated by the electronic medical record system. Of these, patients who are part of the training dataset will be excluded to avoid information leakage to the algorithm. An expected sample size of 1228 index admissions will be required to observe a minimum of 100 30-day re-admission events. Deidentified structured and unstructured data will be fed to the algorithm, and its prediction will be recorded. The overall model performance will be assessed using the concordance statistic. Furthermore, multiple discrimination thresholds for screening high-risk patients will be evaluated according to the sensitivity, specificity, predictive values, and estimated cost savings to our health care system. RESULTS The project received funding in April 2017 and data collection began in June 2017. Enrollment was completed in July 2017. Data analysis is currently underway, and the first results are expected to be submitted for publication in October 2018. CONCLUSIONS To the best of our knowledge, this is one of the first studies to prospectively evaluate a predictive machine learning algorithm in a real-world setting. Findings from this study will help to measure the robustness of predictions made by machine learning algorithms and set a realistic benchmark for expectations of gains that can be made through its application to health care. REGISTERED REPORT IDENTIFIER RR1-10.2196/9466

scholarly journals Gap between real-world data and clinical research within hospitals in China: a qualitative study

BMJ Open ◽  
2020 ◽  
Vol 10 (12) ◽  
pp. e038375
Author(s):  
Feifei Jin ◽  
Chen Yao ◽  
Xiaoyan Yan ◽  
Chongya Dong ◽  
Junkai Lai ◽  
...  
Keyword(s):  
Qualitative Study ◽  
Clinical Research ◽  
Real World ◽  
Data Standards ◽  
Medical Record Data ◽  
Real World Data ◽  
World Data ◽  
Electronic Medical Record Data ◽  
Research Platform ◽  
Record Data

ObjectiveTo investigate the gap between real-world data and clinical research initiated by doctors in China, explore the potential reasons for this gap and collect different stakeholders’ suggestions.DesignThis qualitative study involved three types of hospital personnel based on three interview outlines. The data analysis was performed using the constructivist grounded theory analysis process.SettingSix tertiary hospitals (three general hospitals and three specialised hospitals) in Beijing, China, were included.ParticipantsIn total, 42 doctors from 12 departments, 5 information technology managers and 4 clinical managers were interviewed through stratified purposive sampling.ResultsElectronic medical record data cannot be directly downloaded into clinical research files, which is a major problem in China. The lack of data interoperability, unstructured electronic medical record data and concerns regarding data security create a gap between real-world data and research data. Updating hospital information systems, promoting data standards and establishing an independent clinical research platform may be feasible suggestions for solving the current problems.ConclusionsDetermining the causes of gaps and targeted solutions could contribute to the development of clinical research in China. This research suggests that updating the hospital information system, promoting data standards and establishing a clinical research platform could promote the use of real-world data in the future.

scholarly journals 153 Multiparametric assessment of diastolic function in heart failure

2020 ◽  
Vol 21 (Supplement_1) ◽  
Author(s):  
T Uejima ◽  
J Cho ◽  
H Hayama ◽  
L Takahashi ◽  
J Yajima ◽  
...  
Keyword(s):  
Machine Learning ◽  
Heart Failure ◽  
Cluster Analysis ◽  
Diastolic Function ◽  
Primary Endpoint ◽  
Prognostic Value ◽  
Learning Algorithm ◽  
Left Ventricular ◽  
Machine Learning Techniques ◽  
Machine Learning Algorithm

Abstract Background The assessment of diastolic function is still challenging in the setting of heart failure (HF). We tested the hypothesis that applying a machine learning algorithm would detect heterogeneity in diastolic function and improve risk stratification in HF population. Methods This study included consecutive 279 patients with clinically stable HF referred for echocardiographic assessment, for whom diastolic function variables were measured according to the current guidelines. Cluster analysis, an unsupervised machine learning algorithm, was undertaken on these variables to form homogeneous groups of patients with similar profiles of the variables. Sequential Cox models paralleling the clinical sequence of HF assessment were used to elucidate the benefit of cluster-based classification over guidelines-based classification. The primary endpoint was a hospitalization for worsening HF. Results Cluster analysis identified 3 clusters with distinct properties of diastolic function that shared similarities with guidelines-based classification. The clusters were associated with brain natriuretic peptide level (p < 0.001, figure A). During follow-up period of 2.6 ± 2.0 years, 62 patients (22%) experienced the primary endpoint. Cluster-based classification exhibited a significant prognostic value (c2 = 20.3, p < 0.001, figure B), independent from and incremental to an established clinical risk score for HF (MAGGIC score) and left ventricular end-diastolic volume (hazard ratio = 1.677, p = 0.017, model c2: from 47.5 to 54.1, p = 0.015, figure D). Although guideline-based classification showed a significant prognostic value (c2 = 13.1, p = 0.001, figure C), it did not significantly improve overall prognostication from the baseline (model c2: from 47.5 to 49.9, p = 0.199, figure D). Conclusion Machine learning techniques help grading diastolic function and stratifying the risk for decompensation in HF. Abstract 153 Figure.

Empirical Comparison of Various Discretization Procedures

1998 ◽  
Vol 12 (07) ◽  
pp. 1017-1032 ◽  
Author(s):  
Petr Berka ◽  
Ivan Bruha
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Learning Algorithm ◽  
Machine Learning Algorithms ◽  
The Other ◽  
Machine Learning Algorithm ◽  
Empirical Comparison ◽  
Numerical Attributes ◽  
Real World Problems ◽  
Discretization Procedure

The genuine symbolic machine learning (ML) algorithms are capable of processing symbolic, categorial data only. However, real-world problems, e.g. in medicine or finance, involve both symbolic and numerical attributes. Therefore, there is an important issue of ML to discretize (categorize) numerical attributes. There exist quite a few discretization procedures in the ML field. This paper describes two newer algorithms for categorization (discretization) of numerical attributes. The first one is implemented in the KEX (Knowledge EXplorer) as its preprocessing procedure. Its idea is to discretize the numerical attributes in such a way that the resulting categorization corresponds to KEX knowledge acquisition algorithm. Since the categorization for KEX is done "off-line" before using the KEX machine learning algorithm, it can be used as a preprocessing step for other machine learning algorithms, too. The other discretization procedure is implemented in CN4, a large extension of the well-known CN2 machine learning algorithm. The range of numerical attributes is divided into intervals that may form a complex generated by the algorithm as a part of the class description. Experimental results show a comparison of performance of KEX and CN4 on some well-known ML databases. To make the comparison more exhibitory, we also used the discretization procedure of the MLC++ library. Other ML algorithms such as ID3 and C4.5 were run under our experiments, too. Then, the results are compared and discussed.

scholarly journals Prediction of Acute Kidney Injury With a Machine Learning Algorithm Using Electronic Health Record Data

2018 ◽  
Vol 5 ◽  
pp. 205435811877632 ◽  
Author(s):  
Hamid Mohamadlou ◽  
Anna Lynn-Palevsky ◽  
Christopher Barton ◽  
Uli Chettipally ◽  
Lisa Shieh ◽  
...  
Keyword(s):  
Machine Learning ◽  
Acute Kidney Injury ◽  
Electronic Health Record ◽  
Learning Algorithm ◽  
Kidney Injury ◽  
Health Record ◽  
Machine Learning Algorithm ◽  
Electronic Health Record Data ◽  
Record Data ◽  
Electronic Health
Sign in / Sign up

Export Citation Format

Share Document