Predicting Survival in Veterans with Follicular Lymphoma Using Structured Electronic Health Record Information and Machine Learning

The most accurate prognostic approach for follicular lymphoma (FL), progression of disease at 24 months (POD24), requires two years’ observation after initiating first-line therapy (L1) to predict outcomes. We applied machine learning to structured electronic health record (EHR) data to predict individual survival at L1 initiation. We grouped 523 observations and 1933 variables from a nationwide cohort of FL patients diagnosed 2006–2014 in the Veterans Health Administration into traditionally used prognostic variables (“curated”), commonly measured labs (“labs”), and International Classification of Diseases diagnostic codes (“ICD”) sets. We compared performance of random survival forests (RSF) vs. traditional Cox model using four datasets: curated, curated + labs, curated + ICD, and curated + ICD + labs, also using Cox on curated + POD24. We evaluated variable importance and partial dependence plots with area under the receiver operating characteristic curve (AUC). RSF with curated + labs performed best, with mean AUC 0.73 (95% CI: 0.71–0.75). It approximated, but did not surpass, Cox with POD24 (mean AUC 0.74 [95% CI: 0.71–0.77]). RSF using EHR data achieved better performance than traditional prognostic variables, setting the foundation for the incorporation of our algorithm into the EHR. It also provides for possible future scenarios in which clinicians could be provided an EHR-based tool which approximates the predictive ability of the most accurate known indicator, using information available 24 months earlier.

Download Full-text

Effect of the implementation of an enterprise-wide Electronic Health Record on productivity in the Veterans Health Administration

Health Economics Policy and Law ◽

10.1017/s1744133105001210 ◽

2006 ◽

Vol 1 (2) ◽

pp. 163-169 ◽

Cited By ~ 53

Author(s):

Dwight C. Evans ◽

W. Paul Nichol ◽

Jonathan B. Perlin

Keyword(s):

Information Technology ◽

Electronic Health Record ◽

Veterans Health Administration ◽

Health Record ◽

Dramatic Improvement ◽

Veterans Health ◽

Health Administration ◽

Software Engineers ◽

Electronic Health

Since 1995, the Veterans Health Administration (VHA) has had an ongoing process of systems improvement that has led to dramatic improvement in the quality of care delivered. A major component of the redesign of the VHA has been the creation of a fully developed enterprise-wide Electronic Health Record (EHR). VHA’s Health Information Technology was developed in a collaborative fashion between local clinical champions and central software engineers. Successful national EHR implementation was achieved by 1999, since when the VHA has been able to increase its productivity by nearly 6 per cent per year.

Download Full-text

Using Machine Learning and the Electronic Health Record to Predict Complicated Clostridium difficile Infection

Open Forum Infectious Diseases ◽

10.1093/ofid/ofz186 ◽

2019 ◽

Vol 6 (5) ◽

Cited By ~ 14

Author(s):

Benjamin Y Li ◽

Jeeheh Oh ◽

Vincent B Young ◽

Krishna Rao ◽

Jenna Wiens

Keyword(s):

Machine Learning ◽

Electronic Health Record ◽

Characteristic Curve ◽

Model Performance ◽

Health Record ◽

Data Set ◽

Icu Admission ◽

Clostridioides Difficile ◽

Electronic Health ◽

Health Care Associated Infection

Abstract Background Clostridium (Clostridioides) difficile infection (CDI) is a health care–associated infection that can lead to serious complications. Potential complications include intensive care unit (ICU) admission, development of toxic megacolon, need for colectomy, and death. However, identifying the patients most likely to develop complicated CDI is challenging. To this end, we explored the utility of a machine learning (ML) approach for patient risk stratification for complications using electronic health record (EHR) data. Methods We considered adult patients diagnosed with CDI between October 2010 and January 2013 at the University of Michigan hospitals. Cases were labeled complicated if the infection resulted in ICU admission, colectomy, or 30-day mortality. Leveraging EHR data, we trained a model to predict subsequent complications on each of the 3 days after diagnosis. We compared our EHR-based model to one based on a small set of manually curated features. We evaluated model performance using a held-out data set in terms of the area under the receiver operating characteristic curve (AUROC). Results Of 1118 cases of CDI, 8% became complicated. On the day of diagnosis, the model achieved an AUROC of 0.69 (95% confidence interval [CI], 0.55–0.83). Using data extracted 2 days after CDI diagnosis, performance increased (AUROC, 0.90; 95% CI, 0.83–0.95), outperforming a model based on a curated set of features (AUROC, 0.84; 95% CI, 0.75–0.91). Conclusions Using EHR data, we can accurately stratify CDI cases according to their risk of developing complications. Such an approach could be used to guide future clinical studies investigating interventions that could prevent or mitigate complicated CDI.

Download Full-text

Reply to the Editorial Comment on: Using an Automated Electronic Health Record Score To Estimate Life Expectancy In Men Diagnosed With Prostate Cancer In The Veterans Health Administration. Urology. 2021

Urology ◽

10.1016/j.urology.2021.05.056 ◽

2021 ◽

Author(s):

Simon John Christoph Soerensen ◽

I-Chun Thomas ◽

Bogdana Schmidt ◽

Timothy Daskivich ◽

Ted A. Skolarus ◽

...

Keyword(s):

Prostate Cancer ◽

Life Expectancy ◽

Editorial Comment ◽

Electronic Health Record ◽

Veterans Health Administration ◽

Health Record ◽

Veterans Health ◽

Health Administration ◽

Estimate Life Expectancy ◽

Electronic Health

Download Full-text

Validation of an Internationally Derived Patient Severity Phenotype to Support COVID-19 Analytics from Electronic Health Record Data

Journal of the American Medical Informatics Association ◽

10.1093/jamia/ocab018 ◽

2021 ◽

Author(s):

Jeffrey G Klann ◽

Griffin M Weber ◽

Hossein Estiri ◽

Bertrand Moal ◽

Paul Avillach ◽

...

Keyword(s):

Machine Learning ◽

Electronic Health Record ◽

Chart Review ◽

Learning Approach ◽

Health Record ◽

Learning Approaches ◽

Electronic Health Record Data ◽

Icu Admission ◽

Machine Learning Approach ◽

Electronic Health

Abstract Introduction The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing COVID-19 with federated analyses of electronic health record (EHR) data. Objective We sought to develop and validate a computable phenotype for COVID-19 severity. Methods Twelve 4CE sites participated. First we developed an EHR-based severity phenotype consisting of six code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of ICU admission and/or death. We also piloted an alternative machine-learning approach and compared selected predictors of severity to the 4CE phenotype at one site. Results The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability - up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean AUC 0.903 (95% CI: 0.886, 0.921), compared to AUC 0.956 (95% CI: 0.952, 0.959) for the machine-learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared to chart review. Discussion We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine-learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly due to heterogeneous pandemic conditions. Conclusion We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.

Download Full-text

Assessing stroke severity using electronic health record data: a machine learning approach

BMC Medical Informatics and Decision Making ◽

10.1186/s12911-019-1010-x ◽

2020 ◽

Vol 20 (1) ◽

Cited By ~ 3

Author(s):

Emily Kogan ◽

Kathryn Twyman ◽

Jesse Heap ◽

Dejan Milentijevic ◽

Jennifer H. Lin ◽

...

Keyword(s):

Machine Learning ◽

Electronic Health Record ◽

Patient Outcomes ◽

Stroke Severity ◽

Health Record ◽

Learning Models ◽

Electronic Health Record Data ◽

Record Data ◽

Electronic Health ◽

Machine Learning Models

Abstract Background Stroke severity is an important predictor of patient outcomes and is commonly measured with the National Institutes of Health Stroke Scale (NIHSS) scores. Because these scores are often recorded as free text in physician reports, structured real-world evidence databases seldom include the severity. The aim of this study was to use machine learning models to impute NIHSS scores for all patients with newly diagnosed stroke from multi-institution electronic health record (EHR) data. Methods NIHSS scores available in the Optum© de-identified Integrated Claims-Clinical dataset were extracted from physician notes by applying natural language processing (NLP) methods. The cohort analyzed in the study consists of the 7149 patients with an inpatient or emergency room diagnosis of ischemic stroke, hemorrhagic stroke, or transient ischemic attack and a corresponding NLP-extracted NIHSS score. A subset of these patients (n = 1033, 14%) were held out for independent validation of model performance and the remaining patients (n = 6116, 86%) were used for training the model. Several machine learning models were evaluated, and parameters optimized using cross-validation on the training set. The model with optimal performance, a random forest model, was ultimately evaluated on the holdout set. Results Leveraging machine learning we identified the main factors in electronic health record data for assessing stroke severity, including death within the same month as stroke occurrence, length of hospital stay following stroke occurrence, aphagia/dysphagia diagnosis, hemiplegia diagnosis, and whether a patient was discharged to home or self-care. Comparing the imputed NIHSS scores to the NLP-extracted NIHSS scores on the holdout data set yielded an R2 (coefficient of determination) of 0.57, an R (Pearson correlation coefficient) of 0.76, and a root-mean-squared error of 4.5. Conclusions Machine learning models built on EHR data can be used to determine proxies for stroke severity. This enables severity to be incorporated in studies of stroke patient outcomes using administrative and EHR databases.

Download Full-text

DEVELOPMENT OF A PREDICTION MODEL FOR INCIDENT MYOCARDIAL INFARCTION USING MACHINE LEARNING APPLIED TO HARMONIZED ELECTRONIC HEALTH RECORD DATA

Journal of the American College of Cardiology ◽

10.1016/s0735-1097(20)30821-4 ◽

2020 ◽

Vol 75 (11) ◽

pp. 194

Author(s):

Divneet Mandair ◽

Premanand Tiwari ◽

Steven Simon ◽

Michael Rosenberg

Keyword(s):

Machine Learning ◽

Myocardial Infarction ◽

Prediction Model ◽

Electronic Health Record ◽

Health Record ◽

Electronic Health Record Data ◽

Record Data ◽

Electronic Health

Download Full-text

A Time-Updated, Parsimonious Model to Predict AKI in Hospitalized Children

Journal of the American Society of Nephrology ◽

10.1681/asn.2019070745 ◽

2020 ◽

Vol 31 (6) ◽

pp. 1348-1357 ◽

Cited By ~ 1

Author(s):

Ibrahim Sandokji ◽

Yu Yamamoto ◽

Aditya Biswas ◽

Tanima Arora ◽

Ugochukwu Ugwuowo ◽

...

Keyword(s):

Machine Learning ◽

Prediction Model ◽

Receiver Operating Characteristic Curve ◽

Operating Characteristic ◽

Characteristic Curve ◽

External Validation ◽

Health Record ◽

Hospitalized Children ◽

Operating Characteristic Curve ◽

Electronic Health

BackgroundTimely prediction of AKI in children can allow for targeted interventions, but the wealth of data in the electronic health record poses unique modeling challenges.MethodsWe retrospectively reviewed the electronic medical records of all children younger than 18 years old who had at least two creatinine values measured during a hospital admission from January 2014 through January 2018. We divided the study population into derivation, and internal and external validation cohorts, and used five feature selection techniques to select 10 of 720 potentially predictive variables from the electronic health records. Model performance was assessed by the area under the receiver operating characteristic curve in the validation cohorts. The primary outcome was development of AKI (per the Kidney Disease Improving Global Outcomes creatinine definition) within a moving 48-hour window. Secondary outcomes included severe AKI (stage 2 or 3), inpatient mortality, and length of stay.ResultsAmong 8473 encounters studied, AKI occurred in 516 (10.2%), 207 (9%), and 27 (2.5%) encounters in the derivation, and internal and external validation cohorts, respectively. The highest-performing model used a machine learning-based genetic algorithm, with an overall receiver operating characteristic curve in the internal validation cohort of 0.76 [95% confidence interval (CI), 0.72 to 0.79] for AKI, 0.79 (95% CI, 0.74 to 0.83) for severe AKI, and 0.81 (95% CI, 0.77 to 0.86) for neonatal AKI. To translate this prediction model into a clinical risk-stratification tool, we identified high- and low-risk threshold points.ConclusionsUsing various machine learning algorithms, we identified and validated a time-updated prediction model of ten readily available electronic health record variables to accurately predict imminent AKI in hospitalized children.

Download Full-text