Alert dwell time: introduction of a measure to evaluate interruptive clinical decision support alerts

Abstract Metrics for evaluating interruptive prescribing alerts have many limitations. Additional methods are needed to identify opportunities to improve alerting systems and prevent alert fatigue. In this study, the authors determined whether alert dwell time—the time elapsed from when an interruptive alert is generated to when it is dismissed—could be calculated by using historical alert data from log files. Drug–drug interaction (DDI) alerts from 3 years of electronic health record data were queried. Alert dwell time was calculated for 25,965 alerts, including 777 unique DDIs. The median alert dwell time was 8 s (range, 1–4913 s). Resident physicians had longer median alert dwell times than other prescribers ( P < .001). The 10 most frequent DDI alerts ( n = 8759 alerts) had shorter median dwell times than alerts that only occurred once ( P < .001). This metric can be used in future research to evaluate the effectiveness and efficiency of interruptive prescribing alerts.

Download Full-text

Clinician involvement in research on machine learning–based predictive clinical decision support for the hospital setting: A scoping review

Journal of the American Medical Informatics Association ◽

10.1093/jamia/ocaa296 ◽

2020 ◽

Author(s):

Jessica M Schwartz ◽

Amanda J Moy ◽

Sarah C Rossetti ◽

Noémie Elhadad ◽

Kenrick D Cato

Keyword(s):

Machine Learning ◽

Decision Support ◽

Conference Proceedings ◽

System Design ◽

Hospital Setting ◽

Clinical Decision ◽

Health Record ◽

Electronic Health Record Data ◽

Record Data ◽

Clinical Expert

Abstract Objective The study sought to describe the prevalence and nature of clinical expert involvement in the development, evaluation, and implementation of clinical decision support systems (CDSSs) that utilize machine learning to analyze electronic health record data to assist nurses and physicians in prognostic and treatment decision making (ie, predictive CDSSs) in the hospital. Materials and Methods A systematic search of PubMed, CINAHL, and IEEE Xplore and hand-searching of relevant conference proceedings were conducted to identify eligible articles. Empirical studies of predictive CDSSs using electronic health record data for nurses or physicians in the hospital setting published in the last 5 years in peer-reviewed journals or conference proceedings were eligible for synthesis. Data from eligible studies regarding clinician involvement, stage in system design, predictive CDSS intention, and target clinician were charted and summarized. Results Eighty studies met eligibility criteria. Clinical expert involvement was most prevalent at the beginning and late stages of system design. Most articles (95%) described developing and evaluating machine learning models, 28% of which described involving clinical experts, with nearly half functioning to verify the clinical correctness or relevance of the model (47%). Discussion Involvement of clinical experts in predictive CDSS design should be explicitly reported in publications and evaluated for the potential to overcome predictive CDSS adoption challenges. Conclusions If present, clinical expert involvement is most prevalent when predictive CDSS specifications are made or when system implementations are evaluated. However, clinical experts are less prevalent in developmental stages to verify clinical correctness, select model features, preprocess data, or serve as a gold standard.

Download Full-text

Enhanced Screening and Research Data Collection via Automated EHR Data Capture and Early Identification of Sepsis

SAGE Open Nursing ◽

10.1177/2377960819850972 ◽

2019 ◽

Vol 5 ◽

pp. 237796081985097

Author(s):

Reba Umberger ◽

Chayawat “Yo” Indranoi ◽

Melanie Simpson ◽

Rose Jensen ◽

James Shamiyeh ◽

...

Keyword(s):

Electronic Health Record ◽

Data Extraction ◽

Future Research ◽

Health Record ◽

Electronic Health Record Data ◽

Cross Sectional ◽

Apply Research ◽

Record Data ◽

Electronic Health ◽

Research Criteria

Clinical research in sepsis patients often requires gathering large amounts of longitudinal information. The electronic health record can be used to identify patients with sepsis, improve participant study recruitment, and extract data. The process of extracting data in a reliable and usable format is challenging, despite standard programming language. The aims of this project were to explore infrastructures for capturing electronic health record data and to apply criteria for identifying patients with sepsis. We conducted a prospective feasibility study to locate and capture/abstract electronic health record data for future sepsis studies. We located parameters as displayed to providers within the system and then captured data transmitted in Health Level Seven® interfaces between electronic health record systems into a prototype database. We evaluated our ability to successfully identify patients admitted with sepsis in the target intensive care unit (ICU) at two cross-sectional time points and then over a 2-month period. A majority of the selected parameters were accessible using an iterative process to locate and abstract them to the prototype database. We successfully identified patients admitted to a 20-bed ICU with sepsis using four data interfaces. Retrospectively applying similar criteria to data captured for 319 patients admitted to ICU over a 2-month period was less sensitive in identifying patients admitted directly to the ICU with sepsis. Classification into three admission categories (sepsis, no-sepsis, and other) was fair (Kappa .39) when compared with manual chart review. This project confirms reported barriers in data extraction. Data can be abstracted for future research, although more work is needed to refine and create customizable reports. We recommend that researchers engage their information technology department to electronically apply research criteria for improved research screening at the point of ICU admission. Using clinical electronic health records data to classify patients with sepsis over time is complex and challenging.

Download Full-text

Assessment of the Feasibility of automated, real-time clinical decision support in the emergency department using electronic health record data

BMC Emergency Medicine ◽

10.1186/s12873-018-0170-9 ◽

2018 ◽

Vol 18 (1) ◽

Cited By ~ 3

Author(s):

Warren M. Perry ◽

Rubayet Hossain ◽

Richard A. Taylor

Keyword(s):

Emergency Department ◽

Decision Support ◽

Electronic Health Record ◽

Real Time ◽

Clinical Decision Support ◽

Clinical Decision ◽

Health Record ◽

Electronic Health Record Data ◽

Record Data ◽

Electronic Health

Download Full-text

Toward a More Robust and Efficient Usability Testing Method of Clinical Decision Support for Nurses Derived From Nursing Electronic Health Record Data

International Journal of Nursing Knowledge ◽

10.1111/2047-3095.12146 ◽

2016 ◽

Vol 28 (4) ◽

pp. 211-218 ◽

Cited By ~ 5

Author(s):

Karen Dunn Lopez ◽

Alessandro Febretti ◽

Janet Stifter ◽

Andrew Johnson ◽

Diana J. Wilkie ◽

...

Keyword(s):

Decision Support ◽

Electronic Health Record ◽

Clinical Decision Support ◽

Usability Testing ◽

Clinical Decision ◽

Health Record ◽

Electronic Health Record Data ◽

Testing Method ◽

Record Data ◽

Electronic Health

Download Full-text

Leveraging electronic health records for predictive modeling of post-surgical complications

Statistical Methods in Medical Research ◽

10.1177/0962280217696115 ◽

2017 ◽

Vol 27 (11) ◽

pp. 3271-3285 ◽

Cited By ~ 10

Author(s):

Grant B Weller ◽

Jenna Lovely ◽

David W Larson ◽

Berton A Earnshaw ◽

Marianne Huebner

Keyword(s):

Electronic Health Record ◽

Predictive Modeling ◽

Surgical Complications ◽

Decision Rules ◽

Clinical Decision ◽

Health Record ◽

Electronic Health Record Data ◽

Health Records ◽

Record Data ◽

Electronic Health

Hospital-specific electronic health record systems are used to inform clinical practice about best practices and quality improvements. Many surgical centers have developed deterministic clinical decision rules to discover adverse events (e.g. postoperative complications) using electronic health record data. However, these data provide opportunities to use probabilistic methods for early prediction of adverse health events, which may be more informative than deterministic algorithms. Electronic health record data from a set of 9598 colorectal surgery cases from 2010 to 2014 were used to predict the occurrence of selected complications including surgical site infection, ileus, and bleeding. Consistent with previous studies, we find a high rate of missing values for both covariates and complication information (4–90%). Several machine learning classification methods are trained on an 80% random sample of cases and tested on a remaining holdout set. Predictive performance varies by complication, although an area under the receiver operating characteristic curve as high as 0.86 on testing data was achieved for bleeding complications, and accuracy for all complications compares favorably to existing clinical decision rules. Our results confirm that electronic health records provide opportunities for improved risk prediction of surgical complications; however, consideration of data quality and consistency standards is an important step in predictive modeling with such data.

Download Full-text