Development and Validation of a QTc-Prolongation Risk Score to Optimize Interruptive Medication Alerts

Introduction: Drug-drug interaction (DDI) warnings are employed in many institutions when more than one QTc-prolonging medication is prescribed; however, this leads to alert fatigue where alerts are frequently overridden by clinicians due to patient non-specificity or low risk. This study aimed at reducing alert fatigue through developing a custom alert triggered by a patient-specific QTc-prolongation risk score, and validating it against database-driven DDI warnings for QTc prolongation. Methods and Results: Between November 23, 2019 and January 31, 2020, inpatients with a baseline and a follow-up 12-lead ECG reading within 14 days were identified. Each time a QTc-prolonging medication order was signed or verified, the QTc-prolongation risk score was calculated in the electronic health record (EHR), triggering a custom alert in the background. Follow-up 12-lead ECG readings were used to calculate sensitivity and specificity for both the custom alert and the DDI warning. A total of 100 patients had a risk score calculation and were included in our analysis, representing 521 custom alerts and 449 DDI warnings. The preliminary QTc-prolongation risk score did not achieve a reduction in false positive alerts with a cutoff of 10 points. A multiple logistic regression was performed to re-arrange the components and optimize the risk score. Conclusion: Our adjusted QTc-prolongation risk score, with a cutoff of 5 points, achieved a specificity of 66% and a negative predictive value of 83%. These results will allow us to integrate the risk score into the EHR as a guidance tool to predict QTc-prolongation.

Addressing alert fatigue by reducing radiation oncology software alert volume.

261 Background: Radiation therapists (RTTs) administer radiation treatments to patients with cancer. Treatments are delivered using linear accelerators (LINACs), operated by vendor specific software. Prior to delivering treatment, RTTs perform a time-out, and read aloud critical electronic communications (alerts) entered by members of the radiation oncology care team. Alerts are effective at communicating critical information, including treatment setup and imaging instructions, but can become a source of error due to alert fatigue when placed indiscriminately. Methods: A multicenter retrospective review of alert use per patient was conducted in 4 radiation oncology centers with a total of 6 LINACs. Alert usage was reviewed pre-intervention for 40 randomly selected patients using manual chart review. Each alert was reviewed for frequency and utilization. In attempt of improving communication and reducing alert fatigue, a multidisciplinary process improvement working group (with Radiation Oncologists, RTTs, nursing, physicists, and administration) was formed to review the utilization of alerts in our department and propose interventions. Three months after intervention, an additional 40 chart review was performed. Our aim was to reduce the volume of alerts by 20% within 3 months. A 2-tail t-test was used for statistical analysis. Results: Process improvements were implemented to reduce the volume of alerts per patient. Interventions included 1) defining an alert for all departmental staff, 2) creating guidelines for appropriate utilization of alerts, 3) routing communications not critical to RTTs at the time of radiation treatment administration through other channels, and 4) training staff as to the above. The pre-intervention review yielded 239 alerts. Post-intervention, there were 173 alerts, a reduction of 27% (p =.008). Conclusions: This practice change reduced average alert volume by 27%. As a result, alerts which are critical to safe treatment delivery by RTTs (i.e. daily setup alerts), became more heavily represented. Other alerts, which could be communicated effectively in other ways (i.e. OTVs [weekly on treatment visit with Radiation Oncologist]), were eliminated. By decreasing alert volume, the risk of RTT alert fatigue is reduced, communication improved, and treatment safety enhanced.[Table: see text]

Development and Assessment of PharmaCheck: An Electronic Screening Tool for the Prevention of Twenty Major Adverse Drug Events

Background: Adverse drug events (ADEs) can be prevented by deploying clinical decision support systems (CDSS) that directly assist physicians, via computerized order entry systems, and clinical pharmacists performing medication reviews as part of medical rounds. However, physicians using CDSS are known to be exposed to the alert-fatigue phenomenon. Our study aimed to assess the performance of PharmaCheck—a CDSS to help clinical pharmacists detect high-risk situations with the potential to lead to ADEs—and its impact on clinical pharmacists’ activities.Methods: Twenty clinical rules, divided into four risk classes, were set for the daily screening of high-risk situations in the electronic health records of patients admitted to our General Internal Medicine Department. Alerts to clinical pharmacists encouraged them to telephone prescribers and suggest any necessary treatment adjustments. PharmaCheck’s performance was assessed using the intervention’s positive predictive value (PPV), which characterizes the proportion of interventions for each alert triggered. PharmaCheck’s impact was assessed by considering clinical pharmacists as a filter for ruling out futile alerts and by comparing the final clinical PPV with a pharmacist (the proportion of interventions that led to a change in the medical regimen) to the final clinical PPV without a pharmacist.Results: Over 132 days, 447 alerts were triggered for 383 patients, leading to 90 interventions (overall intervention PPV = 20.1%). By risk class, intervention PPVs made up 26.9% (n = 65/242) of abnormal laboratory value alerts, 3.1% (4/127) of alerts for contraindicated medications or medications to be used with caution, 28.2% (20/71) of drug–drug interaction alerts, and 14.3% (1/7) of inadequate mode of administration alerts. Clinical PPVs reached 71.0% (64/90) when pharmacists filtered alerts and 14% (64/242) if they were not doing it.Conclusion: PharmaCheck enabled clinical pharmacists to improve their traditional processes and broaden their coverage by focusing on 20 high-risk situations. Alert management by pharmacists seemed to be a more effective way of preventing risky situations and alert fatigue than a model addressing alerts to physicians exclusively. Some fine-tuning could enhance PharmaCheck's performance by considering the information quality of triggers, the variability of clinical settings, and the fact that some prescription processes are already highly secured.

Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials

Background Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools. Objective This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care. Methods A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior. Results To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images. Conclusions These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS. Trial Registration Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191

Physician resilience to computer decision support system-generated alerts (Preprint)

UNSTRUCTURED Can computerized clinical decision support system (CDSS) generated alerts increase professional distress in clinical practice? We review published studies exploring potential negative implications associated with the wide implementation of CDSSs linked to electronic health records. CDSSs have been developed to optimize standards of care and warn physicians of potential medication errors. In routine practice, however, clinicians are beset by a barrage of disruptive, often clinically irrelevant alerts, resulting in information overload, alert fatigue and, in worst case scenarios, they might represent a hazard to patient safety. Overriding alerts has become a defense tactic against tedious and time-consuming notifications. This tactic exposes physicians to potential drawbacks and represents a system failure. A critical rethinking of the design and implementation of alerts implies a greater focus on practice-related needs and the contexts in which end-users operate, as well as a hospital-wide re-evaluation of alarm notification systems. Shared responsibility between key stakeholders is essential to minimize alert fatigue and restore physician resilience.

Leveraging the Hyperledger Fabric for Enhancing the Efficacy of Clinical Decision Support Systems

Adopting and implementing the Clinical Decision Support System (CDSS) technology is a critical element in an effort to improve national quality initiatives and evidence-based practice at the point of care. CDSS is envisioned to be a potential solution to many current challenges in the healthcare sphere, which includes information overload, practice improvement, eliminating treatment errors, and reducing medical consultation costs. However, the CDSS did not manage to achieve these goals to the desired levels and provide context-appropriate alerts, although integrated with the electronic health records (EHRs) (1). Clinical decision support alerts can save lives, but frequent ones can cause increased cognitive burden to clinicians, worsen alert fatigue, and increase the duplication of tests. This ultimately increases health care costs without refining patient outcomes. Studies show that 49–96% of clinical alerts are ignored, raising questions about the effectiveness of CDSS (1). Blockchain, a decentralized, distributed digital ledger that contains a plethora of continuously updated, time-stamped, and highly encrypted virtual record, can be a key to addressing these challenges (2). The blockchain technology if integrated with the CDSS can serve as a potential solution to eliminating current drawbacks with CDSS (3). This article addresses the most significant and chronic problems facing the successful implementation of CDSS and how leveraging the Hyperledger Fabric can alleviate the clinical alert fatigue and reduce physician’s burnout using patient-specific information. The proposed architecture framework for this study is designed to equip the CDSS with overall patient information at the point of care. This then empowers the physicians with the blockchain-integrated CDSS, which holds the potential to reduce clinician’s cognitive burden, medical errors, and costs and ultimately enhance patient outcomes. The research study broadly discusses how the blockchain technology can be a potential solution, reasons for selecting the Hyperledger Fabric, and elaborates on how the Hyperledger Fabric can be leveraged to enhance the efficacy of CDSS.

Machine Learning Approach to Reduce Alert Fatigue Using a Disease Medication–Related Clinical Decision Support System: Model Development and Validation

Background Computerized physician order entry (CPOE) systems are incorporated into clinical decision support systems (CDSSs) to reduce medication errors and improve patient safety. Automatic alerts generated from CDSSs can directly assist physicians in making useful clinical decisions and can help shape prescribing behavior. Multiple studies reported that approximately 90%-96% of alerts are overridden by physicians, which raises questions about the effectiveness of CDSSs. There is intense interest in developing sophisticated methods to combat alert fatigue, but there is no consensus on the optimal approaches so far. Objective Our objective was to develop machine learning prediction models to predict physicians’ responses in order to reduce alert fatigue from disease medication–related CDSSs. Methods We collected data from a disease medication–related CDSS from a university teaching hospital in Taiwan. We considered prescriptions that triggered alerts in the CDSS between August 2018 and May 2019. Machine learning models, such as artificial neural network (ANN), random forest (RF), naïve Bayes (NB), gradient boosting (GB), and support vector machine (SVM), were used to develop prediction models. The data were randomly split into training (80%) and testing (20%) datasets. Results A total of 6453 prescriptions were used in our model. The ANN machine learning prediction model demonstrated excellent discrimination (area under the receiver operating characteristic curve [AUROC] 0.94; accuracy 0.85), whereas the RF, NB, GB, and SVM models had AUROCs of 0.93, 0.91, 0.91, and 0.80, respectively. The sensitivity and specificity of the ANN model were 0.87 and 0.83, respectively. Conclusions In this study, ANN showed substantially better performance in predicting individual physician responses to an alert from a disease medication–related CDSS, as compared to the other models. To our knowledge, this is the first study to use machine learning models to predict physician responses to alerts; furthermore, it can help to develop sophisticated CDSSs in real-world clinical settings.