Preliminary Assessment of an Automated Surveillance System for Infection Control

2004 ◽  
Vol 25 (4) ◽  
pp. 325-332 ◽  
Author(s):  
Marc-Oliver Wright ◽  
Eli N. Perencevich ◽  
Christopher Novak ◽  
Joan N. Hebden ◽  
Harold C. Standiford ◽  
...  
Keyword(s):  
Positive Predictive Value ◽  
Infection Control ◽  
Predictive Value ◽  
Surveillance System ◽  
Control Charts ◽  
Medical Center ◽  
Academic Medical Center ◽  
Kappa Score ◽  
Efficient Manner ◽  
Automated Surveillance

AbstractBackground and Objective:Rapid identification and investigation of potential outbreaks is key to limiting transmission in the healthcare setting. Manual review of laboratory results remains a cumbersome, time-consuming task for infection control practitioners (ICPs). Computer-automated techniques have shown promise for improving the efficiency and accuracy of surveillance. We examined the use of automated control charts, provided by an automated surveillance system, for detection of potential outbreaks.Setting:A 656-bed academic medical center.Methods:We retrospectively reviewed 13 months (November 2001 through November 2002) of laboratory-patient data, comparing an automated surveillance application with standard infection control practices. We evaluated positive predictive value, sensitivity, and time required to investigate the alerts. An ICP created 75 control charts. A standardized case investigation form was developed to evaluate each alert for the likelihood of nosocomial transmission based on temporal and spatial overlap and culture results.Results:The 75 control charts were created in 75 minutes and 18 alerts fired above the 3-sigma level. These were independently reviewed by an ICP and associate hospital epidemiologist. The review process required an average of 20 minutes per alert and the kappa score between the reviewers was 0.82. Eleven of the 18 alerts were determined to be potential outbreaks, yielding a positive predictive value of 0.61. Routine surveillance identified 5 of these 11 alerts during this time period.Conclusion:Automated surveillance with user-definable control charts for cluster identification was more sensitive than routine methods and is capable of operating with high specificity and positive predictive value in a time-efficient manner.

scholarly journals Development, implementation, and prospective validation of a model to predict 60-day end-of-life in hospitalized adults upon admission at three sites

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Vincent J. Major ◽  
Yindalon Aphinyanaphongs
Keyword(s):  
Positive Predictive Value ◽  
Receiver Operating Characteristic ◽  
Predictive Value ◽  
Operating Characteristic ◽  
Medical Center ◽  
Academic Medical Center ◽  
Model Performance ◽  
Risk Of Death ◽  
Temporal Validation ◽  
Receiver Operating

Abstract Background Automated systems that use machine learning to estimate a patient’s risk of death are being developed to influence care. There remains sparse transparent reporting of model generalizability in different subpopulations especially for implemented systems. Methods A prognostic study included adult admissions at a multi-site, academic medical center between 2015 and 2017. A predictive model for all-cause mortality (including initiation of hospice care) within 60 days of admission was developed. Model generalizability is assessed in temporal validation in the context of potential demographic bias. A subsequent prospective cohort study was conducted at the same sites between October 2018 and June 2019. Model performance during prospective validation was quantified with areas under the receiver operating characteristic and precision recall curves stratified by site. Prospective results include timeliness, positive predictive value, and the number of actionable predictions. Results Three years of development data included 128,941 inpatient admissions (94,733 unique patients) across sites where patients are mostly white (61%) and female (60%) and 4.2% led to death within 60 days. A random forest model incorporating 9614 predictors produced areas under the receiver operating characteristic and precision recall curves of 87.2 (95% CI, 86.1–88.2) and 28.0 (95% CI, 25.0–31.0) in temporal validation. Performance marginally diverges within sites as the patient mix shifts from development to validation (patients of one site increases from 10 to 38%). Applied prospectively for nine months, 41,728 predictions were generated in real-time (median [IQR], 1.3 [0.9, 32] minutes). An operating criterion of 75% positive predictive value identified 104 predictions at very high risk (0.25%) where 65% (50 from 77 well-timed predictions) led to death within 60 days. Conclusion Temporal validation demonstrates good model discrimination for 60-day mortality. Slight performance variations are observed across demographic subpopulations. The model was implemented prospectively and successfully produced meaningful estimates of risk within minutes of admission.

scholarly journals 362. A Modified Early Warning Score Predicts Decompensation in COVID-19 Patients

2020 ◽  
Vol 7 (Supplement_1) ◽  
pp. S251-S251
Author(s):  
Joanna S Cavalier ◽  
Benjamin Goldstein ◽  
Cara L O’Brien ◽  
Armando Bedoya
Keyword(s):  
Positive Predictive Value ◽  
Early Warning ◽  
Predictive Value ◽  
Risk Model ◽  
Medical Center ◽  
Academic Medical Center ◽  
Significant Proportion ◽  
Case Series ◽  
Early Warning Score ◽  
Severe Illness

Abstract Background The novel coronavirus disease (COVID-19) results in severe illness in a significant proportion of patients, necessitating a way to discern which patients will become critically ill and which will not. In one large case series, 5.0% of patients required an intensive care unit (ICU) and 1.4% died. Several models have been developed to assess decompensating patients. However, research examining their applicability to COVID-19 patients is limited. An accurate predictive model for patients at risk of decompensation is critical for health systems to optimally triage emergencies, care for patients, and allocate resources. Methods An early warning score (EWS) algorithm created within a large academic medical center, with methodology previously described, was applied to COVID-19 patients admitted to this institution. 122 COVID-19 patients were included. A decompensation event was defined as inpatient mortality or an unanticipated transfer to an ICU from an intermediate medical ward. The EWS was calculated at 12-hour and 24-hour intervals. Results Of 122 patients admitted with COVID-19, 28 had a decompensation event, yielding an event rate of 23.0%. 8 patients died, 13 transferred to the ICU, and 6 both transferred to the ICU and died. Decompensation within 12 and 24 hours were predicted with areas under the curve (AUC) of 0.850 and 0.817, respectively. Using a three-tiered risk model, use of the customized EWS score for patients identified as high risk of decompensation had a positive predictive value of 44.4% and 11.1% and specificity of 99.3% and 99.6% and 12- and 24-hour intervals. Amongst medium-risk patients, the score had a specificity of 85.0% and 85.4%, respectively. Conclusion This EWS allows for prediction of decompensation, defined as transfer to an ICU or death, in COVID-19 patients with excellent specificity and a high positive predictive value. Clinically, implementation of this score can help to identify patients before they decompensate in order to triage at time of presentation and allocate step-down beds, ICU beds, and treatments such as remdesivir. Disclosures All Authors: No reported disclosures

Validation of Statewide Surveillance System Data on Central Line–Associated Bloodstream Infection in Intensive Care Units in Australia

2009 ◽  
Vol 30 (11) ◽  
pp. 1045-1049 ◽  
Author(s):  
Emma S. McBryde ◽  
Judy Brett ◽  
Philip L. Russo ◽  
Leon J. Worth ◽  
Ann L. Bull ◽  
...  
Keyword(s):  
Intensive Care ◽  
Positive Predictive Value ◽  
Infection Control ◽  
Negative Predictive Value ◽  
Predictive Value ◽  
Gold Standard ◽  
Surveillance System ◽  
Central Line ◽  
Public Hospitals ◽  
Definition Of

Objective.To measure the interobserver agreement, sensitivity, specificity, positive predictive value, and negative predictive value of data submitted to a statewide surveillance system for identifying central line-associated bloodstream infection (BSI).Design.Retrospective review of hospital medical records comparing reported data with gold standard according to definitions of central line–associated BSI.Setting.Six Victorian public hospitals with more than 100 beds.Methods.Reporting of surveillance outcomes was undertaken by infection control practitioners at the hospital sites. Retrospective evaluation of the surveillance process was carried out by independent infection control practitioners from the Victorian Hospital Acquired Infection Surveillance System (VICNISS). A sample of records of patients reported to have a central line-associated BSI were assessed to determine whether they met the definition of central line–associated BSI. A sample of records of patients with bacteremia in the intensive care unit during the assessment period who were not reported as having central line–associated BSI were also assessed to see whether they met the definition of central line-associated BSI.Results.Records of 108 patients were reviewed; the agreement between surveillance reports and the VICNISS assessment was 67.6% (κ = 0.31). Of the 46 reported central line–associated BSIs, 27 were confirmed to be central line–associated BSIs, for a positive predictive value of 59% (95% confidence interval [CI], 43%–73%). Of the 62 cases of bacteremia reviewed that were not reported as central line–associated BSIs, 45 were not associated with a central line, for a negative predictive value of 73% (95% CI, 60%–83%). Estimated sensitivity was 35%, and specificity was 87%. The positive likelihood ratio was 3.0, and the negative likelihood ratio was 0.72.Discussion.The agreement between the reporting of central line–associated BSI and the gold standard application of definitions was unacceptably low. False-negative results were problematic; more than half of central line–associated BSIs may be missed in Victorian public hospitals.

scholarly journals HAI-Proactive: Development of an Automated Surveillance System for Healthcare-Associated Infections in Sweden

2020 ◽  
Vol 41 (S1) ◽  
pp. s39-s39
Author(s):  
Pontus Naucler ◽  
Suzanne D. van der Werff ◽  
John Valik ◽  
Logan Ward ◽  
Anders Ternhag ◽  
...  
Keyword(s):  
Positive Predictive Value ◽  
Electronic Health Record ◽  
Predictive Value ◽  
Surveillance System ◽  
Free Text ◽  
Health Record ◽  
Electronic Health Record Data ◽  
Record Data ◽  
Electronic Health ◽  
Healthcare Associated

Background: Healthcare-associated infection (HAI) surveillance is essential for most infection prevention programs and continuous epidemiological data can be used to inform healthcare personal, allocate resources, and evaluate interventions to prevent HAIs. Many HAI surveillance systems today are based on time-consuming and resource-intensive manual reviews of patient records. The objective of HAI-proactive, a Swedish triple-helix innovation project, is to develop and implement a fully automated HAI surveillance system based on electronic health record data. Furthermore, the project aims to develop machine-learning–based screening algorithms for early prediction of HAI at the individual patient level. Methods: The project is performed with support from Sweden’s Innovation Agency in collaboration among academic, health, and industry partners. Development of rule-based and machine-learning algorithms is performed within a research database, which consists of all electronic health record data from patients admitted to the Karolinska University Hospital. Natural language processing is used for processing free-text medical notes. To validate algorithm performance, manual annotation was performed based on international HAI definitions from the European Center for Disease Prevention and Control, Centers for Disease Control and Prevention, and Sepsis-3 criteria. Currently, the project is building a platform for real-time data access to implement the algorithms within Region Stockholm. Results: The project has developed a rule-based surveillance algorithm for sepsis that continuously monitors patients admitted to the hospital, with a sensitivity of 0.89 (95% CI, 0.85–0.93), a specificity of 0.99 (0.98–0.99), a positive predictive value of 0.88 (0.83–0.93), and a negative predictive value of 0.99 (0.98–0.99). The healthcare-associated urinary tract infection surveillance algorithm, which is based on free-text analysis and negations to define symptoms, had a sensitivity of 0.73 (0.66–0.80) and a positive predictive value of 0.68 (0.61–0.75). The sensitivity and positive predictive value of an algorithm based on significant bacterial growth in urine culture only was 0.99 (0.97–1.00) and 0.39 (0.34–0.44), respectively. The surveillance system detected differences in incidences between hospital wards and over time. Development of surveillance algorithms for pneumonia, catheter-related infections and Clostridioides difficile infections, as well as machine-learning–based models for early prediction, is ongoing. We intend to present results from all algorithms. Conclusions: With access to electronic health record data, we have shown that it is feasible to develop a fully automated HAI surveillance system based on algorithms using both structured data and free text for the main healthcare-associated infections.Funding: Sweden’s Innovation Agency and Stockholm County CouncilDisclosures: None

Abstract P389: External Validation of the 2CAN Score for Inpatient Stroke Detection

Stroke ◽  
2021 ◽  
Vol 52 (Suppl_1) ◽  
Author(s):  
oleg otlivanchik ◽  
Jenny Lu ◽  
Natalie Cheng ◽  
Daniel L Labovitz ◽  
charles esenwa ◽  
...  
Keyword(s):  
Logistic Regression ◽  
Negative Predictive Value ◽  
Predictive Value ◽  
Medical Center ◽  
Academic Medical Center ◽  
External Validation ◽  
Acute Stroke Care ◽  
Metabolic Encephalopathy ◽  
Cardiac Procedure ◽  
Academic Medical

Introduction: Up to 15% of all strokes occur in patients who are already hospitalized for other conditions. A validated clinical tool to help rapidly discriminate between mimics and stroke among inpatients could greatly improve acute stroke care. Recently, the 2CAN score was developed and validated at a single Midwest academic medical center to identify inpatient strokes; a score of ≥2 was highly sensitive and specific for stroke. We sought to externally validate the 2CAN score at our institution. Methods: We conducted a retrospective cohort study of consecutive inpatient stroke codes at a single Northeast academic medical center from 7/1/2018 to 11/1/2019. Pre-specified variables, including patient demographics, vascular risk factors, and clinical features (neurological examination, vital signs, laboratory values, and final diagnoses), were abstracted from the electronic medical record. We determined the sensitivity, specificity, positive and negative predictive value of a 2CAN score ≥2 for stroke (ischemic stroke, hemorrhagic stroke, or TIA) in our cohort. The 2CAN score consists of clinical deficit score (0-3 points), recent cardiac procedure (1 point), atrial fibrillation (1 point), and code called within 24 hours of admission (1 point). We used multivariate logistic regression to identify additional determinants of stroke. Results: We identified 111 inpatient stroke codes on 110 patients, mean age 67 ± 1 year, 46.8% women, and 73.8% Black or Hispanic. Final diagnosis was stroke for 54 codes (48.6%) and mimic for 57 codes (51.3%), most commonly toxic-metabolic encephalopathy. 2CAN score ≥2 had 96.3% sensitivity, 45.6% specificity, 62.7% positive predictive value, and 92.3% negative predictive value for stroke. In a multivariable logistic regression model, only recent cardiac procedure (OR: 5.5; 95% CI: 1.1-27.5) and high clinical deficit score (OR: 3.9; 95% CI: 1.9-6.1) predicted stroke. Conclusion: The 2CAN score is externally valid and helps distinguish stroke from mimic in inpatients; having a score of <2 makes stroke very unlikely.

scholarly journals Utility of Routine Genomic Sequencing for Infection Control Surveillance

2017 ◽  
Vol 4 (suppl_1) ◽  
pp. S35-S35
Author(s):  
Richard T Ellison ◽  
Andrew Hoss ◽  
Jomol Mathew ◽  
Jeff Halperin ◽  
Brian Gross ◽  
...  
Keyword(s):  
Infection Control ◽  
Medical Center ◽  
Academic Medical Center ◽  
Microbial Pathogens ◽  
Genomic Sequencing ◽  
Sequencing Analysis ◽  
Nucleotide Polymorphisms ◽  
Philips Healthcare ◽  
Recent Transmission ◽  
Research Grant

Abstract Background Recent work indicates that comprehensive genomic sequencing can be a highly effective tool in defining the transmission of microbial pathogens. We have studied the utility of the routine use of genomic sequencing for infection control surveillance in an academic medical center. Methods The genomes of inpatient and emergency department isolates of Staphylococcus aureus, Pseudomonas aeruginosa, Klebsiella pneumoniae, and Enterococcus faecium were sequenced. Within each species, single-nucleotide polymorphisms (SNP) were identified in the core genome for all isolates using alignment-based methods. The number of SNP differences between isolate pairs was determined and used, in combination with the patient’s electronic medical records to identify potential transmission events. Results Between September 2016 and March 2017, 388 S. aureus, 66 P. aeruginosa, 48 K. pneumoniae, and 29 E. faecium isolates were sequenced from 373 patients. There was variation in the distribution of SNP differences between intrapatient isolates for the four pathogens; with the least variability for E. faecium and greatest for P. aeruginosa. The majority of the bacterial isolates from separate patients appeared to be genetically unique exhibiting marked SNP differences from other isolates. There were 19 sets of isolates where the SNP variation between interpatient isolates was either comparable to that of intrapatient variation (12) and suggestive of recent transmission events, or with SNP variation somewhat greater than the intrapatient SNP variation (7) suggesting relative relatedness. Only one of the highly related sets had been previously identified by standard infection control surveillance. Likely transmissions appeared to have occurred both in the inpatient and outpatient settings, and the transmission routes were not always apparent. Conclusion The routine use of genomic sequencing analysis identified previously unrecognized likely transmission events within the institution’s patient population that are of relevance to infection control surveillance. This capacity should significantly enhance our understanding of the epidemiology of hospital acquired infections, and assist in developing and implementing new prevention strategies. Disclosures R. T. Ellison III, Philips Healthcare: Consultant and Grant Investigator, Consulting fee and Research grant; A. Hoss, Philips: Employee, Salary; J. Mathew, Philips Healthcare: Investigator, Research grant; J. Halperin, Philips Healthcare: Employee and Shareholder, Salary; B. Gross, Philips: Employee and Shareholder, Salary; D. V. Ward, Philips Healthcare: Consultant, Investigator and Research Contractor, Consulting fee, Research support and Salary

Accuracy of Electronic Surveillance of Catheter-Associated Urinary Tract Infection at an Academic Medical Center

2014 ◽  
Vol 35 (06) ◽  
pp. 685-691 ◽  
Author(s):  
H. L. Wald ◽  
B. Bandle ◽  
A. Richard ◽  
S. Min
Keyword(s):  
Urinary Tract ◽  
Diagnostic Accuracy ◽  
Predictive Value ◽  
Medical Center ◽  
Academic Medical Center ◽  
University Hospital ◽  
Reference Standard ◽  
Electronic Surveillance ◽  
Tract Infections ◽  
Sensitivity Specificity

Objective.To develop and validate a methodology for electronic surveillance of catheter-associated urinary tract infections (CAUTIs).Design.Diagnostic accuracy study.Setting.A 425-bed university hospital.Subjects.A total of 1,695 unique inpatient encounters from November 2009 through November 2010 with a high clinical suspicion of CAUTI.Methods.An algorithm was developed to identify incident CAUTIs from electronic health records (EHRs) on the basis of the Centers for Disease Control and Prevention (CDC) surveillance definition. CAUTIs identified by electronic surveillance were compared with the reference standard of manual surveillance by infection preventionists. To determine diagnostic accuracy, we created 2 × 2 tables, one unadjusted and one adjusted for misclassification using chart review and case adjudication. Unadjusted and adjusted test statistics (percent agreement, sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], and κ) were calculated.Results.Electronic surveillance identified 64 CAUTIs compared with manual surveillance, which identified 19 CAUTIs for 97% agreement, 79% sensitivity, 97% sensitivity, 23% PPV, 100% NPV, and κ of .33. Compared with the reference standard adjusted for misclassification, which identified 55 CAUTIs, electronic surveillance had 98% agreement, 80% sensitivity, 99% specificity, 69% PPV, 99% NPV, and κ of .71.Conclusion.The electronic surveillance methodology had a high NPV and a low PPV compared with the reference standard, indicating a role of the electronic algorithm in screening data sets to exclude cases. However, the PPV markedly improved compared with the reference standard adjusted for misclassification, suggesting a future role in surveillance with improvements in EHRs.Infect Control Hosp Epidemiol2014;35(6):685–691

scholarly journals Comparison of early warning scores for sepsis early identification and prediction in the general ward setting

JAMIA Open ◽  
2021 ◽  
Vol 4 (3) ◽  
Author(s):  
Sean C Yu ◽  
Nirmala Shivakumar ◽  
Kevin Betthauser ◽  
Aditi Gupta ◽  
Albert M Lai ◽  
...  
Keyword(s):  
Positive Predictive Value ◽  
Early Warning ◽  
Predictive Value ◽  
Early Identification ◽  
Academic Medical Center ◽  
Scoring Systems ◽  
General Ward ◽  
Early Warning Score ◽  
Patient Acuity ◽  
Early Warning Scores

Abstract The objective of this study was to directly compare the ability of commonly used early warning scores (EWS) for early identification and prediction of sepsis in the general ward setting. For general ward patients at a large, academic medical center between early-2012 and mid-2018, common EWS and patient acuity scoring systems were calculated from electronic health records (EHR) data for patients that both met and did not meet Sepsis-3 criteria. For identification of sepsis at index time, National Early Warning Score 2 (NEWS 2) had the highest performance (area under the receiver operating characteristic curve: 0.803 [95% confidence interval [CI]: 0.795–0.811], area under the precision recall curves: 0.130 [95% CI: 0.121–0.140]) followed NEWS, Modified Early Warning Score, and quick Sequential Organ Failure Assessment (qSOFA). Using validated thresholds, NEWS 2 also had the highest recall (0.758 [95% CI: 0.736–0.778]) but qSOFA had the highest specificity (0.950 [95% CI: 0.948–0.952]), positive predictive value (0.184 [95% CI: 0.169–0.198]), and F1 score (0.236 [95% CI: 0.220–0.253]). While NEWS 2 outperformed all other compared EWS and patient acuity scores, due to the low prevalence of sepsis, all scoring systems were prone to false positives (low positive predictive value without drastic sacrifices in sensitivity), thus leaving room for more computationally advanced approaches.

Sign in / Sign up

Export Citation Format

Share Document