17. Comparative Assessment of a Machine Learning Model and Rectal Swab Surveillance to Predict Hospital Onset Clostridioides difficile

Background Hospital onset Clostridioides difficile infection (HO-CDI) is associated with significant morbidity and mortality. Screening individuals at risk could help limit transmission, however swab-based surveillance for HO-CDI is resource intensive. Applied to electronic health records (EHR) data, machine learning (ML) models present an efficient approach to assess patient risk. We compare the effectiveness of swab surveillance against daily risk estimates produced by a ML model in detecting patients who will develop HO-CDI. Methods Patients presenting to Michigan Medicine’s ICUs and oncology wards between June 6th and October 8th 2020 had rectal swabs collected on admission, weekly, and at discharge from the unit, as part of VRE surveillance. We performed anaerobic culture on the residual media followed by a custom, multiplex PCR on isolates to identify toxigenic C. difficile. Risk of HO-CDI was calculated daily for each patient using a previously validated EHR-based ML model. Swab results and model risk scores were aggregated for each admission and assessed as predictors of HO-CDI. Holding sensitivity equal, we evaluated both approaches in terms of accuracy, specificity, and positive predictive value (PPV). Results Of 2,044 admissions representing 1,859 patients, 39 (1.9%) developed HO-CDI. 23.1% (95% CI: 11.1–37.8%) of HO-CDI cases had at least one positive swab. At this sensitivity, model performance was significantly better than random but worse compared to swab surveillance—accuracy: 87.5% (86.0–88.9%) vs. 94.3% (93.3–95.3%), specificity: 88.7% (87.3–90.0%) vs. 95.7% (94.8–96.6%), PPV: 3.8% (1.6–6.4%) vs. 9.4% (4.3–16.1%). Combining swab AND model yielded lower sensitivity 2.6% (0.0–8.9%) compared to combining swab OR model at 43.6% (27.3–60.0%), and yielded PPV 7.1% (0.0–25.0%) vs. 43.6% (27.3–60.0%) respectively (Figure 1). Figure 1. Surveillance & risk score performance. Binary classification performance metrics of ML model (Model), toxigenic C. difficile rectal swab surveillance (Swab), and combination approaches (Model AND Swab and Model OR Swab), reported in terms of percentage points. Bold numbers highlight the best performing approach for a given performance metric. The combined approach of monitoring the Model AND Swab yielded the highest accuracy 97.5% (95% confidence interval: 96.8%, 98.1%), it also had the highest specificity 99.4% (99.0%, 99.7%). The combined approach of monitoring the Model OR Swab yielded the highest sensitivity 43.6% (27.3%, 60.0%) and negative predictive value (NPV) 98.7% (98.2, 99.2%). Using the Swab alone yielded the highest PPV 9.4% (4.3%, 16.1%) and F1 score 13.3% (6.2%, 21.8%). These results highlight the complementarity of the model and swab-based approaches. Conclusion Compared to swab surveillance using a ML model for predicting HO-CDI results in more false positives. The ML model provides daily risk scores and can be deployed using different thresholds. Thus, it can inform varied prevention strategies for different risk categories, without the need for resource intensive swabbing. Additionally, the approaches may be complimentary as the patients with HO-CDI identified by each approach differ. Disclosures Vincent B. Young, MD, PhD, American Society for Microbiology (Other Financial or Material Support, Senior Editor for mSphere)Vedanta Biosciences (Consultant) Krishna Rao, MD, MS, Bio-K+ International, Inc. (Consultant)Merck & Co., Inc. (Grant/Research Support)Roche Molecular Systems, Inc. (Consultant)Seres Therapeutics (Consultant)

The Development of a Small and Medium-Sized Business Risk Management Intervention Tool

Risk is inevitable in business. For large companies, risk management is formalised and structured through compliance with industry standards. However, small and medium-sized businesses (SMEs) rarely have adequate resources to develop their own standards or conform to pre-established criteria. This results in an increased vulnerability to risk, which tends to undermine SMEs’ sustainability. The primary reasons for the low adoption rate of risk management are related to the tremendous initial difficulty in orientating the business concerning risk and the significant investment of the workforce in developing and implementing a structured managerial process. The objective of this paper is to produce a guided process tool for small and medium-sized businesses with which they can identify, evaluate, and appropriately address risks from an SME perspective. Moreover, this intervention would offer enhancements at no cost beyond the time of its implementation. In order to identify what constitutes holistic risk management, document analysis was applied, which utilised risk management standards, academic articles, books, and regulatory policy and strategy documentation. The identified elements were integrated with a tool that improves business owners’ capacity to position themselves in context with their daily risk management challenges.

Implementing Mandatory Testing and a Public Health Commitment to Control COVID-19 on a College Campus

The COVID-19 pandemic has greatly impacted US colleges and universities. As The George Washington University (GWU), a large urban university, prepared to reopen for the Fall 2020 semester, GWU established protocols to protect the health and wellness of all members of campus community. Reopening efforts included a cadre of COVID-19 surveillance systems including development of a public health COVID-19 laboratory, weekly and symptomatic SARS-CoV-2 testing and daily risk screening and symptom monitoring. Other activities included completion of a mandatory COVID-19 training and influenza vaccination for the on-campus population, quarantining of students returning to campus, campus-focused case investigations and quarantining of suspected close contacts, clinical follow-up of infected persons, and regular communication and monitoring. A smaller on-campus population of 4,435 students, faculty and staff returned to campus with later expansion of testing to accommodate GWU students living in the surrounding area. Between August 17 and December 4, 2020, 38,288 tests were performed; 220 were positive. The surveillance program demonstrated a relatively low positivity rate, with temporal clustering of infected persons mirroring community spread, and little evidence for transmission among the GWU on-campus population. These efforts demonstrate the feasibility of safely partially reopening a large urban college campus by applying core principles of public health surveillance, infectious disease epidemiology, behavioral measures, and increased testing capacity, while continuing to promote educational and research opportunities. GWU will continue to monitor the program as the pandemic evolves and periodically reassess to determine if these strategies will be successful upon a full return to in-person learning.

Development of a Real-Time Risk Model (RTRM) for Predicting In-Hospital COVID-19 Mortality

ABSTRACTBackgroundWith over 83 million cases and 1.8 million deaths reported worldwide by the end of 2020 for SARS-CoV-2 (COVID-19), there is an urgent need to enhance identification of high-risk populations to properly evaluate therapy effectiveness with real-world evidence and improve outcomes.MethodsBaseline and daily indicators were evaluated using electronic health records for 46,971 patients hospitalized with COVID-19 from 176 HCA Healthcare-affiliated hospitals, presenting from March to September 2020, to develop a real-time risk model (RTRM) of all-cause, hospitalized mortality. Patient facility, dates-of-care, clinico-demographics, comorbidities, vitals, laboratory markers, and respiratory support findings were aggregated in a logistic regression model.FindingsThe RTRM predicted overall mortality as well as mortality 1, 3, and 7 days in advance with an area under the receiver operating characteristic curve (AUCROC) of 0.905, 0.911, 0.905, and 0.901 respectively, significantly outperforming a combined model of age and daily modified WHO progression scale (all p<0.0001; AUCROC, 0.846, 0.848, 0.850, and 0.852). The RTRM delineated risk at presentation from ongoing risk associated with medical care and showed that mortality rates decreased over time due to both decreased severity and changes in care.InterpretationTo our knowledge, this study is the largest of its kind to comprehensively evaluate predictors and incorporate daily risk measures of COVID-19 mortality. The RTRM validates current literature trends in mortality across time and allows direct translation for research and clinical applications.Research in contextEvidence before this studyDue to the rapidly evolving nature of the COVID-19 pandemic, the body of evidence and published literature was considered prior to study initiation and throughout the course of the study. Although at study initiation there was a growing consensus that age and disease severity at presentation were the greatest contributors to predicting in-hospital mortality, there was less of a consensus on the key demographics, comorbidities, vitals and laboratory values. In addition, early on, most potential predictors of in-hospital mortality had been assessed by univariable analysis. In April of 2020, a systematic review of prediction studies for COVID-19 revealed that there were only 8 publications for prognosis of hospital mortality. All were deemed to have high potential for bias due to low sample size, model overfitting, vague reporting and/or insufficient follow-up. Over the duration of the study, in-hospital prediction models were published ranging from simplified scores to machine learning. There were at least 8 prediction studies that were published during the course of our own that had comparable sample size or extensive multivariable analysis with the greatest accuracy of prediction reported as 74%. Moreover, a report in December of 2020 independently validated 4 simple prediction models, with none achieving greater than an AUCROC of 0.72%. Lastly, an eight-variable score developed by a UK consortium on a comparable sample size demonstrated an AUCROC of 0.77. To our knowledge, however, none to-date have modeled daily risk beyond baseline.We frequently assessed World Health Organization (WHO) resources as well as queried both MedRXIV and PubMed with the search terms “COVID”, “prediction”, “hospital” and “mortality” to ensure we were assessing all potential predictors of hospitalized mortality. The last search was performed on January 5, 2021 with the addition of “multi”, “daily”, “real time” or “longitudinal” terms to confirm the novelty of our study. No date restrictions or language filters were applied.Added value of this studyTo our knowledge, this study is the largest and most geographically diverse of its kind to comprehensively evaluate predictors of in-hospital COVID-19 mortality that are available retrospectively in electronic health records and to incorporate longitudinal, daily risk measures to create risk trajectories over the entire hospital stay. Not only does our Real-Time Risk Model (RTRM) validate current literature, demonstrating reduced mortality over the course of the COVID-19 pandemic and identifying age and WHO severity as major drivers of mortality in regards to baseline characteristics, but it also outperforms a model of age and daily WHO score combined, achieving an AUCROC of 0.91 on the test set. Furthermore, the fact that the RTRM delineates risk at baseline from risk over the course of care allows more granular interpretation of the impact of various parameters on mortality risk, as demonstrated in the current study using both sex disparity and calendar epochs that were based on evolving treatment recommendations as proofs-of-principle.Implications of all the available evidenceThe goal of the RTRM was to create a flexible tool that could be used to assess intervention and treatment efficacy in real-world, evidence-based studies as well as provide real-time risk assessment to aid clinical decisions and resourcing with further development. Implications of this work are broad. The depth of the multi-facility, harmonized electronic health record (EHR) dataset coupled with the transparency we provide in the RTRM results provides a resource for others to interpret impact of markers of interest and utilize data that is relevant to their own studies. The RTRM will allow optimal matching in retrospective cohort studies and provide a more granular endpoint for evaluation of interventions beyond general effectiveness, such as optimal delivery, including dosing and timing, and identification of the population/s benefiting from an intervention or combination of interventions. In addition, beyond the scope of the current study, the RTRM and its resultant daily risk scores allow for flexibility in developing prediction models for other clinical outcomes, such as progression of pulmonary disease, need for invasive mechanical ventilation, and development of sepsis and/or multiorgan failure, all of which could provide a framework for real-time personalized care.

Index tracking strategy based on mixed-frequency financial data

To obtain market average return, investment managers need to construct index tracking portfolio to replicate target index. Currently, most literatures use financial data that has homogenous frequency when constructing the index tracking portfolio. To make up for this limitation, we propose a methodology based on mixed-frequency financial data, called FACTOR-MIDAS-POET model. The proposed model can utilize the intraday return data, daily risk factors data and monthly or quarterly macro economy data, simultaneously. Meanwhile, the out-of-sample analysis demonstrates that our model can improve the tracking accuracy.

Assessing the Risk of Legionella Infection through Showering with Untreated Rain Cistern Water in a Tropical Environment

In September 2017, two category-5 hurricanes Irma and Maria swept through the Caribbean Sea in what is now known as the region’s most active hurricane season on record, leaving disastrous effects on infrastructure and people’s lives. In the U.S. Virgin Islands, rain cisterns are commonly used for harvesting roof-top rainwater for household water needs. High prevalence of Legionella spp. was found in the cistern water after the hurricanes. This study carried out a quantitative microbial risk assessment to estimate the health risks associated with Legionella through inhalation of aerosols from showering using water from cisterns after the hurricanes. Legionella concentrations were modeled based on the Legionella detected in post-hurricane water samples and reported total viable heterotrophic bacterial counts in cistern water. The inhalation dose was modeled using a Monte Carlo simulation of shower water aerosol concentrations according to shower water temperature, shower duration, inhalation rates, and shower flow rates. The risk of infection was calculated based on a previously established dose–response model from Legionella infection of guinea pigs. The results indicated median daily risk of 2.5 × 10−6 to 2.5 × 10−4 depending on shower temperature, and median annual risk of 9.1 × 10−4 to 1.4 × 10−2. Results were discussed and compared with household survey results for a better understanding of local perceived risk versus objective risk surrounding local water supplies.

Predicting drowning from sea and weather forecasts: development and validation of a model on surf beaches of southwestern France

ObjectiveTo predict the coast-wide risk of drowning along the surf beaches of Gironde, southwestern France.MethodsData on rescues and drownings were collected from the Medical Emergency Center of Gironde (SAMU 33). Seasonality, holidays, weekends, weather and metocean conditions were considered potentially predictive. Logistic regression models were fitted with data from 2011 to 2013 and used to predict 2015–2017 events employing weather and ocean forecasts.ResultsAir temperature, wave parameters, seasonality and holidays were associated with drownings. Prospective validation was performed on 617 days, covering 232 events (rescues and drownings) reported on 104 different days. The area under the curve (AUC) of the daily risk prediction model (combined with 3-day forecasts) was 0.82 (95% CI 0.79 to 0.86). The AUC of the 3-hour step model was 0.85 (95% CI 0.81 to 0.88).ConclusionsDrowning events along the Gironde surf coast can be anticipated up to 3 days in advance. Preventative messages and rescue preparations could be increased as the forecast risk increased, especially during the off-peak season, when the number of available rescuers is low.

The Importance of Low Daily Risk for the Prediction of Treatment Events of Individual Dairy Cows with Sensor Systems

The prediction of health disorders is the goal of many sensor systems in dairy farming. Although mastitis and lameness are the most common health disorders in dairy cows, these diseases or treatments are a rare event related to a single day and cow. A number of studies already developed and evaluated models for classifying cows in need of treatment for mastitis and lameness with machine learning methods, but few have illustrated the effects of the positive predictive value (PPV) on practical application. The objective of this study was to investigate the importance of low-frequency treatments of mastitis or lameness for the applicability of these classification models in practice. Data from three German dairy farms contained animal individual sensor data (milkings, activity, feed intake) and were classified using machine learning models developed in a previous study. Subsequently, different risk criteria (previous treatments, information from milk recording, early lactation) were designed to isolate high-risk groups. Restricting selection to cows with previous mastitis or hoof treatment achieved the highest increase in PPV from 0.07 to 0.20 and 0.15, respectively. However, the known low daily risk of a treatment per cow remains the critical factor that prevents the reduction of daily false-positive alarms to a satisfactory level. Sensor systems should be seen as additional decision-support aid to the farmers’ expert knowledge.