scholarly journals 17. Comparative Assessment of a Machine Learning Model and Rectal Swab Surveillance to Predict Hospital Onset Clostridioides difficile

2021 ◽  
Vol 8 (Supplement_1) ◽  
pp. S12-S12
Author(s):  
Erkin Ötleş ◽  
Jeeheh Oh ◽  
Alieysa Patel ◽  
Micah Keidan ◽  
Vincent B Young ◽  
...  
Keyword(s):  
Machine Learning ◽  
Predictive Value ◽  
Performance Metrics ◽  
Rectal Swab ◽  
Binary Classification ◽  
Risk Scores ◽  
Lower Sensitivity ◽  
Combined Approach ◽  
Clostridioides Difficile ◽  
Daily Risk

Abstract Background Hospital onset Clostridioides difficile infection (HO-CDI) is associated with significant morbidity and mortality. Screening individuals at risk could help limit transmission, however swab-based surveillance for HO-CDI is resource intensive. Applied to electronic health records (EHR) data, machine learning (ML) models present an efficient approach to assess patient risk. We compare the effectiveness of swab surveillance against daily risk estimates produced by a ML model in detecting patients who will develop HO-CDI. Methods Patients presenting to Michigan Medicine’s ICUs and oncology wards between June 6th and October 8th 2020 had rectal swabs collected on admission, weekly, and at discharge from the unit, as part of VRE surveillance. We performed anaerobic culture on the residual media followed by a custom, multiplex PCR on isolates to identify toxigenic C. difficile. Risk of HO-CDI was calculated daily for each patient using a previously validated EHR-based ML model. Swab results and model risk scores were aggregated for each admission and assessed as predictors of HO-CDI. Holding sensitivity equal, we evaluated both approaches in terms of accuracy, specificity, and positive predictive value (PPV). Results Of 2,044 admissions representing 1,859 patients, 39 (1.9%) developed HO-CDI. 23.1% (95% CI: 11.1–37.8%) of HO-CDI cases had at least one positive swab. At this sensitivity, model performance was significantly better than random but worse compared to swab surveillance—accuracy: 87.5% (86.0–88.9%) vs. 94.3% (93.3–95.3%), specificity: 88.7% (87.3–90.0%) vs. 95.7% (94.8–96.6%), PPV: 3.8% (1.6–6.4%) vs. 9.4% (4.3–16.1%). Combining swab AND model yielded lower sensitivity 2.6% (0.0–8.9%) compared to combining swab OR model at 43.6% (27.3–60.0%), and yielded PPV 7.1% (0.0–25.0%) vs. 43.6% (27.3–60.0%) respectively (Figure 1). Figure 1. Surveillance & risk score performance. Binary classification performance metrics of ML model (Model), toxigenic C. difficile rectal swab surveillance (Swab), and combination approaches (Model AND Swab and Model OR Swab), reported in terms of percentage points. Bold numbers highlight the best performing approach for a given performance metric. The combined approach of monitoring the Model AND Swab yielded the highest accuracy 97.5% (95% confidence interval: 96.8%, 98.1%), it also had the highest specificity 99.4% (99.0%, 99.7%). The combined approach of monitoring the Model OR Swab yielded the highest sensitivity 43.6% (27.3%, 60.0%) and negative predictive value (NPV) 98.7% (98.2, 99.2%). Using the Swab alone yielded the highest PPV 9.4% (4.3%, 16.1%) and F1 score 13.3% (6.2%, 21.8%). These results highlight the complementarity of the model and swab-based approaches. Conclusion Compared to swab surveillance using a ML model for predicting HO-CDI results in more false positives. The ML model provides daily risk scores and can be deployed using different thresholds. Thus, it can inform varied prevention strategies for different risk categories, without the need for resource intensive swabbing. Additionally, the approaches may be complimentary as the patients with HO-CDI identified by each approach differ. Disclosures Vincent B. Young, MD, PhD, American Society for Microbiology (Other Financial or Material Support, Senior Editor for mSphere)Vedanta Biosciences (Consultant) Krishna Rao, MD, MS, Bio-K+ International, Inc. (Consultant)Merck & Co., Inc. (Grant/Research Support)Roche Molecular Systems, Inc. (Consultant)Seres Therapeutics (Consultant)

scholarly journals 2409. External Validation and Comparison of Clostridioides difficile Severity Scoring Systems

2019 ◽  
Vol 6 (Supplement_2) ◽  
pp. S831-S832
Author(s):  
Donald A Perry ◽  
Daniel Shirley ◽  
Dejan Micic ◽  
Rosemary K B Putler ◽  
Pratish Patel ◽  
...  
Keyword(s):  
Binary Classification ◽  
Characteristic Curve ◽  
External Validation ◽  
Receiver Operator Characteristic Curve ◽  
Adverse Outcomes ◽  
Risk Scores ◽  
Composite Outcome ◽  
Clostridioides Difficile ◽  
Severity Scores ◽  
Primary Composite Outcome

Abstract Background Annually in the US alone, Clostridioides difficile infection (CDI) afflicts nearly 500,000 patients causing 29,000 deaths. Since early and aggressive interventions could save lives but are not optimally deployed in all patients, numerous studies have published predictive models for adverse outcomes. These models are usually developed at a single institution, and largely are not externally validated. This aim of this study was to validate the predictability for severe CDI with previously published risk scores in a multicenter cohort of patients with CDI. Methods We conducted a retrospective study on four separate inpatient cohorts with CDI from three distinct sites: the Universities of Michigan (2010–2012 and 2016), Chicago (2012), and Wisconsin (2012). The primary composite outcome was admission to an intensive care unit, colectomy, and/or death attributed to CDI within 30 days of positive test. Structured query and manual chart review abstracted data from the medical record at each site. Published CDI severity scores were assessed and compared with each other and the IDSA guideline definition of severe CDI. Sensitivity, specificity, area under the receiver operator characteristic curve (AuROC), precision-recall curves, and net reclassification index (NRI) were calculated to compare models. Results We included 3,775 patients from the four cohorts (Table 1) and evaluated eight severity scores (Table 2). The IDSA (baseline comparator) model showed poor performance across cohorts(Table 3). Of the binary classification models, including those that were most predictive of the primary composite outcome, Jardin, performed poorly with minimal to no NRI improvement compared with IDSA. The continuous score models, Toro and ATLAS, performed better, but the AuROC varied by site by up to 17% (Table 3). The Gujja model varied the most: from most predictive in the University of Michigan 2010–2012 cohort to having no predictive value in the 2016 cohort (Table 3). Conclusion No published CDI severity score showed stable, acceptable predictive ability across multiple cohorts/institutions. To maximize performance and clinical utility, future efforts should focus on a multicenter-derived and validated scoring system, and/or incorporate novel biomarkers. Disclosures All authors: No reported disclosures.

scholarly journals A machine learning approach to predict ethnicity using personal name and census location in Canada

PLoS ONE ◽  
2020 ◽  
Vol 15 (11) ◽  
pp. e0241239
Author(s):  
Kai On Wong ◽  
Osmar R. Zaïane ◽  
Faith G. Davis ◽  
Yutaka Yasui
Keyword(s):  
Machine Learning ◽  
First Nations ◽  
Predictive Value ◽  
Large Scale ◽  
Performance Metrics ◽  
Characteristic Curve ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Learning Approach ◽  
Machine Learning Approach

Background Canada is an ethnically-diverse country, yet its lack of ethnicity information in many large databases impedes effective population research and interventions. Automated ethnicity classification using machine learning has shown potential to address this data gap but its performance in Canada is largely unknown. This study conducted a large-scale machine learning framework to predict ethnicity using a novel set of name and census location features. Methods Using census 1901, the multiclass and binary class classification machine learning pipelines were developed. The 13 ethnic categories examined were Aboriginal (First Nations, Métis, Inuit, and all-combined)), Chinese, English, French, Irish, Italian, Japanese, Russian, Scottish, and others. Machine learning algorithms included regularized logistic regression, C-support vector, and naïve Bayes classifiers. Name features consisted of the entire name string, substrings, double-metaphones, and various name-entity patterns, while location features consisted of the entire location string and substrings of province, district, and subdistrict. Predictive performance metrics included sensitivity, specificity, positive predictive value, negative predictive value, F1, Area Under the Curve for Receiver Operating Characteristic curve, and accuracy. Results The census had 4,812,958 unique individuals. For multiclass classification, the highest performance achieved was 76% F1 and 91% accuracy. For binary classifications for Chinese, French, Italian, Japanese, Russian, and others, the F1 ranged 68–95% (median 87%). The lower performance for English, Irish, and Scottish (F1 ranged 63–67%) was likely due to their shared cultural and linguistic heritage. Adding census location features to the name-based models strongly improved the prediction in Aboriginal classification (F1 increased from 50% to 84%). Conclusions The automated machine learning approach using only name and census location features can predict the ethnicity of Canadians with varying performance by specific ethnic categories.

scholarly journals A Study of One-Class Classification Algorithms for Wearable Fall Sensors

Biosensors ◽  
2021 ◽  
Vol 11 (8) ◽  
pp. 284
Author(s):  
José Antonio Santoyo-Ramón ◽  
Eduardo Casilari ◽  
José Manuel Cano-García
Keyword(s):  
Machine Learning ◽  
Performance Metrics ◽  
Inertial Sensors ◽  
Binary Classification ◽  
Fall Detection ◽  
Supervised Machine Learning ◽  
Single Type ◽  
False Alarms ◽  
Mobility Patterns ◽  
Learning Mechanisms

In recent years, the popularity of wearable devices has fostered the investigation of automatic fall detection systems based on the analysis of the signals captured by transportable inertial sensors. Due to the complexity and variety of human movements, the detection algorithms that offer the best performance when discriminating falls from conventional Activities of Daily Living (ADLs) are those built on machine learning and deep learning mechanisms. In this regard, supervised machine learning binary classification methods have been massively employed by the related literature. However, the learning phase of these algorithms requires mobility patterns caused by falls, which are very difficult to obtain in realistic application scenarios. An interesting alternative is offered by One-Class Classifiers (OCCs), which can be exclusively trained and configured with movement traces of a single type (ADLs). In this paper, a systematic study of the performance of various typical OCCs (for diverse sets of input features and hyperparameters) is performed when applied to nine public repositories of falls and ADLs. The results show the potentials of these classifiers, which are capable of achieving performance metrics very similar to those of supervised algorithms (with values for the specificity and the sensitivity higher than 95%). However, the study warns of the need to have a wide variety of types of ADLs when training OCCs, since activities with a high degree of mobility can significantly increase the frequency of false alarms (ADLs identified as falls) if not considered in the data subsets used for training.

D-Dimer Test and Diagnosis of Deep Vein Thrombosis: A Comparative Study of 7 Assays

1996 ◽  
Vol 76 (04) ◽  
pp. 518-522 ◽  
Author(s):  
A Elias ◽  
I Aptel ◽  
B Huc ◽  
J J Chale ◽  
F Nguyen ◽  
...  
Keyword(s):  
Deep Vein Thrombosis ◽  
Negative Predictive Value ◽  
Predictive Value ◽  
Latex Agglutination ◽  
Lower Limbs ◽  
First Episode ◽  
Lower Sensitivity ◽  
Vein Thrombosis ◽  
Deep Vein ◽  
D Dimer

SummaryThe current D-Dimer ELISA methods provide high sensitivity and negative predictive value for the diagnosis of deep vein thrombosis but these methods are not suitable for emergency or for individual determination. We have evaluated the performance of 3 newly available fast D-Dimer assays (Vidas D-Di, BioMerieux; Instant IA D-Di, Stago; Nycocard D-Dimer, Nycomed) in comparison with 3 classic ELISA methods (Stago, Organon, Behring) and a Latex agglutination technique (Stago). One-hundred-and-seventy-one patients suspected of presenting a first episode of deep vein thrombosis were investigated. A deep vein thrombosis was detected in 75 patients (43.8%) by ultrasonic duplex scanning of the lower limbs; in 11 of them the thrombi were distal and very limited in size (<2 cm). We compared the performance of the tests by calculating their sensitivity, specificity, positive and negative predictive value for different cut-off levels and by calculating the area under ROC curves. The concordance of the different methods was evaluated by calculating the kappa coefficient. The performances of the 3 classic ELISA and of the Vidas D-Di were comparable and kappa coefficients indicated a good concordance between the results provided by these assays. Their sensitivity slightly declined for detection of the very small thrombi. Instant IA D-Di had a non-significantly lower sensitivity and negative predictive value than the 4 previous assays; however its performance was excellent for out-patients. As expected, the Latex assay had too low a sensitivity and negative predictive value to be recommended. In our hands, Nycocard D-Dimer also exhibited low sensitivity and negative predictive value, which were significantly improved when the plasma samples were tested by the manufacturer. Thus significant progress has been made, allowing clinical studies to be planned to compare the safety and cost-effectiveness of D-Dimer strategy to those of the conventional methods for the diagnosis of venous thrombosis.

scholarly journals Advancing Stress Detection Methodology with Deep Learning Techniques Targeting UX Evaluation in AAL Scenarios: Applying Embeddings for Categorical Variables

Electronics ◽  
2021 ◽  
Vol 10 (13) ◽  
pp. 1550
Author(s):  
Alexandros Liapis ◽  
Evanthia Faliagka ◽  
Christos P. Antonopoulos ◽  
Georgios Keramidas ◽  
Nikolaos Voros
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
User Experience ◽  
Electrodermal Activity ◽  
Binary Classification ◽  
Research Question ◽  
Classification Problem ◽  
Categorical Variables ◽  
Stress Detection ◽  
Software Failures

Physiological measurements have been widely used by researchers and practitioners in order to address the stress detection challenge. So far, various datasets for stress detection have been recorded and are available to the research community for testing and benchmarking. The majority of the stress-related available datasets have been recorded while users were exposed to intense stressors, such as songs, movie clips, major hardware/software failures, image datasets, and gaming scenarios. However, it remains an open research question if such datasets can be used for creating models that will effectively detect stress in different contexts. This paper investigates the performance of the publicly available physiological dataset named WESAD (wearable stress and affect detection) in the context of user experience (UX) evaluation. More specifically, electrodermal activity (EDA) and skin temperature (ST) signals from WESAD were used in order to train three traditional machine learning classifiers and a simple feed forward deep learning artificial neural network combining continues variables and entity embeddings. Regarding the binary classification problem (stress vs. no stress), high accuracy (up to 97.4%), for both training approaches (deep-learning, machine learning), was achieved. Regarding the stress detection effectiveness of the created models in another context, such as user experience (UX) evaluation, the results were quite impressive. More specifically, the deep-learning model achieved a rather high agreement when a user-annotated dataset was used for validation.

scholarly journals Machine-Learning-Based Radiomics MRI Model for Survival Prediction of Recurrent Glioblastomas Treated with Bevacizumab

Diagnostics ◽  
2021 ◽  
Vol 11 (7) ◽  
pp. 1263
Author(s):  
Samy Ammari ◽  
Raoul Sallé de Chou ◽  
Tarek Assi ◽  
Mehdi Touat ◽  
Emilie Chouzenoux ◽  
...  
Keyword(s):  
Machine Learning ◽  
Therapeutic Option ◽  
Binary Classification ◽  
Progression Free Survival ◽  
Recurrent Glioblastoma ◽  
Machine Learning Algorithms ◽  
Survival Prediction ◽  
Classification Models ◽  
Angiogenic Therapy ◽  
Recurrent Gbm

Anti-angiogenic therapy with bevacizumab is a widely used therapeutic option for recurrent glioblastoma (GBM). Nevertheless, the therapeutic response remains highly heterogeneous among GBM patients with discordant outcomes. Recent data have shown that radiomics, an advanced recent imaging analysis method, can help to predict both prognosis and therapy in a multitude of solid tumours. The objective of this study was to identify novel biomarkers, extracted from MRI and clinical data, which could predict overall survival (OS) and progression-free survival (PFS) in GBM patients treated with bevacizumab using machine-learning algorithms. In a cohort of 194 recurrent GBM patients (age range 18–80), radiomics data from pre-treatment T2 FLAIR and gadolinium-injected MRI images along with clinical features were analysed. Binary classification models for OS at 9, 12, and 15 months were evaluated. Our classification models successfully stratified the OS. The AUCs were equal to 0.78, 0.85, and 0.76 on the test sets (0.79, 0.82, and 0.87 on the training sets) for the 9-, 12-, and 15-month endpoints, respectively. Regressions yielded a C-index of 0.64 (0.74) for OS and 0.57 (0.69) for PFS. These results suggest that radiomics could assist in the elaboration of a predictive model for treatment selection in recurrent GBM patients.

scholarly journals Machine learning approach for differentiating cytomegalovirus esophagitis from herpes simplex virus esophagitis

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Jung Su Lee ◽  
Jihye Yun ◽  
Sungwon Ham ◽  
Hyunjung Park ◽  
Hyunsu Lee ◽  
...  
Keyword(s):  
Machine Learning ◽  
Differential Diagnosis ◽  
Herpes Simplex Virus ◽  
Herpes Simplex ◽  
Predictive Value ◽  
Characteristic Curve ◽  
Previous History ◽  
Clinical Factor ◽  
Endoscopic Images ◽  
Simplex Virus

AbstractThe endoscopic features between herpes simplex virus (HSV) and cytomegalovirus (CMV) esophagitis overlap significantly, and hence the differential diagnosis between HSV and CMV esophagitis is sometimes difficult. Therefore, we developed a machine-learning-based classifier to discriminate between CMV and HSV esophagitis. We analyzed 87 patients with HSV esophagitis and 63 patients with CMV esophagitis and developed a machine-learning-based artificial intelligence (AI) system using a total of 666 endoscopic images with HSV esophagitis and 416 endoscopic images with CMV esophagitis. In the five repeated five-fold cross-validations based on the hue–saturation–brightness color model, logistic regression with a least absolute shrinkage and selection operation showed the best performance (sensitivity, specificity, positive predictive value, negative predictive value, accuracy, and area under the receiver operating characteristic curve: 100%, 100%, 100%, 100%, 100%, and 1.0, respectively). Previous history of transplantation was included in classifiers as a clinical factor; the lower the performance of these classifiers, the greater the effect of including this clinical factor. Our machine-learning-based AI system for differential diagnosis between HSV and CMV esophagitis showed high accuracy, which could help clinicians with diagnoses.

scholarly journals Machine Learning Methods Applied to the Prediction of Pseudo-nitzschia spp. Blooms in the Galician Rias Baixas (NW Spain)

2021 ◽  
Vol 10 (4) ◽  
pp. 199
Author(s):  
Francisco M. Bellas Aláez ◽  
Jesus M. Torres Palenzuela ◽  
Evangelos Spyrakos ◽  
Luis González Vilas
Keyword(s):  
Machine Learning ◽  
Performance Metrics ◽  
Prediction Models ◽  
Support Vector ◽  
False Alarms ◽  
Learning Approaches ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Rías Baixas ◽  
New Algorithms

This work presents new prediction models based on recent developments in machine learning methods, such as Random Forest (RF) and AdaBoost, and compares them with more classical approaches, i.e., support vector machines (SVMs) and neural networks (NNs). The models predict Pseudo-nitzschia spp. blooms in the Galician Rias Baixas. This work builds on a previous study by the authors (doi.org/10.1016/j.pocean.2014.03.003) but uses an extended database (from 2002 to 2012) and new algorithms. Our results show that RF and AdaBoost provide better prediction results compared to SVMs and NNs, as they show improved performance metrics and a better balance between sensitivity and specificity. Classical machine learning approaches show higher sensitivities, but at a cost of lower specificity and higher percentages of false alarms (lower precision). These results seem to indicate a greater adaptation of new algorithms (RF and AdaBoost) to unbalanced datasets. Our models could be operationally implemented to establish a short-term prediction system.

Sign in / Sign up

Export Citation Format

Share Document