scholarly journals External validation of two prediction tools for patients at risk for recurrent Clostridioides difficile infection

2021 ◽  
Vol 14 ◽  
pp. 175628482097738
Author(s):  
Tessel M. van Rossen ◽  
Laura J. van Dijk ◽  
Martijn W. Heymans ◽  
Olaf M. Dekkers ◽  
Christina M. J. E. Vandenbroucke-Grauls ◽  
...  
Keyword(s):  
At Risk ◽  
Validation Cohort ◽  
Characteristic Curve ◽  
External Validation ◽  
Risk Scores ◽  
Prediction Tools ◽  
Clostridioides Difficile ◽  
Clostridioides Difficile Infection ◽  
Patients At Risk ◽  
Definition Of

Background: One in four patients with primary Clostridioides difficile infection (CDI) develops recurrent CDI (rCDI). With every recurrence, the chance of a subsequent CDI episode increases. Early identification of patients at risk for rCDI might help doctors to guide treatment. The aim of this study was to externally validate published clinical prediction tools for rCDI. Methods: The validation cohort consisted of 129 patients, diagnosed with CDI between 2018 and 2020. rCDI risk scores were calculated for each individual patient in the validation cohort using the scoring tools described in the derivation studies. Per score value, we compared the average predicted risk of rCDI with the observed number of rCDI cases. Discrimination was assessed by calculating the area under the receiver operating characteristic curve (AUC). Results: Two prediction tools were selected for validation (Cobo 2018 and Larrainzar-Coghen 2016). The two derivation studies used different definitions for rCDI. Using Cobo’s definition, rCDI occurred in 34 patients (26%) of the validation cohort: using the definition of Larrainzar-Coghen, we observed 19 recurrences (15%). The performance of both prediction tools was poor when applied to our validation cohort. The estimated AUC was 0.43 [95% confidence interval (CI); 0.32–0.54] for Cobo’s tool and 0.42 (95% CI; 0.28–0.56) for Larrainzar-Coghen’s tool. Conclusion: Performance of both prediction tools was disappointing in the external validation cohort. Currently identified clinical risk factors may not be sufficient for accurate prediction of rCDI.

scholarly journals 2409. External Validation and Comparison of Clostridioides difficile Severity Scoring Systems

2019 ◽  
Vol 6 (Supplement_2) ◽  
pp. S831-S832
Author(s):  
Donald A Perry ◽  
Daniel Shirley ◽  
Dejan Micic ◽  
Rosemary K B Putler ◽  
Pratish Patel ◽  
...  
Keyword(s):  
Binary Classification ◽  
Characteristic Curve ◽  
External Validation ◽  
Receiver Operator Characteristic Curve ◽  
Adverse Outcomes ◽  
Risk Scores ◽  
Composite Outcome ◽  
Clostridioides Difficile ◽  
Severity Scores ◽  
Primary Composite Outcome

Abstract Background Annually in the US alone, Clostridioides difficile infection (CDI) afflicts nearly 500,000 patients causing 29,000 deaths. Since early and aggressive interventions could save lives but are not optimally deployed in all patients, numerous studies have published predictive models for adverse outcomes. These models are usually developed at a single institution, and largely are not externally validated. This aim of this study was to validate the predictability for severe CDI with previously published risk scores in a multicenter cohort of patients with CDI. Methods We conducted a retrospective study on four separate inpatient cohorts with CDI from three distinct sites: the Universities of Michigan (2010–2012 and 2016), Chicago (2012), and Wisconsin (2012). The primary composite outcome was admission to an intensive care unit, colectomy, and/or death attributed to CDI within 30 days of positive test. Structured query and manual chart review abstracted data from the medical record at each site. Published CDI severity scores were assessed and compared with each other and the IDSA guideline definition of severe CDI. Sensitivity, specificity, area under the receiver operator characteristic curve (AuROC), precision-recall curves, and net reclassification index (NRI) were calculated to compare models. Results We included 3,775 patients from the four cohorts (Table 1) and evaluated eight severity scores (Table 2). The IDSA (baseline comparator) model showed poor performance across cohorts(Table 3). Of the binary classification models, including those that were most predictive of the primary composite outcome, Jardin, performed poorly with minimal to no NRI improvement compared with IDSA. The continuous score models, Toro and ATLAS, performed better, but the AuROC varied by site by up to 17% (Table 3). The Gujja model varied the most: from most predictive in the University of Michigan 2010–2012 cohort to having no predictive value in the 2016 cohort (Table 3). Conclusion No published CDI severity score showed stable, acceptable predictive ability across multiple cohorts/institutions. To maximize performance and clinical utility, future efforts should focus on a multicenter-derived and validated scoring system, and/or incorporate novel biomarkers. Disclosures All authors: No reported disclosures.

Identification of patients at risk of Clostridioides difficile infection for enrollment in vaccine clinical trials

Vaccine ◽  
2021 ◽  
Vol 39 (3) ◽  
pp. 536-544
Author(s):  
Vanessa W. Stevens ◽  
Ellyn M. Russo ◽  
Yinong Young-Xu ◽  
Molly Leecaster ◽  
Yue Zhang ◽  
...  
Keyword(s):  
At Risk ◽  
Clinical Trials ◽  
Clostridioides Difficile ◽  
Clostridioides Difficile Infection ◽  
Patients At Risk

External Validation and Comparison of Clostridioides difficile Severity Scoring Systems

2021 ◽  
Author(s):  
D Alexander Perry ◽  
Daniel Shirley ◽  
Dejan Micic ◽  
C Pratish Patel ◽  
Rosemary Putler ◽  
...  
Keyword(s):  
Characteristic Curve ◽  
External Validation ◽  
Predictive Ability ◽  
Scoring Systems ◽  
Receiver Operator Characteristic Curve ◽  
Adverse Outcomes ◽  
Risk Scores ◽  
Clostridioides Difficile ◽  
Multicenter Cohort ◽  
Severity Scores

Abstract Background Many models have been developed to predict severe outcomes from Clostridioides difficile infection. These models are usually developed at a single institution and largely are not externally validated. This aim of this study was to validate previously published risk scores in a multicenter cohort of patients with CDI. Methods Retrospective study on four separate inpatient cohorts with CDI from three distinct sites: The Universities of Michigan (2010-2012 and 2016), Chicago (2012), and Wisconsin (2012). The primary composite outcome was admission to an intensive care unit, colectomy, and/or death attributed to CDI within 30 days of positive testing. Both within each cohort and combined across all cohorts, published CDI severity scores were assessed and compared to each other and the IDSA guideline definitions of severe and fulminant CDI. Results A total of 3,646 patients were included for analysis. Including the two IDSA guideline definitions, fourteen scores were assessed. Performance of scores varied within each cohort and in the combined set (mean area under the receiver operator characteristic curve(AUC 0.61, range 0.53-0.66). Only half of the scores had performance at or better than IDSA severe and fulminant definitions (AUCs 0.64 and 0.63, respectively). Most of the scoring systems had more false than true positives in the combined set (mean: 81.5%, range:0-91.5%). Conclusions No published CDI severity score showed stable, good predictive ability for adverse outcomes across multiple cohorts/institutions or in a combined multicenter cohort.

scholarly journals Risk Score for Predicting In-Hospital Mortality in COVID-19 (RIM Score)

Diagnostics ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. 596 ◽  
Author(s):  
Alejandro López-Escobar ◽  
Rodrigo Madurga ◽  
José María Castellano ◽  
Sara Velázquez ◽  
Rafael Suárez del Villar ◽  
...  
Keyword(s):  
At Risk ◽  
Hospital Mortality ◽  
Validation Cohort ◽  
Health Care Systems ◽  
Characteristic Curve ◽  
Receiver Operator Characteristic Curve ◽  
Oxygen Saturation Level ◽  
Training Cohort ◽  
Care Systems ◽  
Patients At Risk

Infection by SARS-CoV2 has devastating consequences on health care systems. It is a global health priority to identify patients at risk of fatal outcomes. 1955 patients admitted to HM-Hospitales from 1 March to 10 June 2020 due to COVID-19, were were divided into two groups, 1310 belonged to the training cohort and 645 to validation cohort. Four different models were generated to predict in-hospital mortality. Following variables were included: age, sex, oxygen saturation, level of C-reactive-protein, neutrophil-to-platelet-ratio (NPR), neutrophil-to-lymphocyte-ratio (NLR) and the rate of changes of both hemogram ratios (VNLR and VNPR) during the first week after admission. The accuracy of the models in predicting in-hospital mortality were evaluated using the area under the receiver-operator-characteristic curve (AUC). AUC for models including NLR and NPR performed similarly in both cohorts: NLR 0.873 (95% CI: 0.849–0.898), NPR 0.875 (95% CI: 0.851–0.899) in training cohort and NLR 0.856 (95% CI: 0.818–0.895), NPR 0.863 (95% CI: 0.826–0.901) in validation cohort. AUC was 0.885 (95% CI: 0.885–0.919) for VNLR and 0.891 (95% CI: 0.861–0.922) for VNPR in the validation cohort. According to our results, models are useful in predicting in-hospital mortality risk due to COVID-19. The RIM Score proposed is a simple, widely available tool that can help identify patients at risk of fatal outcomes.

scholarly journals Deep learning detects heart failure with preserved ejection fraction using a baseline electrocardiogram

2021 ◽  
Author(s):  
Matthias Unterhuber ◽  
Karl-Philipp Rommel ◽  
Karl-Patrik Kresoja ◽  
Julia Lurz ◽  
Jelena Kornej ◽  
...  
Keyword(s):  
Heart Failure ◽  
At Risk ◽  
Deep Learning ◽  
Ejection Fraction ◽  
Predictive Value ◽  
Validation Cohort ◽  
External Validation ◽  
Preserved Ejection Fraction ◽  
Derivation Cohort ◽  
Patients At Risk

Abstract Background Heart failure with preserved ejection fraction (HFpEF) is a rapidly growing global health problem. To date, diagnosis of HFpEF is based on clinical, invasive and laboratory examinations. Electrocardiographic findings may vary, and there are no known typical ECG features for HFpEF. Methods This study included two patient cohorts. In the derivation cohort, we included n = 1884 patients who presented with exertional dyspnea or equivalent and preserved ejection fraction (≥50%) and clinical suspicion for coronary artery disease. The ECGs were divided in segments, yielding a total of 77.558 samples. We trained a convolutional neural network (CNN) to classify HFpEF and control patients according to ESC criteria. An external group of 203 volunteers in a prospective heart failure screening program served as validation cohort of the CNN. Results The external validation of the CNN yielded an AUC of 0.80 (95% CI 0.74–0.86) for detection of HFpEF according to ESC criteria, with a sensitivity of 0.99 (CI 0.98–0.99) and a specificity of 0.60 (95% CI 0.56–0.64), with a positive predictive value of 0.68 (95%CI 0.64–0.72) and a negative predictive value of 0.98 (95% CI 0.95–0.99). Conclusion In this study, we report the first deep learning-enabled CNN for identifying patients with HFpEF according to ESC criteria including NT-proBNP measurements in the diagnostic algorithm among patients at risk. The suitability of the CNN was validated on an external validation cohort of patients at risk for developing heart failure, showing a convincing screening performance.

scholarly journals A Modified TNM Classification for Primary Operable Colorectal Cancer

2020 ◽  
Author(s):  
Chundong Zhang ◽  
Zubing Mei ◽  
Junpeng Pei ◽  
Masanobu Abe ◽  
Xiantao Zeng ◽  
...  
Keyword(s):  
Colorectal Cancer ◽  
Validation Cohort ◽  
Tnm Classification ◽  
Characteristic Curve ◽  
External Validation ◽  
Model Fitting ◽  
Information Criteria ◽  
Stage I ◽  
Internal Validation ◽  
Clinical Benefits

Abstract Background The American Joint Committee on Cancer (AJCC) 8th tumor/node/metastasis (TNM) classification for colorectal cancer (CRC) has limited ability to predict prognosis. Methods We included 45,379 eligible stage I-III CRC patients from the Surveillance, Epidemiology, and End Results Program. Patients were randomly assigned individually to a training (N =31,772) or an internal validation cohort (N =13,607). External validation was performed in 10,902 additional patients. Patients were divided according to T and N stage permutations. Survival analyses were conducted by a Cox proportional hazard model and Kaplan-Meier analysis, with T1N0 as the reference. Area under receiver operating characteristic curve (AUC) and Akaike information criteria (AIC) were applied for prognostic discrimination and model-fitting, respectively. Clinical benefits were further assessed by decision curve analyses. Results We created a modified TNM (mTNM) classification: stages I (T1-2N0-1a), IIA (T1N1b, T2N1b, T3N0), IIB (T1-2N2a-2b, T3N1a-1b, T4aN0), IIC (T3N2a, T4aN1a-2a, T4bN0), IIIA (T3N2b, T4bN1a), IIIB (T4aN2b, T4bN1b), and IIIC (T4bN2a-2b). In the internal validation cohort, compared to the AJCC 8th TNM classification, the mTNM classification showed superior prognostic discrimination (AUC = 0.675 vs. 0.667, respectively; two-sided P <0.001) and better model-fitting (AIC = 70,937 vs. 71,238, respectively). Similar findings were obtained in the external validation cohort. Decision curve analyses revealed that the mTNM had superior net benefits over the AJCC 8th TNM classification in the internal and external validation cohorts. Conclusions The mTNM classification provides better prognostic discrimination than AJCC 8th TNM classification, with good applicability in various populations and settings, to help better stratify stage I-III CRC patients into prognostic groups.

scholarly journals Real-time utilisation of administrative data in the ED to identify older patients at risk: development and validation of the Dynamic Silver Code

BMJ Open ◽  
2019 ◽  
Vol 9 (12) ◽  
pp. e033374 ◽  
Author(s):  
Daniela Balzi ◽  
Giulia Carreras ◽  
Francesco Tonarelli ◽  
Luca Degli Esposti ◽  
Paola Michelozzi ◽  
...  
Keyword(s):  
At Risk ◽  
Cohort Study ◽  
Real Time ◽  
Administrative Data ◽  
Older Patients ◽  
Clinical Decision Making ◽  
External Validation ◽  
Clinical Decision ◽  
Patients At Risk ◽  
Ed Visits

ObjectiveIdentification of older patients at risk, among those accessing the emergency department (ED), may support clinical decision-making. To this purpose, we developed and validated the Dynamic Silver Code (DSC), a score based on real-time linkage of administrative data.Design and settingThe ‘Silver Code National Project (SCNP)’, a non-concurrent cohort study, was used for retrospective development and internal validation of the DSC. External validation was obtained in the ‘Anziani in DEA (AIDEA)’ concurrent cohort study, where the DSC was generated by the software routinely used in the ED.ParticipantsThe SCNP contained 281 321 records of 180 079 residents aged 75+ years from Tuscany and Lazio, Italy, admitted via the ED to Internal Medicine or Geriatrics units. The AIDEA study enrolled 4425 subjects aged 75+ years (5217 records) accessing two EDs in the area of Florence, Italy.InterventionsNone.Outcome measuresPrimary outcome: 1-year mortality. Secondary outcomes: 7 and 30-day mortality and 1-year recurrent ED visits.ResultsAdvancing age, male gender, previous hospital admission, discharge diagnosis, time from discharge and polypharmacy predicted 1-year mortality and contributed to the DSC in the development subsample of the SCNP cohort. Based on score quartiles, participants were classified into low, medium, high and very high-risk classes. In the SCNP validation sample, mortality increased progressively from 144 to 367 per 1000 person-years, across DSC classes, with HR (95% CI) of 1.92 (1.85 to 1.99), 2.71 (2.61 to 2.81) and 5.40 (5.21 to 5.59) in class II, III and IV, respectively versus class I (p<0.001). Findings were similar in AIDEA, where the DSC predicted also recurrent ED visits in 1 year. In both databases, the DSC predicted 7 and 30-day mortality.ConclusionsThe DSC, based on administrative data available in real time, predicts prognosis of older patients and might improve their management in the ED.

scholarly journals Development of an Automated, Real Time Surveillance Tool for Predicting Readmissions at a Community Hospital

2013 ◽  
Vol 04 (02) ◽  
pp. 153-169 ◽  
Author(s):  
R. Gildersleeve ◽  
P. Cooper
Keyword(s):  
Real Time ◽  
Community Hospital ◽  
Validation Cohort ◽  
Data Extraction ◽  
Characteristic Curve ◽  
Hospital Setting ◽  
Data Repository ◽  
Derivation Cohort ◽  
Effective Interventions ◽  
Patients At Risk

SummaryBackground: The Centers for Medicare and Medicaid Services’ Readmissions Reduction Program adjusts payments to hospitals based on 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia. This holds hospitals accountable for a complex phenomenon about which there is little evidence regarding effective interventions. Further study may benefit from a method for efficiently and inexpensively identifying patients at risk of readmission. Several models have been developed to assess this risk, many of which may not translate to a U.S. community hospital setting.Objective: To develop a real-time, automated tool to stratify risk of 30-day readmission at a semi-rural community hospital.Methods: A derivation cohort was created by extracting demographic and clinical variables from the data repository for adult discharges from calendar year 2010. Multivariate logistic regression identified variables that were significantly associated with 30-day hospital readmission. Those variables were incorporated into a formula to produce a Risk of Readmission Score (RRS). A validation cohort from 2011 assessed the predictive value of the RRS. A SQL stored procedure was created to calculate the RRS for any patient and publish its value, along with an estimate of readmission risk and other factors, to a secure intranet site.Results: Eleven variables were significantly associated with readmission in the multivariate analysis of each cohort. The RRS had an area under the receiver operating characteristic curve (c-statistic) of 0.74 (95% CI 0.73-0.75) in the derivation cohort and 0.70 (95% CI 0.69-0.71) in the validation cohort.Conclusion: Clinical and administrative data available in a typical community hospital database can be used to create a validated, predictive scoring system that automatically assigns a probability of 30-day readmission to hospitalized patients. This does not require manual data extraction or manipulation and uses commonly available systems. Additional study is needed to refine and confirm the findings.Citation: Gildersleeve R, Cooper P. Development of an automated, real time surveillance tool for predicting readmissions at a community hospital. Appl Clin Inf 2013; 4: 153–169http://dx.doi.org/10.4338/ACI-2012-12-RA-0058

Sign in / Sign up

Export Citation Format

Share Document