External Validation of Three Prediction Tools for Patients at Risk of a Complicated Course of Clostridium difficile Infection: Disappointing in an Outbreak Setting

2017 ◽  
Vol 38 (8) ◽  
pp. 897-905 ◽  
Author(s):  
Yvette H. van Beurden ◽  
Marjolein P. M. Hensgens ◽  
Olaf M. Dekkers ◽  
Saskia Le Cessie ◽  
Chris J. J. Mulder ◽  
...  

OBJECTIVEEstimating the risk of a complicated course of Clostridium difficile infection (CDI) might help doctors guide treatment. We aimed to validate 3 published prediction models: Hensgens (2014), Na (2015), and Welfare (2011).METHODSThe validation cohort comprised 148 patients diagnosed with CDI between May 2013 and March 2014. During this period, 70 endemic cases of CDI occurred as well as 78 cases of CDI related to an outbreak of C. difficile ribotype 027. Model calibration and discrimination were assessed for the 3 prediction rules.RESULTSA complicated course (ie, death, colectomy, or ICU admission due to CDI) was observed in 31 patients (21%), and 23 patients (16%) died within 30 days of CDI diagnosis. The performance of all 3 prediction models was poor when applied to the total validation cohort with an estimated area under the curve (AUC) of 0.68 for the Hensgens model, 0.54 for the Na model, and 0.61 for the Welfare model. For those patients diagnosed with CDI due to non-outbreak strains, the prediction model developed by Hensgens performed the best, with an AUC of 0.78.CONCLUSIONAll 3 prediction models performed poorly when using our total cohort, which included CDI cases from an outbreak as well as endemic cases. The prediction model of Hensgens performed relatively well for patients diagnosed with CDI due to non-outbreak strains, and this model may be useful in endemic settings.Infect Control Hosp Epidemiol 2017;38:897–905

2021 ◽  
Vol 20 (1) ◽  
pp. 4-14
Author(s):  
K. Azijli ◽  
◽  
A.W.E. Lieveld ◽  
S.F.B. van der Horst ◽  
N. de Graaf ◽  
...  

Background: A recent systematic review recommends against the use of any of the current COVID-19 prediction models in clinical practice. To enable clinicians to appropriately profile and treat suspected COVID-19 patients at the emergency department (ED), externally validated models that predict poor outcome are desperately needed. Objective: Our aims were to identify predictors of poor outcome, defined as mortality or ICU admission within 30 days, in patients presenting to the ED with a clinical suspicion of COVID-19, and to develop and externally validate a prediction model for poor outcome. Methods: In this prospective, multi-centre study, we enrolled suspected COVID-19 patients presenting at the EDs of two hospitals in the Netherlands. We used backward logistic regression to develop a prediction model. We used the area under the curve (AUC), Brier score and pseudo-R2 to assess model performance. The model was externally validated in an Italian cohort. Results: We included 1193 patients between March 12 and May 27 2020, of whom 196 (16.4%) had a poor outcome. We identified 10 predictors of poor outcome: current malignancy (OR 2.774; 95%CI 1.682-4.576), systolic blood pressure (OR 0.981; 95%CI 0.964-0.998), heart rate (OR 1.001; 95%CI 0.97-1.028), respiratory rate (OR 1.078; 95%CI 1.046-1.111), oxygen saturation (OR 0.899; 95%CI 0.850-0.952), body temperature (OR 0.505; 95%CI 0.359-0.710), serum urea (OR 1.404; 95%CI 1.198-1.645), C-reactive protein (OR 1.013; 95%CI 1.001-1.024), lactate dehydrogenase (OR 1.007; 95%CI 1.002-1.013) and SARS-CoV-2 PCR result (OR 2.456; 95%CI 1.526-3.953). The AUC was 0.86 (95%CI 0.83-0.89), with a Brier score of 0.32 and, and R2 of 0.41. The AUC in the external validation in 500 patients was 0.70 (95%CI 0.65-0.75). Conclusion: The COVERED risk score showed excellent discriminatory ability, also in external validation. It may aid clinical decision making, and improve triage at the ED in health care environments with high patient throughputs.


2017 ◽  
Vol 4 (suppl_1) ◽  
pp. S402-S402 ◽  
Author(s):  
Catherine Beauregard-Paultre ◽  
Claire Nour Abou Chakra ◽  
Allison Mcgeer ◽  
Annie-Claude Labbé ◽  
Andrew E Simor ◽  
...  

Abstract Background Clostridium difficile infection (CDI) is the most common cause of nosocomial diarrhea. About one in 5 patients with CDI (median 18%) develop a complication (cCDI), including mortality. Many predictive scores have been published to identify patients at risk of cCDI but none is currently recommended for clinical use and few were validated. We conducted an external validation study of predictive tools for cCDI. Methods Predictive tools were identified through a systematic review. We included those reporting at least an internal validation process. We performed the external validation on a multicenter prospective cohort of 1380 Canadian adults with confirmed CDI. Most cases were elderly (median age 71), had a healthcare facility-associated CDI (90%), and cCDI occurred in 8%. NAP1 strain was found in 52%. The performance of each scoring system was analyzed using individual outcomes. Modifications in predictors were made to match available data in the validation cohort. Results We assessed 3 predictive scores and one predictive model. The performance (95% CI) of higher thresholds are shown in the Table. All scores had a low sensitivity and PPV, and moderate specificity and NPV. The model of Shivashankar 2013 (age, WBC> 15, narcotic use, antacids use, creatinine ratio > 1.5) predicted 25% probability of cCDI. All showed similar AUC (0.63–0.67). Conclusion The predictive tools included in our study showed moderate performance in a validation cohort with a low rate of cCDI and high proportion of NAP1 strains. Further research is needed to develop an accurate predictive tool to guide clinicians in the management of CDI. Disclosures J. Powis, Merck: Grant Investigator, Research grant; GSK: Grant Investigator, Research grant; Roche: Grant Investigator, Research grant; Synthetic Biologicals: Investigator, Research grant


2021 ◽  
Author(s):  
Arjun Chandna ◽  
Raman Mahajan ◽  
Priyanka Gautam ◽  
Lazaro Mwandigha ◽  
Karthik Gunasekaran ◽  
...  

ABSTRACTBackgroundIn locations where few people have received COVID-19 vaccines, health systems remain vulnerable to surges in SARS-CoV-2 infections. Tools to identify patients suitable for community-based management are urgently needed.MethodsWe prospectively recruited adults presenting to two hospitals in India with moderate symptoms of laboratory-confirmed COVID-19 in order to develop and validate a clinical prediction model to rule-out progression to supplemental oxygen requirement. The primary outcome was defined as any of the following: SpO2 < 94%; respiratory rate > 30 bpm; SpO2/FiO2 < 400; or death. We specified a priori that each model would contain three clinical parameters (age, sex and SpO2) and one of seven shortlisted biochemical biomarkers measurable using near-patient tests (CRP, D-dimer, IL-6, NLR, PCT, sTREM-1 or suPAR), to ensure the models would be suitable for resource-limited settings. We evaluated discrimination, calibration and clinical utility of the models in a temporal external validation cohort.Findings426 participants were recruited, of whom 89 (21·0%) met the primary outcome. 257 participants comprised the development cohort and 166 comprised the validation cohort. The three models containing NLR, suPAR or IL-6 demonstrated promising discrimination (c-statistics: 0·72 to 0·74) and calibration (calibration slopes: 1·01 to 1·05) in the validation cohort, and provided greater utility than a model containing the clinical parameters alone.InterpretationWe present three clinical prediction models that could help clinicians identify patients with moderate COVID-19 suitable for community-based management. The models are readily implementable and of particular relevance for locations with limited resources.FundingMédecins Sans Frontières, India.RESEARCH IN CONTEXTEvidence before this studyA living systematic review by Wynants et al. identified 137 COVID-19 prediction models, 47 of which were derived to predict whether patients with COVID-19 will have an adverse outcome. Most lacked external validation, relied on retrospective data, did not focus on patients with moderate disease, were at high risk of bias, and were not practical for use in resource-limited settings. To identify promising biochemical biomarkers which may have been evaluated independently of a prediction model and therefore not captured by this review, we searched PubMed on 1 June 2020 using synonyms of “SARS-CoV-2” AND [“biomarker” OR “prognosis”]. We identified 1,214 studies evaluating biochemical biomarkers of potential value in the prognostication of COVID-19 illness. In consultation with FIND (Geneva, Switzerland) we shortlisted seven candidates for evaluation in this study, all of which are measurable using near-patient tests which are either currently available or in late-stage development.Added value of this studyWe followed the TRIPOD guidelines to develop and validate three promising clinical prediction models to help clinicians identify which patients presenting with moderate COVID-19 can be safely managed in the community. Each model contains three easily ascertained clinical parameters (age, sex, and SpO2) and one biochemical biomarker (NLR, suPAR or IL-6), and would be practical for implementation in high-patient-throughput low resource settings. The models showed promising discrimination and calibration in the validation cohort. The inclusion of a biomarker test improved prognostication compared to a model containing the clinical parameters alone, and extended the range of contexts in which such a tool might provide utility to include situations when bed pressures are less critical, for example at earlier points in a COVID-19 surge.Implications of all the available evidencePrognostic models should be developed for clearly-defined clinical use-cases. We report the development and temporal validation of three clinical prediction models to rule-out progression to supplemental oxygen requirement amongst patients presenting with moderate COVID-19. The models are readily implementable and should prove useful in triage and resource allocation. We provide our full models to enable independent validation.


2021 ◽  
Vol 80 (Suppl 1) ◽  
pp. 1004.1-1004
Author(s):  
D. Xu ◽  
R. Mu

Background:Scleroderma renal crisis (SRC) is a life-threatening syndrome. The early identification of patients at risk is essential for timely treatment to improve the outcome[1].Objectives:We aimed to provide a personalized tool to predict risk of SRC in systemic sclerosis (SSc).Methods:We tried to set up a SRC prediction model based on the PKUPH-SSc cohort of 302 SSc patients. The least absolute shrinkage and selection operator (Lasso) regression was used to optimize disease features. Multivariable logistic regression analysis was applied to build a SRC prediction model incorporating the features of SSc selected in the Lasso regression. Then, a multi-predictor nomogram combining clinical characteristics was constructed and evaluated by discrimination and calibration.Results:A multi-predictor nomogram for evaluating the risk of SRC was successfully developed. In the nomogram, four easily available predictors were contained including disease duration <2 years, cardiac involvement, anemia and corticosteroid >15mg/d exposure. The nomogram displayed good discrimination with an area under the curve (AUC) of 0.843 (95% CI: 0.797-0.882) and good calibration.Conclusion:The multi-predictor nomogram for SRC could be reliably and conveniently used to predict the individual risk of SRC in SSc patients, and be a step towards more personalized medicine.References:[1]Woodworth TG, Suliman YA, Li W, Furst DE, Clements P (2016) Scleroderma renal crisis and renal involvement in systemic sclerosis. Nat Rev Nephrol 12 (11):678-91.Disclosure of Interests:None declared


2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Espen Jimenez-Solem ◽  
Tonny S. Petersen ◽  
Casper Hansen ◽  
Christian Hansen ◽  
Christina Lioma ◽  
...  

AbstractPatients with severe COVID-19 have overwhelmed healthcare systems worldwide. We hypothesized that machine learning (ML) models could be used to predict risks at different stages of management and thereby provide insights into drivers and prognostic markers of disease progression and death. From a cohort of approx. 2.6 million citizens in Denmark, SARS-CoV-2 PCR tests were performed on subjects suspected for COVID-19 disease; 3944 cases had at least one positive test and were subjected to further analysis. SARS-CoV-2 positive cases from the United Kingdom Biobank was used for external validation. The ML models predicted the risk of death (Receiver Operation Characteristics—Area Under the Curve, ROC-AUC) of 0.906 at diagnosis, 0.818, at hospital admission and 0.721 at Intensive Care Unit (ICU) admission. Similar metrics were achieved for predicted risks of hospital and ICU admission and use of mechanical ventilation. Common risk factors, included age, body mass index and hypertension, although the top risk features shifted towards markers of shock and organ dysfunction in ICU patients. The external validation indicated fair predictive performance for mortality prediction, but suboptimal performance for predicting ICU admission. ML may be used to identify drivers of progression to more severe disease and for prognostication patients in patients with COVID-19. We provide access to an online risk calculator based on these findings.


2021 ◽  
Vol 156 (A4) ◽  
Author(s):  
N Hifi ◽  
N Barltrop

This paper applies a newly developed methodology to calibrate the corrosion model within a structural reliability analysis. The methodology combines data from experience (measurements and expert judgment) and prediction models to adjust the structural reliability models. Two corrosion models published in the literature have been used to demonstrate the technique used for the model calibration. One model is used as a prediction for a future degradation and a second one to represent the inspection recorded data. The results of the calibration process are presented and discussed.


Circulation ◽  
2019 ◽  
Vol 140 (Suppl_2) ◽  
Author(s):  
Marinos Kosmopoulos ◽  
Jason A Bartos ◽  
Demetris Yannopoulos

Introduction: Veno-Arterial Extracorporeal Membrane Oxygenation (VA ECMO) has emerged as a prominent tool for management of patients with Inability to Wean Off Cardiopulmonary Bypass (IWOCB), extracorporeal cardiopulmonary resuscitation (eCPR) or refractory cardiogenic shock (RCS). The high mortality that is still associated with these diseases urges for the development of reliable prediction models for mortality after cannulation. Survival After VA ECMO (SAVE) Score consists one of the most widely used prediction tools and the only model with external validation. However, its predictive value is still under debate. Hypothesis: Whether VA ECMO indication affects the predictive value of SAVE Score. Methods: 317 patients treated with VA ECMO in a quaternary center (n= 52 for IWOCB, n=179 for eCPR and n=86 for RCS) were retrospectively assessed for differences in SAVE Score and their primary outcomes. The Receiver Operating Characteristic (ROC) curve for SAVE Score and mortality was calculated separately for each VA ECMO indication. Results: The three groups had significant differences in SAVE Score (p<0.01) without significant differences in mortality (p=0.176). ROC Curve calculation indicated significant differences in predictive value of SAVE Score for survival among its different indications. (Area Under the Curve= 81.69% for IWOCB, 53.79% for eCPR and 69.46% for RCS). Conclusion: VA ECMO indication markedly affects the predictive value of SAVE Score. Prediction of primary outcome in IWOCB patients was reliable. On the contrary, routine application for survival estimation in eCPR patients is not supported from our results.


2021 ◽  
Vol 36 (Supplement_1) ◽  
Author(s):  
A Youssef

Abstract Study question Which models that predict pregnancy outcome in couples with unexplained RPL exist and what is the performance of the most used model? Summary answer We identified seven prediction models; none followed the recommended prediction model development steps. Moreover, the most used model showed poor predictive performance. What is known already RPL remains unexplained in 50–75% of couples For these couples, there is no effective treatment option and clinical management rests on supportive care. Essential part of supportive care consists of counselling on the prognosis of subsequent pregnancies. Indeed, multiple prediction models exist, however the quality and validity of these models varies. In addition, the prediction model developed by Brigham et al is the most widely used model, but has never been externally validated. Study design, size, duration We performed a systematic review to identify prediction models for pregnancy outcome after unexplained RPL. In addition we performed an external validation of the Brigham model in a retrospective cohort, consisting of 668 couples with unexplained RPL that visited our RPL clinic between 2004 and 2019. Participants/materials, setting, methods A systematic search was performed in December 2020 in Pubmed, Embase, Web of Science and Cochrane library to identify relevant studies. Eligible studies were selected and assessed according to the TRIPOD) guidelines, covering topics on model performance and validation statement. The performance of predicting live birth in the Brigham model was evaluated through calibration and discrimination, in which the observed pregnancy rates were compared to the predicted pregnancy rates. Main results and the role of chance Seven models were compared and assessed according to the TRIPOD statement. This resulted in two studies of low, three of moderate and two of above average reporting quality. These studies did not follow the recommended steps for model development and did not calculate a sample size. Furthermore, the predictive performance of neither of these models was internally- or externally validated. We performed an external validation of Brigham model. Calibration showed overestimation of the model and too extreme predictions, with a negative calibration intercept of –0.52 (CI 95% –0.68 – –0.36), with a calibration slope of 0.39 (CI 95% 0.07 – 0.71). The discriminative ability of the model was very low with a concordance statistic of 0.55 (CI 95% 0.50 – 0.59). Limitations, reasons for caution None of the studies are specifically named prediction models, therefore models may have been missed in the selection process. The external validation cohort used a retrospective design, in which only the first pregnancy after intake was registered. Follow-up time was not limited, which is important in counselling unexplained RPL couples. Wider implications of the findings: Currently, there are no suitable models that predict on pregnancy outcome after RPL. Moreover, we are in need of a model with several variables such that prognosis is individualized, and factors from both the female as the male to enable a couple specific prognosis. Trial registration number Not applicable


Gut ◽  
2020 ◽  
pp. gutjnl-2019-319926 ◽  
Author(s):  
Waku Hatta ◽  
Yosuke Tsuji ◽  
Toshiyuki Yoshio ◽  
Naomi Kakushima ◽  
Shu Hoteya ◽  
...  

ObjectiveBleeding after endoscopic submucosal dissection (ESD) for early gastric cancer (EGC) is a frequent adverse event after ESD. We aimed to develop and externally validate a clinically useful prediction model (BEST-J score: Bleeding after ESD Trend from Japan) for bleeding after ESD for EGC.DesignThis retrospective study enrolled patients who underwent ESD for EGC. Patients in the derivation cohort (n=8291) were recruited from 25 institutions, and patients in the external validation cohort (n=2029) were recruited from eight institutions in other areas. In the derivation cohort, weighted points were assigned to predictors of bleeding determined in the multivariate logistic regression analysis and a prediction model was established. External validation of the model was conducted to analyse discrimination and calibration.ResultsA prediction model comprised 10 variables (warfarin, direct oral anticoagulant, chronic kidney disease with haemodialysis, P2Y12 receptor antagonist, aspirin, cilostazol, tumour size >30 mm, lower-third in tumour location, presence of multiple tumours and interruption of each kind of antithrombotic agents). The rates of bleeding after ESD at low-risk (0 to 1 points), intermediate-risk (2 points), high-risk (3 to 4 points) and very high-risk (≥5 points) were 2.8%, 6.1%, 11.4% and 29.7%, respectively. In the external validation cohort, the model showed moderately good discrimination, with a c-statistic of 0.70 (95% CI, 0.64 to 0.76), and good calibration (calibration-in-the-large, 0.05; calibration slope, 1.01).ConclusionsIn this nationwide multicentre study, we derived and externally validated a prediction model for bleeding after ESD. This model may be a good clinical decision-making support tool for ESD in patients with EGC.


2019 ◽  
Vol 8 (1) ◽  
pp. 37 ◽  
Author(s):  
Byuk Ko ◽  
Youn-Jung Kim ◽  
Dae Jung ◽  
Chang Sohn ◽  
Dong Seo ◽  
...  

Risk assessment for upper gastrointestinal bleeding (UGIB) is important; however, current scoring systems are insufficient. We aimed to develop and validate a prediction model for rapidly determining the occurrence of hypotension in non-variceal UGIB patients with normotension (systolic blood pressure ≥90 mmHg) at emergency department presentation. In this prospective observational cohort study, consecutive non-variceal UGIB patients between January 2012 and April 2017 were enrolled. We developed and validated a new prediction model through logistic regression, with the occurrence of hypotension <24 h as the primary outcome. Among 3363 UGIB patients, 1439 non-variceal UGIB patients were included. The risk factors for the occurrence of hypotension were lactate level, blood in nasogastric tube, and systolic blood pressure. The area under the curve (AUC) of the new scoring model (LBS—Lactate, Blood in nasogastric tube, Systolic blood pressure) in the development cohort was 0.74, higher than the value of 0.64 of the Glasgow–Blatchford score for predicting the occurrence of hypotension. The AUC of the LBS score in the validation cohort was 0.83. An LBS score of ≤2 had a negative predictive value of 99.5% and an LBS score of ≥7 had a specificity of 97.5% in the validation cohort. The new LBS score stratifies normotensive patients with non-variceal UGIB at risk for developing hypotension.


Sign in / Sign up

Export Citation Format

Share Document