Predicting poor outcome in patients with suspected COVID-19 presenting to the Emergency Department (COVERED) – Development, internal and external validation of a prediction model

2021 ◽  
Vol 20 (1) ◽  
pp. 4-14
Author(s):  
K. Azijli ◽  
◽  
A.W.E. Lieveld ◽  
S.F.B. van der Horst ◽  
N. de Graaf ◽  
...  

Background: A recent systematic review recommends against the use of any of the current COVID-19 prediction models in clinical practice. To enable clinicians to appropriately profile and treat suspected COVID-19 patients at the emergency department (ED), externally validated models that predict poor outcome are desperately needed. Objective: Our aims were to identify predictors of poor outcome, defined as mortality or ICU admission within 30 days, in patients presenting to the ED with a clinical suspicion of COVID-19, and to develop and externally validate a prediction model for poor outcome. Methods: In this prospective, multi-centre study, we enrolled suspected COVID-19 patients presenting at the EDs of two hospitals in the Netherlands. We used backward logistic regression to develop a prediction model. We used the area under the curve (AUC), Brier score and pseudo-R2 to assess model performance. The model was externally validated in an Italian cohort. Results: We included 1193 patients between March 12 and May 27 2020, of whom 196 (16.4%) had a poor outcome. We identified 10 predictors of poor outcome: current malignancy (OR 2.774; 95%CI 1.682-4.576), systolic blood pressure (OR 0.981; 95%CI 0.964-0.998), heart rate (OR 1.001; 95%CI 0.97-1.028), respiratory rate (OR 1.078; 95%CI 1.046-1.111), oxygen saturation (OR 0.899; 95%CI 0.850-0.952), body temperature (OR 0.505; 95%CI 0.359-0.710), serum urea (OR 1.404; 95%CI 1.198-1.645), C-reactive protein (OR 1.013; 95%CI 1.001-1.024), lactate dehydrogenase (OR 1.007; 95%CI 1.002-1.013) and SARS-CoV-2 PCR result (OR 2.456; 95%CI 1.526-3.953). The AUC was 0.86 (95%CI 0.83-0.89), with a Brier score of 0.32 and, and R2 of 0.41. The AUC in the external validation in 500 patients was 0.70 (95%CI 0.65-0.75). Conclusion: The COVERED risk score showed excellent discriminatory ability, also in external validation. It may aid clinical decision making, and improve triage at the ED in health care environments with high patient throughputs.

2017 ◽  
Vol 38 (8) ◽  
pp. 897-905 ◽  
Author(s):  
Yvette H. van Beurden ◽  
Marjolein P. M. Hensgens ◽  
Olaf M. Dekkers ◽  
Saskia Le Cessie ◽  
Chris J. J. Mulder ◽  
...  

OBJECTIVEEstimating the risk of a complicated course of Clostridium difficile infection (CDI) might help doctors guide treatment. We aimed to validate 3 published prediction models: Hensgens (2014), Na (2015), and Welfare (2011).METHODSThe validation cohort comprised 148 patients diagnosed with CDI between May 2013 and March 2014. During this period, 70 endemic cases of CDI occurred as well as 78 cases of CDI related to an outbreak of C. difficile ribotype 027. Model calibration and discrimination were assessed for the 3 prediction rules.RESULTSA complicated course (ie, death, colectomy, or ICU admission due to CDI) was observed in 31 patients (21%), and 23 patients (16%) died within 30 days of CDI diagnosis. The performance of all 3 prediction models was poor when applied to the total validation cohort with an estimated area under the curve (AUC) of 0.68 for the Hensgens model, 0.54 for the Na model, and 0.61 for the Welfare model. For those patients diagnosed with CDI due to non-outbreak strains, the prediction model developed by Hensgens performed the best, with an AUC of 0.78.CONCLUSIONAll 3 prediction models performed poorly when using our total cohort, which included CDI cases from an outbreak as well as endemic cases. The prediction model of Hensgens performed relatively well for patients diagnosed with CDI due to non-outbreak strains, and this model may be useful in endemic settings.Infect Control Hosp Epidemiol 2017;38:897–905


2021 ◽  
Author(s):  
Jill Hardin ◽  
Jenna M. Reps

Abstract Background: The goal of our study is to provide guidance for deciding which length of lookback to implement when engineering features to use when developing predictive models using observational healthcare data. Using a longer lookback for feature engineering gives more insight about patients but increases the issue of left-censoring. Methods: We used five US observational databases to develop patient-level prediction models. A target cohort of subjects with hypertensive drug exposures and outcome cohorts of subjects with acute (stroke and gastrointestinal bleeding) and chronic outcomes (diabetes and chronic kidney disease) were developed. Candidate predictors that exist on or prior to the target index date were derived within the following lookback periods: 14, 30, 90, 180, 365, 730, and all days prior to index were evaluated. We predicted the risk of outcomes occurring 1 day until 365 days after index. Ten lasso logistic models for each lookback period were generated to create a distribution of area under the curve (AUC) metrics to evaluate the discriminative performance of the models. Impact on external validation performance was investigated across five databases. Results: Our results show that a shorter lookback time for the acute outcomes, stroke and gastrointestinal bleeding results in equivalent performance as using longer lookback times. A lookback time of at least 365 days for the chronic outcomes, diabetes and renal impairment, results in equivalent performance as using lookback times greater than 365 days.Conclusions: Our results suggest the optimal model performance and choice of length of lookback is dependent on the outcome type (acute or chronic). Our study illustrates that use of at least 365 days results in equivalent performance as using longer lookback periods. Researchers should evaluate lookback in the context of the prediction question to determine optimal model performance.


2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S181-S181
Author(s):  
Marlene Rosen ◽  
Linda Betz ◽  
Alessandro Bertolino ◽  
Stefan Borgwardt ◽  
Paolo Brambilla ◽  
...  

Abstract Background A multitude of clinical models to predict transition to psychosis in individuals at clinical high risk (CHR) have been proposed. However, only limited efforts have been made to systematically compare these models and to validate their performance in independent samples. Therefore, in this study we identified psychosis risk models based on information readily obtainable in general clinical settings, such as clinical and neuropsychological data, and compared their performance in the PRONIA study (Personalised Prognostic Tools for Early Psychosis Management, www.pronia.eu) as an independent sample. Methods Of the 278 CHR participants in the PRONIA sample, 150 had available data until month 18 and were included in the validation of eleven psychosis prediction models identified through systematic literature search. Discrimination performance was assessed with the area under the receiver operating characteristic curve (AUC), and compared to the performance of the prognosis of clinical raters. Psychosocial functioning was explored as an alternative outcome. Results Discrimination performance varied considerably across models (AUC ranging from 0.42 to 0.79). High model performance was associated with the inclusion of neurocognitive variables as predictors. Low model performance was associated with predictors based on dichotomized variables. Clinical raters performed comparable to the best data-driven models (AUC = 0.75). Combining raters’ prognosis and model-based predictions improved discrimination performance (AUC = 0.84), particularly for less experienced raters. One of the tested models predicted transition to psychosis and psychosocial outcomes comparably well. Discussion The present external validation study highlights the benefit of enriching clinical information with neuropsychological data in predicting transition to psychosis satisfactorily and with good generalizability across samples. Integration of data-driven risk models and clinical expertise may improve clinical decision-making in CHR for psychosis, particularly for less experienced raters. This external validation study provides an important step toward early intervention and the personalized treatment of psychotic disorders.


Cancers ◽  
2021 ◽  
Vol 13 (11) ◽  
pp. 2808
Author(s):  
Tzong-Yun Tsai ◽  
Jeng-Fu You ◽  
Yu-Jen Hsu ◽  
Jing-Rong Jhuang ◽  
Yih-Jong Chern ◽  
...  

(1) Background: The aim of this study was to develop a prediction model for assessing individual mPC risk in patients with pT4 colon cancer. Methods: A total of 2003 patients with pT4 colon cancer undergoing R0 resection were categorized into the training or testing set. Based on the training set, 2044 Cox prediction models were developed. Next, models with the maximal C-index and minimal prediction error were selected. The final model was then validated based on the testing set using a time-dependent area under the curve and Brier score, and a scoring system was developed. Patients were stratified into the high- or low-risk group by their risk score, with the cut-off points determined by a classification and regression tree (CART). (2) Results: The five candidate predictors were tumor location, preoperative carcinoembryonic antigen value, histologic type, T stage and nodal stage. Based on the CART, patients were categorized into the low-risk or high-risk groups. The model has high predictive accuracy (prediction error ≤5%) and good discrimination ability (area under the curve >0.7). (3) Conclusions: The prediction model quantifies individual risk and is feasible for selecting patients with pT4 colon cancer who are at high risk of developing mPC.


2021 ◽  
pp. 036354652199382
Author(s):  
Mario Hevesi ◽  
Devin P. Leland ◽  
Philip J. Rosinsky ◽  
Ajay C. Lall ◽  
Benjamin G. Domb ◽  
...  

Background: Hip arthroscopy is rapidly advancing and increasingly commonly performed. The most common surgery after arthroscopy is total hip arthroplasty (THA), which unfortunately occurs within 2 years of arthroscopy in up to 10% of patients. Predictive models for conversion to THA, such as that proposed by Redmond et al, have potentially substantial value in perioperative counseling and decreasing early arthroscopy failures; however, these models need to be externally validated to demonstrate broad applicability. Purpose: To utilize an independent, prospectively collected database to externally validate a previously published risk calculator by determining its accuracy in predicting conversion of hip arthroscopy to THA at a minimum 2-year follow-up. Study Design: Cohort study (diagnosis); Level of evidence, 1. Methods: Hip arthroscopies performed at a single center between November 2015 and March 2017 were reviewed. Patients were assessed pre- and intraoperatively for components of the THA risk score studied—namely, age, modified Harris Hip Score, lateral center-edge angle, revision procedure, femoral version, and femoral and acetabular Outerbridge scores—and followed for a minimum of 2 years. Conversion to THA was determined along with the risk score’s receiver operating characteristic (ROC) curve and Brier score calibration characteristics. Results: A total of 187 patients (43 men, 144 women, mean age, 36.0 ± 12.4 years) underwent hip arthroscopy and were followed for a mean of 2.9 ± 0.85 years (range, 2.0-5.5 years), with 13 patients (7%) converting to THA at a mean of 1.6 ± 0.9 years. Patients who converted to THA had a mean predicted arthroplasty risk of 22.6% ± 12.0%, compared with patients who remained arthroplasty-free with a predicted risk of 4.6% ± 5.3% ( P < .01). The Brier score for the calculator was 0.04 ( P = .53), which was not statistically different from ideal calibration, and the calculator demonstrated a satisfactory area under the curve of 0.894 ( P < .001). Conclusion: This external validation study supported our hypothesis in that the THA risk score described by Redmond et al was found to accurately predict which patients undergoing hip arthroscopy were at risk for converting to subsequent arthroplasty, with satisfactory discriminatory, ROC curve, and Brier score calibration characteristics. These findings are important in that they provide surgeons with validated tools to identify the patients at greatest risk for failure after hip arthroscopy and assist in perioperative counseling and decision making.


BMJ Open ◽  
2021 ◽  
Vol 11 (2) ◽  
pp. e044500
Author(s):  
Yauhen Statsenko ◽  
Fatmah Al Zahmi ◽  
Tetiana Habuza ◽  
Klaus Neidl-Van Gorkom ◽  
Nazar Zaki

BackgroundDespite the necessity, there is no reliable biomarker to predict disease severity and prognosis of patients with COVID-19. The currently published prediction models are not fully applicable to clinical use.ObjectivesTo identify predictive biomarkers of COVID-19 severity and to justify their threshold values for the stratification of the risk of deterioration that would require transferring to the intensive care unit (ICU).MethodsThe study cohort (560 subjects) included all consecutive patients admitted to Dubai Mediclinic Parkview Hospital from February to May 2020 with COVID-19 confirmed by the PCR. The challenge of finding the cut-off thresholds was the unbalanced dataset (eg, the disproportion in the number of 72 patients admitted to ICU vs 488 non-severe cases). Therefore, we customised supervised machine learning (ML) algorithm in terms of threshold value used to predict worsening.ResultsWith the default thresholds returned by the ML estimator, the performance of the models was low. It was improved by setting the cut-off level to the 25th percentile for lymphocyte count and the 75th percentile for other features. The study justified the following threshold values of the laboratory tests done on admission: lymphocyte count <2.59×109/L, and the upper levels for total bilirubin 11.9 μmol/L, alanine aminotransferase 43 U/L, aspartate aminotransferase 32 U/L, D-dimer 0.7 mg/L, activated partial thromboplastin time (aPTT) 39.9 s, creatine kinase 247 U/L, C reactive protein (CRP) 14.3 mg/L, lactate dehydrogenase 246 U/L, troponin 0.037 ng/mL, ferritin 498 ng/mL and fibrinogen 446 mg/dL.ConclusionThe performance of the neural network trained with top valuable tests (aPTT, CRP and fibrinogen) is admissible (area under the curve (AUC) 0.86; 95% CI 0.486 to 0.884; p<0.001) and comparable with the model trained with all the tests (AUC 0.90; 95% CI 0.812 to 0.902; p<0.001). Free online tool at https://med-predict.com illustrates the study results.


2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Espen Jimenez-Solem ◽  
Tonny S. Petersen ◽  
Casper Hansen ◽  
Christian Hansen ◽  
Christina Lioma ◽  
...  

AbstractPatients with severe COVID-19 have overwhelmed healthcare systems worldwide. We hypothesized that machine learning (ML) models could be used to predict risks at different stages of management and thereby provide insights into drivers and prognostic markers of disease progression and death. From a cohort of approx. 2.6 million citizens in Denmark, SARS-CoV-2 PCR tests were performed on subjects suspected for COVID-19 disease; 3944 cases had at least one positive test and were subjected to further analysis. SARS-CoV-2 positive cases from the United Kingdom Biobank was used for external validation. The ML models predicted the risk of death (Receiver Operation Characteristics—Area Under the Curve, ROC-AUC) of 0.906 at diagnosis, 0.818, at hospital admission and 0.721 at Intensive Care Unit (ICU) admission. Similar metrics were achieved for predicted risks of hospital and ICU admission and use of mechanical ventilation. Common risk factors, included age, body mass index and hypertension, although the top risk features shifted towards markers of shock and organ dysfunction in ICU patients. The external validation indicated fair predictive performance for mortality prediction, but suboptimal performance for predicting ICU admission. ML may be used to identify drivers of progression to more severe disease and for prognostication patients in patients with COVID-19. We provide access to an online risk calculator based on these findings.


2021 ◽  
Author(s):  
Patricia Pauline M. Remalante-Rayco ◽  
Evelyn Osio-Salido

Objective. To assess the performance of prognostic models in predicting mortality or clinical deterioration among patients with COVID-19, both hospitalized and non-hospitalized Methods. We conducted a systematic review of the literature until March 8, 2021. We included models for the prediction of mortality or clinical deterioration in COVID-19 with external validation. We used the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and the GRADEpro Guideline Development Tool (GDT) to assess the evidence obtained. Results. We reviewed 33 cohort studies. Two studies had a low risk of bias, four unclear risks, and 27 with a high risk of bias due to participant selection and analysis. For the outcome of mortality, the QCOVID model had excellent prediction with high certainty of evidence but was specific for use in England. The COVID Outcome Prediction in the Emergency Department (COPE) model, the 4C Mortality Score, the Age, BUN, number of comorbidities, CRP, SpO2/FiO2 ratio, platelet count, heart rate (ABC2-SPH) risk score, the Confusion Urea Respiration Blood Pressure (CURB-65) severity score, the Rapid Emergency Medicine Score (REMS), and the Risk Stratification in the Emergency Department in Acutely Ill Older Patients (RISE UP) score had fair to good prediction of death among inpatients, while the quick Sepsis-related Organ Failure Assessment (qSOFA) score had poor to fair prediction. The certainty of evidence for these models was very low to low. For the outcome of clinical deterioration, the 4C Deterioration Score had fair prediction, the National Early Warning Score 2 (NEWS2) score poor to good, and the Modified Early Warning Score (MEWS) had poor prediction. The certainty of evidence for these three models was also very low to low. None of these models had been validated in the Philippine setting. Conclusion. The QCOVID, COPE, ABC2-SPH, 4C, CURB-65, REMS, RISE-UP models for prediction of mortality and the 4C Deterioration and NEWS2 models for prediction of clinical deterioration are potentially useful but need to be validated among patients with COVID-19 of varying severity in the Philippine setting.


Author(s):  
Donatella Poz ◽  
Danila Crobu ◽  
Elena Sukhacheva ◽  
Marco Bruno Luigi Rocchi ◽  
Maria Chiara Anelli ◽  
...  

Abstract Objectives Sepsis is a time-dependent and life-threating condition. Despite several biomarkers are available, none of them is completely reliable for the diagnosis. This study aimed to evaluate the diagnostic utility of monocyte distribution width (MDW) to early detect sepsis in adult patients admitted in the Emergency Department (ED) with a five part differential analysis as part of the standard clinical practice. Methods A prospective cohort study was conducted on 985 patients aged from 18 to 96 and included in the study between November 2019 and December 2019. Enrolled subjects were classified into four groups based on sepsis-2 diagnostic criteria: control, Systemic Inflammatory Response Syndrome (SIRS), infection and sepsis. The hematology analyzer DxH 900 (Beckman Coulter Inc.) provides the new reportable parameter MDW, included in the leukocyte 5 part differential analysis, cleared by Food and Drug administration (FDA) and European Community In-Vitro-Diagnostic Medical Device (CE IVD) marked as early sepsis indicator (ESId). Results MDW was able to differentiate the sepsis group from all other groups with Area Under the Curve (AUC) of 0.849, sensitivity of 87.3% and specificity of 71.7% at cut-off of 20.1. MDW in combination with white blood cell (WBC) improves the performance for sepsis detection with a sensitivity increased up to 96.8% when at least one of the two biomarkers are abnormal, and a specificity increased up to 94.6% when both biomarkers are abnormal. Conclusions MDW can predict sepsis increasing the clinical value of Leukocyte 5 Part Differential analysis and supporting the clinical decision making in sepsis management at the admission to the ED.


2021 ◽  
Vol 11 ◽  
Author(s):  
Jin-feng Pan ◽  
Rui Su ◽  
Jian-zhou Cao ◽  
Zhen-ya Zhao ◽  
Da-wei Ren ◽  
...  

PurposeThe purpose of this study is to explore the value of combining bpMRI and clinical indicators in the diagnosis of clinically significant prostate cancer (csPCa), and developing a prediction model and Nomogram to guide clinical decision-making.MethodsWe retrospectively analyzed 530 patients who underwent prostate biopsy due to elevated serum prostate specific antigen (PSA) levels and/or suspicious digital rectal examination (DRE). Enrolled patients were randomly assigned to the training group (n = 371, 70%) and validation group (n = 159, 30%). All patients underwent prostate bpMRI examination, and T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI) sequences were collected before biopsy and were scored, which were respectively named T2WI score and DWI score according to Prostate Imaging Reporting and Data System version 2 (PI-RADS v.2) scoring protocol, and then PI-RADS scoring was performed. We defined a new bpMRI-based parameter named Total score (Total score = T2WI score + DWI score). PI-RADS score and Total score were separately included in the multivariate analysis of the training group to determine independent predictors for csPCa and establish prediction models. Then, prediction models and clinical indicators were compared by analyzing the area under the curve (AUC) and decision curves. A Nomogram for predicting csPCa was established using data from the training group.ResultsIn the training group, 160 (43.1%) patients had prostate cancer (PCa), including 128 (34.5%) with csPCa. Multivariate regression analysis showed that the PI-RADS score, Total score, f/tPSA, and PSA density (PSAD) were independent predictors of csPCa. The prediction model that was defined by Total score, f/tPSA, and PSAD had the highest discriminatory power of csPCa (AUC = 0.931), and the diagnostic sensitivity and specificity were 85.1% and 87.5%, respectively. Decision curve analysis (DCA) showed that the prediction model achieved an optimal overall net benefit in both the training group and the validation group. In addition, the Nomogram predicted csPCa revealed good estimation when compared with clinical indicators.ConclusionThe prediction model and Nomogram based on bpMRI and clinical indicators exhibit a satisfactory predictive value and improved risk stratification for csPCa, which could be used for clinical biopsy decision-making.


Sign in / Sign up

Export Citation Format

Share Document