scholarly journals Development, implementation, and prospective validation of a model to predict 60-day end-of-life in hospitalized adults upon admission at three sites

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Vincent J. Major ◽  
Yindalon Aphinyanaphongs
Keyword(s):  
Positive Predictive Value ◽  
Receiver Operating Characteristic ◽  
Predictive Value ◽  
Operating Characteristic ◽  
Medical Center ◽  
Academic Medical Center ◽  
Model Performance ◽  
Risk Of Death ◽  
Temporal Validation ◽  
Receiver Operating

Abstract Background Automated systems that use machine learning to estimate a patient’s risk of death are being developed to influence care. There remains sparse transparent reporting of model generalizability in different subpopulations especially for implemented systems. Methods A prognostic study included adult admissions at a multi-site, academic medical center between 2015 and 2017. A predictive model for all-cause mortality (including initiation of hospice care) within 60 days of admission was developed. Model generalizability is assessed in temporal validation in the context of potential demographic bias. A subsequent prospective cohort study was conducted at the same sites between October 2018 and June 2019. Model performance during prospective validation was quantified with areas under the receiver operating characteristic and precision recall curves stratified by site. Prospective results include timeliness, positive predictive value, and the number of actionable predictions. Results Three years of development data included 128,941 inpatient admissions (94,733 unique patients) across sites where patients are mostly white (61%) and female (60%) and 4.2% led to death within 60 days. A random forest model incorporating 9614 predictors produced areas under the receiver operating characteristic and precision recall curves of 87.2 (95% CI, 86.1–88.2) and 28.0 (95% CI, 25.0–31.0) in temporal validation. Performance marginally diverges within sites as the patient mix shifts from development to validation (patients of one site increases from 10 to 38%). Applied prospectively for nine months, 41,728 predictions were generated in real-time (median [IQR], 1.3 [0.9, 32] minutes). An operating criterion of 75% positive predictive value identified 104 predictions at very high risk (0.25%) where 65% (50 from 77 well-timed predictions) led to death within 60 days. Conclusion Temporal validation demonstrates good model discrimination for 60-day mortality. Slight performance variations are observed across demographic subpopulations. The model was implemented prospectively and successfully produced meaningful estimates of risk within minutes of admission.

scholarly journals Age-Adjusted Endothelial Activation and Stress Index for Coronavirus Disease 2019 at Admission Is a Reliable Predictor for 28-Day Mortality in Hospitalized Patients With Coronavirus Disease 2019

2021 ◽  
Vol 8 ◽  
Author(s):  
Felipe Pérez-García ◽  
Rebeca Bailén ◽  
Juan Torres-Macho ◽  
Amanda Fernández-Rodríguez ◽  
Maria Ángeles Jiménez-Sousa ◽  
...  
Keyword(s):  
Positive Predictive Value ◽  
Receiver Operating Characteristic ◽  
Negative Predictive Value ◽  
Predictive Value ◽  
Operating Characteristic ◽  
Predictive Performance ◽  
Endothelial Activation ◽  
Stress Index ◽  
Reliable Predictor ◽  
Receiver Operating

Background: Endothelial Activation and Stress Index (EASIX) predict death in patients undergoing allogeneic hematopoietic stem cell transplantation who develop endothelial complications. Because coronavirus disease 2019 (COVID-19) patients also have coagulopathy and endotheliitis, we aimed to assess whether EASIX predicts death within 28 days in hospitalized COVID-19 patients.Methods: We performed a retrospective study on COVID-19 patients from two different cohorts [derivation (n = 1,200 patients) and validation (n = 1,830 patients)]. The endpoint was death within 28 days. The main factors were EASIX [(lactate dehydrogenase * creatinine)/thrombocytes] and aEASIX-COVID (EASIX * age), which were log2-transformed for analysis.Results: Log2-EASIX and log2-aEASIX-COVID were independently associated with an increased risk of death in both cohorts (p < 0.001). Log2-aEASIX-COVID showed a good predictive performance for 28-day mortality both in the derivation cohort (area under the receiver-operating characteristic = 0.827) and in the validation cohort (area under the receiver-operating characteristic = 0.820), with better predictive performance than log2-EASIX (p < 0.001). For log2 aEASIX-COVID, patients with low/moderate risk (<6) had a 28-day mortality probability of 5.3% [95% confidence interval (95% CI) = 4–6.5%], high (6–7) of 17.2% (95% CI = 14.7–19.6%), and very high (>7) of 47.6% (95% CI = 44.2–50.9%). The cutoff of log2 aEASIX-COVID = 6 showed a positive predictive value of 31.7% and negative predictive value of 94.7%, and log2 aEASIX-COVID = 7 showed a positive predictive value of 47.6% and negative predictive value of 89.8%.Conclusion: Both EASIX and aEASIX-COVID were associated with death within 28 days in hospitalized COVID-19 patients. However, aEASIX-COVID had significantly better predictive performance than EASIX, particularly for discarding death. Thus, aEASIX-COVID could be a reliable predictor of death that could help to manage COVID-19 patients.

scholarly journals 215 Assessment of the Diagnostic Sensitivity and Specificity of Pain Biomarkers in Cattle Using Receiver Operating Characteristic Curves

2020 ◽  
Vol 98 (Supplement_3) ◽  
pp. 7-8
Author(s):  
Miriam S Martin ◽  
Michael Kleinhenz ◽  
Karen Schwartzkopf-Genswein ◽  
Johann Coetzee
Keyword(s):  
Diagnostic Accuracy ◽  
Receiver Operating Characteristic ◽  
Sensitivity And Specificity ◽  
Predictive Value ◽  
Roc Analysis ◽  
Operating Characteristic ◽  
Diagnostic Sensitivity ◽  
Diagnostic Sensitivity And Specificity ◽  
Good Diagnostic Accuracy ◽  
Receiver Operating

Abstract Biomarkers are commonly used to assess pain and analgesic drug efficacy in livestock. However, the diagnostic sensitivity and specificity of these biomarkers for different pain conditions over time have not been described. Receiver operating characteristic (ROC) curves are graphical plots that illustrate the diagnostic ability of a test as its discrimination threshold is varied. The objective of this analysis was to use area under the curve (AUC) values derived from ROC analysis to assess the predictive value of pain biomarkers at specific timepoints. The biomarkers included in the analysis were blood cortisol, salivary cortisol, hair cortisol, infrared thermography (IRT), mechanical nociceptive threshold (MNT), substance P, and outcomes from a pressure/force measurement system and visual analog scale. A total sample size of 7,992 biomarker outcomes were collected from 6 pain studies involving pain associated with castration, dehorning, lameness, and surgery were included in the analysis. Each study consisted of three treatments; pain, no pain, and analgesia. All statistics were performed using statistical software (JMP Pro 14.0, SAS Institute, Inc., Cary, NC). Results comparing analgesia verses pain yielded good diagnostic accuracy (AUC > 0.7; 95% CI: 0.40 to 0.99) for blood cortisol (timepoints 1.5, 2, and 6 hours); IRT (timepoints 6, 8, 12, and 72 hours); and MNT (timepoints 6, 25, and 49 hours). These results indicate that ROC analysis can be a useful indicator of the predictive value of pain biomarkers and certain timepoints seem to yield good diagnostic accuracy while many do not.

scholarly journals Predictive value of a novel lncRNA LINC02518 in evaluating the prognosis of patients with hepatocellular carcinoma

2021 ◽  
Author(s):  
Wei Cui ◽  
Jingzhi Huang ◽  
Ruiqi Wang ◽  
Yu Wang ◽  
Xiaoming Chen ◽  
...  
Keyword(s):  
Hepatocellular Carcinoma ◽  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Predictive Value ◽  
Long Noncoding Rna ◽  
Noncoding Rna ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Operating Characteristic Curve ◽  
Receiver Operating

Aim: The potential of long noncoding RNA in hepatocellular carcinoma (HCC) has led to promising insights into therapeutic intervention. The clinical significance of LINC02518 in HCC is unclear. This study aimed to evaluate the predictive value of a novel long noncoding RNA, LINC02518, for the prognosis of patients with HCC. Methods: Between December 2005 and November 2011, 125 and 75 HCC patients in the training and validation groups, respectively, who underwent liver surgery were included in our study. The LINC02518 expression of HCC and corresponding nontumor liver tissues was detected using microarray and reverse transcription quantitative polymerase chain reaction (RT-qPCR). These HCC patients were assigned into high and low LINC02518 expression groups based on the threshold of the receiver operating characteristic curve. Kaplan-Meier analysis was performed to determine the prognosis of HCC patients. Results: LINC02518 expression was upregulated in paired tumor samples compared with corresponding nontumor samples in the two groups. The area under the receiver operating characteristic curve for the levels of LINC02518 in the diagnosis of HCC was 0.66, 95% CI: 0.59–0.73. HCC patients with high LINC02518 expression had significantly worse tumor recurrence-free, metastasis-free, disease-free and overall survival than those with low LINC02518 expression. Conclusion: LINC02518 is negatively correlated with the prognosis of HCC and provides a promising strategy for the treatment and prognosis of HCC.

scholarly journals 362. A Modified Early Warning Score Predicts Decompensation in COVID-19 Patients

2020 ◽  
Vol 7 (Supplement_1) ◽  
pp. S251-S251
Author(s):  
Joanna S Cavalier ◽  
Benjamin Goldstein ◽  
Cara L O’Brien ◽  
Armando Bedoya
Keyword(s):  
Positive Predictive Value ◽  
Early Warning ◽  
Predictive Value ◽  
Risk Model ◽  
Medical Center ◽  
Academic Medical Center ◽  
Significant Proportion ◽  
Case Series ◽  
Early Warning Score ◽  
Severe Illness

Abstract Background The novel coronavirus disease (COVID-19) results in severe illness in a significant proportion of patients, necessitating a way to discern which patients will become critically ill and which will not. In one large case series, 5.0% of patients required an intensive care unit (ICU) and 1.4% died. Several models have been developed to assess decompensating patients. However, research examining their applicability to COVID-19 patients is limited. An accurate predictive model for patients at risk of decompensation is critical for health systems to optimally triage emergencies, care for patients, and allocate resources. Methods An early warning score (EWS) algorithm created within a large academic medical center, with methodology previously described, was applied to COVID-19 patients admitted to this institution. 122 COVID-19 patients were included. A decompensation event was defined as inpatient mortality or an unanticipated transfer to an ICU from an intermediate medical ward. The EWS was calculated at 12-hour and 24-hour intervals. Results Of 122 patients admitted with COVID-19, 28 had a decompensation event, yielding an event rate of 23.0%. 8 patients died, 13 transferred to the ICU, and 6 both transferred to the ICU and died. Decompensation within 12 and 24 hours were predicted with areas under the curve (AUC) of 0.850 and 0.817, respectively. Using a three-tiered risk model, use of the customized EWS score for patients identified as high risk of decompensation had a positive predictive value of 44.4% and 11.1% and specificity of 99.3% and 99.6% and 12- and 24-hour intervals. Amongst medium-risk patients, the score had a specificity of 85.0% and 85.4%, respectively. Conclusion This EWS allows for prediction of decompensation, defined as transfer to an ICU or death, in COVID-19 patients with excellent specificity and a high positive predictive value. Clinically, implementation of this score can help to identify patients before they decompensate in order to triage at time of presentation and allocate step-down beds, ICU beds, and treatments such as remdesivir. Disclosures All Authors: No reported disclosures

scholarly journals Usefulness of the 5MPB-Toledo model to predict bacteremia in patients with community-acquired pneumonia in the Emergency Department

2021 ◽  
Author(s):  
Agustín Julián-Jiménez ◽  
◽  
Juan González del Castillo ◽  
Eric Jorge García-Lamberechts ◽  
Rafael Rubio Díaz ◽  
...  
Keyword(s):  
Positive Predictive Value ◽  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Predictive Value ◽  
Emergency Departments ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Community Acquired Pneumonia ◽  
Prognostic Performance ◽  
Operating Characteristic Curve

Objective. To analyse a new risk score to predict bacteremia in the patients with Community-acquired Pneumonia (CAP) in the emergency departments. Patients and methods. Prospective and multicenter observational cohort study of the blood cultures ordered in 74 Spanish emergency departments for patients with CAP seen from November 1, 2019, to March 31, 2020. The predictive ability of the model was analyzed with the area under the Receiver Operating Characteristic curve (AUC-ROC). The prognostic performance for true bacteremia was calculated with the chosen cut-off for getting the sensitivity, specificity, positive predictive value and negative predictive value. Results. A total of 1,020 blood samples wered cultured. True cases of bacteremia were confirmed in 162 (15.9%). The remaining 858 cultures (84.1%) wered negative. And, 59 (5.8%) were judged to be contaminated. The model´s area under the receiver operating characteristic curve was 0.915 (95% CI, 0.898-0.933). The prognostic performance with a model´s cut-off value of ≥ 5 points achieved 97.5% (95% CI, 95.1-99.9) sensitivity, 73.2% (95% CI, 70.2-76.2) specificity, 40.9% (95% CI, 36.4-45.1) positive predictive value and 99.4% (95% CI, 99.1-99.8) negative predictive value. Conclusion. The 5MPB-Toledo score is useful for predicting bacteremia in the patients with CAP seen in the emergency departments.

scholarly journals “Emergency Room Evaluation and Recommendations” (ER2) Tool for the Screening of Older Emergency Department Visitors With Major Neurocognitive Disorders: Results From the ER2 Database

2022 ◽  
Vol 12 ◽  
Author(s):  
Olivier Beauchet ◽  
Liam A. Cooper-Brown ◽  
Joshua Lubov ◽  
Gilles Allali ◽  
Marc Afilalo ◽  
...  
Keyword(s):  
Emergency Department ◽  
High Risk ◽  
Emergency Room ◽  
Receiver Operating Characteristic ◽  
Likelihood Ratio ◽  
Predictive Value ◽  
Operating Characteristic ◽  
Neurocognitive Disorders ◽  
Risk Level ◽  
Receiver Operating

Purpose: The Emergency Room Evaluation and Recommendation (ER2) is an application in the electronic medical file of patients visiting the Emergency Department (ED) of the Jewish General Hospital (JGH; Montreal, Quebec, Canada). It screens for older ED visitors at high risk of undesirable events. The aim of this study is to examine the performance criteria (i.e., sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], positive likelihood ratio [LR+], negative likelihood ratio [LR-] and area under the receiver operating characteristic curve [AUROC]) of the ER2 high-risk level and its “temporal disorientation” item alone to screen for major neurocognitive disorders in older ED visitors at the JGH.Methods: Based on a cross-sectional design, 999 older adults (age 84.9 ± 5.6, 65.1% female) visiting the ED of the JGH were selected from the ER2 database. ER2 was completed upon the patients' arrival at the ED. The outcomes were ER2's high-risk level, the answer to ER2's temporal disorientation item (present vs. absent), and the diagnosis of major neurocognitive disorders (yes vs. no) which was confirmed when it was present in a letter or other files signed by a physician.Results: The sensitivities of both ER2's high-risk level and temporal disorientation item were high (≥0.91). Specificity, the PPV, LR+, and AROC were higher for the temporal disorientation item compared to ER2's high-risk level, whereas a highest sensitivity, LR-, and NPV were obtained with the ER2 high-risk level. Both area under the receiver operating characteristic curves were high (0.71 for ER2's high-risk level and 0.82 for ER2 temporal disorientation item). The odds ratios (OR) of ER2's high-risk level and of temporal disorientation item for the diagnosis of major neurocognitive disorders were positive and significant with all OR above 18, the highest OR being reported for the temporal disorientation item in the unadjusted model [OR = 26.4 with 95% confidence interval (CI) = 17.7–39.3].Conclusion: Our results suggest that ER2 and especially its temporal disorientation item may be used to screen for major neurocognitive disorders in older ED users.

scholarly journals Clinical Significance of Serum HE4, CA125, CA724, and CA19-9 in Patients With Endometrial Cancer

2016 ◽  
Vol 16 (4) ◽  
pp. 435-439 ◽  
Author(s):  
Jing Bian ◽  
Xiaoxu Sun ◽  
Bo Li ◽  
Liang Ming
Keyword(s):  
Endometrial Cancer ◽  
Receiver Operating Characteristic ◽  
Predictive Value ◽  
Operating Characteristic ◽  
Area Under The Curve ◽  
International Federation ◽  
Characteristic Analysis ◽  
Predictive Values ◽  
Gynecology And Obstetrics ◽  
Receiver Operating

Purpose: Serum markers with increased sensitivity and specificity for endometrial cancer are required. To date, no good marker has met this standard. The aims of our study were to evaluate the utility of tumor markers HE4, CA125, CA724, and CA19-9 as potential markers in patients diagnosed with endometrial cancer. Methods: Blood samples from 105 patients with endometrial cancer and 87 healthy women were analyzed by Roche electrochemiluminescent immunoassay, and serum values were measured for the following biomarkers: HE4, CA125, CA724, and CA19-9. Results: Serum HE4, CA125, CA724, and CA19-9 concentrations were significantly higher in patients with endometrial cancer, compared with controls ( P < .001). In the receiver operating characteristic analysis, the area under the curve value for combination of HE4, CA125, CA724, and CA19-9 was 82.1% (95% confidence interval: 75.3%-86.2%), the maximum area of the test groups. For all stages of patients with endometrial cancer, HE4 had higher sensitivity (58%), positive predictive value (60%), and negative predictive value (67%) than any other single tumor marker, and in the combination of HE4, CA125, CA724, and CA19-9, the sensitivity and positive predictive values reached 59.1% and 88%, respectively. Meanwhile, the receiver operating characteristic area under the curve of the combination of the 4 markers was significantly increased than any other group, either in stage I or in stage II to IV cases. HE4 and CA125 both correlate with advanced age; in addition, HE4 was related to pathology subtypes and positive adnexal involvement, CA125 was related to International Federation of Gynecology and Obstetrics stage, CA19-9 was related to International Federation of Gynecology and Obstetrics stage, and CA724 was correlated with positive lymph node. Conclusion: Combination of HE4, CA125, CA724, and CA19-9 has the highest value in diagnosing endometrial cancer, and they can be a useful tissue immune marker for patients with endometrial cancer.

Evaluation of intraventricular hemorrhage assessment methods for predicting outcome following intracerebral hemorrhage

2012 ◽  
Vol 116 (1) ◽  
pp. 185-192 ◽  
Author(s):  
Brian Y. Hwang ◽  
Samuel S. Bruce ◽  
Geoffrey Appelboom ◽  
Matthew A. Piazza ◽  
Amanda M. Carpenter ◽  
...  
Keyword(s):  
Intracerebral Hemorrhage ◽  
Receiver Operating Characteristic ◽  
Intraventricular Hemorrhage ◽  
Operating Characteristic ◽  
Medical Center ◽  
External Ventricular Drain ◽  
Characteristic Analysis ◽  
Poor Outcome ◽  
The Impact ◽  
Receiver Operating

Object Intraventricular hemorrhage (IVH) associated with intracerebral hemorrhage (ICH) is an independent predictor of poor outcome. Clinical methods for evaluating IVH, however, are not well established. This study sought to determine the best IVH grading scale by evaluating the predictive accuracies of IVH, Graeb, and LeRoux scores in an independent cohort of ICH patients with IVH. Subacute IVH dynamics as well as the impact of external ventricular drain (EVD) placement on IVH and outcome were also investigated. Methods A consecutive cohort of 142 primary ICH patients with IVH was admitted to Columbia University Medical Center between February 2009 and February 2011. Baseline demographics, clinical presentation, and hospital course were prospectively recorded. Admission CT scans performed within 24 hours of onset were reviewed for ICH location, hematoma volume, and presence of IVH. Intraventricular hemorrhage was categorized according to IVH, Graeb, and LeRoux scores. For each patient, the last scan performed within 6 days of ictus was similarly evaluated. Outcomes at discharge were assessed using the modified Rankin Scale (mRS). Receiver operating characteristic analysis was used to determine the predictive accuracies of the grading scales for poor outcome (mRS score ≥ 3). Results Seventy-three primary ICH patients (51%) had IVH. Median admission IVH, Graeb, and LeRoux scores were 13, 6, and 8, respectively. Median IVH, Graeb and LeRoux scores decreased to 9 (p = 0.005), 4 (p = 0.002), and 4 (p = 0.003), respectively, within 6 days of ictus. Poor outcome was noted in 55 patients (75%). Areas under the receiver operating characteristic curve were similar among the IVH, Graeb, and LeRoux scores (0.745, 0.743, and 0.744, respectively) and within 6 days postictus (0.765, 0.722, 0.723, respectively). Moreover, the IVH, Graeb, and LeRoux scores had similar maximum Youden Indices both at admission (0.515 vs 0.477 vs 0.440, respectively) and within 6 days postictus (0.515 vs 0.339 vs 0.365, respectively). Patients who received EVDs had higher mean IVH volumes (23 ± 26 ml vs 9 ± 11 ml, p = 0.003) and increased incidence of Glasgow Coma Scale scores < 8 (67% vs 38%, p = 0.015) and hydrocephalus (82% vs 50%, p = 0.004) at admission but had similar outcome as those who did not receive an EVD. Conclusions The IVH, Graeb, and LeRoux scores predict outcome well with similarly good accuracy in ICH patients with IVH when assessed at admission and within 6 days after hemorrhage. Therefore, any of one of the scores would be equally useful for assessing IVH severity and risk-stratifying ICH patients with regard to outcome. These results suggest that EVD placement may be beneficial for patients with severe IVH, who have particularly poor prognosis at admission, but a randomized clinical trial is needed to conclusively demonstrate its therapeutic value.

scholarly journals Predictive Value of 5 Early Warning Scores for Critical COVID-19 Patients

2020 ◽  
pp. 1-8
Author(s):  
Hai Hu ◽  
Ni Yao ◽  
Yanru Qiu
Keyword(s):  
Receiver Operating Characteristic Curve ◽  
Receiver Operating Characteristic ◽  
Early Warning ◽  
Predictive Value ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Early Warning Score ◽  
Early Warning Scores ◽  
Operating Characteristic Curve ◽  
Receiver Operating

ABSTRACT Objectives: A simple evaluation tool for patients with novel coronavirus disease 2019 (COVID-19) could assist the physicians to triage COVID-19 patients effectively and rapidly. This study aimed to evaluate the predictive value of 5 early warning scores based on the admission data of critical COVID-19 patients. Methods: Overall, medical records of 319 COVID-19 patients were included in the study. Demographic and clinical characteristics on admission were used for calculating the Standardized Early Warning Score (SEWS), National Early Warning Score (NEWS), National Early Warning Score2 (NEWS2), Hamilton Early Warning Score (HEWS), and Modified Early Warning Score (MEWS). Data on the outcomes (survival or death) were collected for each case and extracted for overall and subgroup analysis. Receiver operating characteristic curve analyses were performed. Results: The area under the receiver operating characteristic curve for the SEWS, NEWS, NEWS2, HEWS, and MEWS in predicting mortality were 0.841 (95% CI: 0.765-0.916), 0.809 (95% CI: 0.727-0.891), 0.809 (95% CI: 0.727-0.891), 0.821 (95% CI: 0.748-0.895), and 0.670 (95% CI: 0.573-0.767), respectively. Conclusions: SEWS, NEWS, NEWS2, and HEWS demonstrated moderate discriminatory power and, therefore, offer potential utility as prognostic tools for screening severely ill COVID-19 patients. However, MEWS is not a good prognostic predictor for COVID-19.

Sign in / Sign up

Export Citation Format

Share Document