External validation of the 4C mortality score among COVID-19 patients admitted to hospital in Ontario, Canada: a retrospective study

AbstractRisk prediction scores are important tools to support clinical decision-making for patients with coronavirus disease (COVID-19). The objective of this paper was to validate the 4C mortality score, originally developed in the United Kingdom, for a Canadian population, and to examine its performance over time. We conducted an external validation study within a registry of COVID-19 positive hospital admissions in the Kitchener-Waterloo and Hamilton regions of southern Ontario between March 4, 2020 and June 13, 2021. We examined the validity of the 4C score to prognosticate in-hospital mortality using the area under the receiver operating characteristic curve (AUC) with 95% confidence intervals calculated via bootstrapping. The study included 959 individuals, of whom 224 (23.4%) died in-hospital. Median age was 72 years and 524 individuals (55%) were male. The AUC of the 4C score was 0.77, 95% confidence interval 0.79–0.87. Overall mortality rates across the pre-defined risk groups were 0% (Low), 8.0% (Intermediate), 27.2% (High), and 54.2% (Very High). Wave 1, 2 and 3 values of the AUC were 0.81 (0.76, 0.86), 0.74 (0.69, 0.80), and 0.76 (0.69, 0.83) respectively. The 4C score is a valid tool to prognosticate mortality from COVID-19 in Canadian hospitals and can be used to prioritize care and resources for patients at greatest risk of death.

Download Full-text

External Validation of the 4C Mortality Score among COVID-19 Patients Visiting the Emergency Department or admitted to Hospital in Ontario, Canada

10.21203/rs.3.rs-268410/v1 ◽

2021 ◽

Author(s):

Aaron Jones ◽

Tyler Pitre ◽

Mats Junek ◽

Jessica Kapralik ◽

Rina Patel ◽

...

Keyword(s):

Emergency Department ◽

Clinical Decision Making ◽

Hospital Admissions ◽

Characteristic Curve ◽

External Validation ◽

Risk Groups ◽

Clinical Decision ◽

Emergency Department Visits ◽

The United Kingdom ◽

Canadian Population

Abstract Objectives: Risk prediction scores are important tools to support clinical decision-making for patients with coronavirus disease (COVID-19). The objective of this paper was to validate the 4C mortality score, originally developed in the United Kingdom, for a Canadian population. Methods: We conducted an external validation study within a registry of COVID-19 positive emergency department visits and hospital admissions in the Kitchener-Waterloo and Hamilton regions of southern Ontario between March 4 and January 9, 2020. We examined the validity of the 4C score to prognosticate in-hospital mortality using the area under the receiver operating characteristic curve (AUC) with 95% confidence intervals calculated via bootstrapping. Results: The study included 560 individuals, of whom 115 (20.5%) died in-hospital. Median age was 69 years and 281 individuals (51%) were male. The AUC of the 4C score was 0.83, 95% confidence interval 0.79-0.87. Mortality rates across the pre-defined risk groups were 0% (Low), 3.2% (Intermediate), 25.9% (High), and 59.5% (Very High). The AUC was 0.80 (0.76-0.85) among hospital inpatients. Interpretation: The 4C score is a valid tool to prognosticate mortality from COVID-19 in Canadian emergency departments and hospitals.

Download Full-text

Developing and validating COVID-19 adverse outcome risk prediction models from a bi-national European cohort of 5594 patients

Scientific Reports ◽

10.1038/s41598-021-81844-x ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Espen Jimenez-Solem ◽

Tonny S. Petersen ◽

Casper Hansen ◽

Christian Hansen ◽

Christina Lioma ◽

...

Keyword(s):

Prediction Models ◽

Severe Disease ◽

External Validation ◽

Area Under The Curve ◽

Predictive Performance ◽

Mortality Prediction ◽

Icu Admission ◽

Risk Of Death ◽

Online Risk ◽

The United Kingdom

AbstractPatients with severe COVID-19 have overwhelmed healthcare systems worldwide. We hypothesized that machine learning (ML) models could be used to predict risks at different stages of management and thereby provide insights into drivers and prognostic markers of disease progression and death. From a cohort of approx. 2.6 million citizens in Denmark, SARS-CoV-2 PCR tests were performed on subjects suspected for COVID-19 disease; 3944 cases had at least one positive test and were subjected to further analysis. SARS-CoV-2 positive cases from the United Kingdom Biobank was used for external validation. The ML models predicted the risk of death (Receiver Operation Characteristics—Area Under the Curve, ROC-AUC) of 0.906 at diagnosis, 0.818, at hospital admission and 0.721 at Intensive Care Unit (ICU) admission. Similar metrics were achieved for predicted risks of hospital and ICU admission and use of mechanical ventilation. Common risk factors, included age, body mass index and hypertension, although the top risk features shifted towards markers of shock and organ dysfunction in ICU patients. The external validation indicated fair predictive performance for mortality prediction, but suboptimal performance for predicting ICU admission. ML may be used to identify drivers of progression to more severe disease and for prognostication patients in patients with COVID-19. We provide access to an online risk calculator based on these findings.

Download Full-text

Early Identification of Patients with Acute Gastrointestinal Bleeding using Electronic Health Record Phenotyping

10.1101/2020.07.06.20136374 ◽

2020 ◽

Author(s):

Dennis Shung ◽

Cynthia Tsay ◽

Loren Laine ◽

Prem Thomas ◽

Caitlin Partridge ◽

...

Keyword(s):

Risk Stratification ◽

Gastrointestinal Bleeding ◽

Real Time ◽

Decision Rule ◽

Language Processing ◽

Clinical Decision Making ◽

External Validation ◽

Clinical Decision ◽

Internal Validation ◽

Electronic Health

Background and AimGuidelines recommend risk stratification scores in patients presenting with gastrointestinal bleeding (GIB), but such scores are uncommonly employed in practice. Automation and deployment of risk stratification scores in real time within electronic health records (EHRs) would overcome a major impediment. This requires an automated mechanism to accurately identify (“phenotype”) patients with GIB at the time of presentation. The goal is to identify patients with acute GIB by developing and evaluating EHR-based phenotyping algorithms for emergency department (ED) patients.MethodsWe specified criteria using structured data elements to create rules for identifying patients, and also developed a natural-language-processing (NLP)-based algorithm for automated phenotyping of patients, tested them with tenfold cross-validation (n=7144) and external validation (n=2988), and compared them with the standard method for encoding patient conditions in the EHR, Systematized Nomenclature of Medicine (SNOMED). The gold standard for GIB diagnosis was independent dual manual review of medical records. The primary outcome was positive predictive value (PPV).ResultsA decision rule using GIB-specific terms from ED triage and from ED review-of-systems assessment performed better than SNOMED on internal validation (PPV=91% [90%-93%] vs. 74% [71%-76%], P<0.001) and external validation (PPV=85% [84%-87%] vs. 69% [67%-71%], P<0.001). The NLP algorithm (external validation PPV=80% [79-82%]) was not superior to the structured-datafields decision rule.ConclusionsAn automated decision rule employing GIB-specific triage and review-of-systems terms can be used to trigger EHR-based deployment of risk stratification models to guide clinical decision-making in real time for patients with acute GIB presenting to the ED.

Download Full-text

Validation of parsimonious prognostic models for patients infected with COVID-19

BMJ Health & Care Informatics ◽

10.1136/bmjhci-2020-100267 ◽

2021 ◽

Vol 28 (1) ◽

pp. e100267

Author(s):

Keerthi Harish ◽

Ben Zhang ◽

Peter Stella ◽

Kevin Hauck ◽

Marwa M Moussa ◽

...

Keyword(s):

Clinical Decision Making ◽

Validation Cohort ◽

External Validation ◽

Clinical Decision ◽

Prognostic Models ◽

Clinical Prediction ◽

Inclusion Criteria ◽

Local Data ◽

External Data ◽

Operator Curve

ObjectivesPredictive studies play important roles in the development of models informing care for patients with COVID-19. Our concern is that studies producing ill-performing models may lead to inappropriate clinical decision-making. Thus, our objective is to summarise and characterise performance of prognostic models for COVID-19 on external data.MethodsWe performed a validation of parsimonious prognostic models for patients with COVID-19 from a literature search for published and preprint articles. Ten models meeting inclusion criteria were either (a) externally validated with our data against the model variables and weights or (b) rebuilt using original features if no weights were provided. Nine studies had internally or externally validated models on cohorts of between 18 and 320 inpatients with COVID-19. One model used cross-validation. Our external validation cohort consisted of 4444 patients with COVID-19 hospitalised between 1 March and 27 May 2020.ResultsMost models failed validation when applied to our institution’s data. Included studies reported an average validation area under the receiver–operator curve (AUROC) of 0.828. Models applied with reported features averaged an AUROC of 0.66 when validated on our data. Models rebuilt with the same features averaged an AUROC of 0.755 when validated on our data. In both cases, models did not validate against their studies’ reported AUROC values.DiscussionPublished and preprint prognostic models for patients infected with COVID-19 performed substantially worse when applied to external data. Further inquiry is required to elucidate mechanisms underlying performance deviations.ConclusionsClinicians should employ caution when applying models for clinical prediction without careful validation on local data.

Download Full-text

Flow cytometry, cellular DNA content, and prognosis in human malignancy.

Journal of Clinical Oncology ◽

10.1200/jco.1987.5.10.1690 ◽

1987 ◽

Vol 5 (10) ◽

pp. 1690-1703 ◽

Cited By ~ 192

Author(s):

D E Merkel ◽

L G Dressler ◽

W L McGuire

Keyword(s):

Flow Cytometry ◽

Data Base ◽

Dna Content ◽

Dna Analysis ◽

Clinical Decision Making ◽

Risk Groups ◽

Clinical Decision ◽

Tumor Type ◽

Cellular Dna ◽

Bladder Carcinomas

The use of flow cytometry to analyze the cellular DNA content of human malignancies has become increasingly commonplace. The relationship between abnormalities in DNA content or proliferative characteristics and prognosis is becoming clear for a variety of malignancies in part through new techniques that permit analysis of archival material. High- and low-risk groups of patients with early breast and bladder carcinomas, non-small-cell lung cancer, and colorectal, ovarian, and cervical carcinoma can be distinguished on the basis of abnormal stemline DNA content. In several hematologic and common pediatric malignancies, the prognostic relevance of DNA content flow cytometry has been similarly established. Though the interpretation of tumor cell cycle analyses is less certain, this characteristic may also be prognostically important. However, generalizations cannot be made when applying flow cytometric DNA analysis to clinical decision making. The prognostic importance of an abnormal DNA histogram for an individual patient must be assessed on the basis of the relevant data base for that particular tumor type. The current extent of this data base for various malignancies is reviewed.

Download Full-text

Real-time utilisation of administrative data in the ED to identify older patients at risk: development and validation of the Dynamic Silver Code

BMJ Open ◽

10.1136/bmjopen-2019-033374 ◽

2019 ◽

Vol 9 (12) ◽

pp. e033374 ◽

Cited By ~ 2

Author(s):

Daniela Balzi ◽

Giulia Carreras ◽

Francesco Tonarelli ◽

Luca Degli Esposti ◽

Paola Michelozzi ◽

...

Keyword(s):

At Risk ◽

Cohort Study ◽

Real Time ◽

Administrative Data ◽

Older Patients ◽

Clinical Decision Making ◽

External Validation ◽

Clinical Decision ◽

Patients At Risk ◽

Ed Visits

ObjectiveIdentification of older patients at risk, among those accessing the emergency department (ED), may support clinical decision-making. To this purpose, we developed and validated the Dynamic Silver Code (DSC), a score based on real-time linkage of administrative data.Design and settingThe ‘Silver Code National Project (SCNP)’, a non-concurrent cohort study, was used for retrospective development and internal validation of the DSC. External validation was obtained in the ‘Anziani in DEA (AIDEA)’ concurrent cohort study, where the DSC was generated by the software routinely used in the ED.ParticipantsThe SCNP contained 281 321 records of 180 079 residents aged 75+ years from Tuscany and Lazio, Italy, admitted via the ED to Internal Medicine or Geriatrics units. The AIDEA study enrolled 4425 subjects aged 75+ years (5217 records) accessing two EDs in the area of Florence, Italy.InterventionsNone.Outcome measuresPrimary outcome: 1-year mortality. Secondary outcomes: 7 and 30-day mortality and 1-year recurrent ED visits.ResultsAdvancing age, male gender, previous hospital admission, discharge diagnosis, time from discharge and polypharmacy predicted 1-year mortality and contributed to the DSC in the development subsample of the SCNP cohort. Based on score quartiles, participants were classified into low, medium, high and very high-risk classes. In the SCNP validation sample, mortality increased progressively from 144 to 367 per 1000 person-years, across DSC classes, with HR (95% CI) of 1.92 (1.85 to 1.99), 2.71 (2.61 to 2.81) and 5.40 (5.21 to 5.59) in class II, III and IV, respectively versus class I (p<0.001). Findings were similar in AIDEA, where the DSC predicted also recurrent ED visits in 1 year. In both databases, the DSC predicted 7 and 30-day mortality.ConclusionsThe DSC, based on administrative data available in real time, predicts prognosis of older patients and might improve their management in the ED.

Download Full-text

THUR 122 Parkinson’s kinetigraph improves movement disorder service provision

Journal of Neurology Neurosurgery & Psychiatry ◽

10.1136/jnnp-2018-abn.49 ◽

2018 ◽

Vol 89 (10) ◽

pp. A13.3-A13

Author(s):

Kobylecki Christopher ◽

Partington-Smith Lucy ◽

Silverdale Monty

Keyword(s):

Movement Disorder ◽

Service Provision ◽

Clinical Decision Making ◽

Hospital Admissions ◽

Clinical Decision ◽

Objective Evaluation ◽

Advanced Therapies ◽

Subsequent Effect ◽

Outpatient Appointments ◽

Deep Brain

IntroductionObjective evaluation of symptoms of Parkinson’s disease (PD) can be challenging. There is increasing interest in technological solutions to assess, monitor and manage people with PD.ObjectiveTo evaluate the effect of the Parkinson’s Kinetigraph (PKG) on management of patients with PD in a large tertiary movement disorder service.MethodsWe retrospectively reviewed the notes of 47 patients with PD (22 female, 25 male) who underwent PKG recording over a six month period. The indications and PKG findings, and the subsequent effect on clinical decision making and service provision were recorded.ResultsManagement was significantly altered in 25 patients (53%), while in 13 patients (28%) PKG confirmed the use of advanced therapies such as deep brain stimulation. Significant effects were seen with regard to service provision. Outpatient appointments could be deferred with advice following PKG in 15 (32%), advanced therapies assessment was improved in 16 (34%), while inpatient admission was avoided in six patients (13%).ConclusionThe use of PKG has enhanced service provision in our movement disorder service. In particular, it enhances our assessment of patients considered for high-cost advanced therapies, allows more efficient use of clinic appointments, and has the potential to reduce hospital admissions.

Download Full-text

Comparação da Aplicação da EER Hema-Obs a 250 Gestações da Consulta de Hematologia-Obstetrícia do Centro Hospitalar São João, Portugal Versus EER Galit Sarig a 90 Gestações no Rambam Health Care Campus, Israel

Acta Médica Portuguesa ◽

10.20344/amp.5200 ◽

2015 ◽

Vol 28 (2) ◽

pp. 189

Author(s):

Ana Salselas ◽

Inês Pestana ◽

Francisco Bischoff ◽

Mariana Guimarães ◽

Joaquim Aguiar Andrade

Keyword(s):

High Risk ◽

Risk Stratification ◽

Pregnant Women ◽

Receiver Operating Characteristic ◽

Operating Characteristic ◽

Characteristic Curve ◽

Risk Groups ◽

Low Risk ◽

Very High ◽

Receiver Operating

Introduction: Pregnant women with thromboembolic diseases, previous thrombotic episodes or thrombophilia family history were supervised in a multidisciplinary Obstetrics/ Hematology consultation in Centro Hospitalar São João EPE, Porto, Portugal. For the evaluation and medication of these women, a risk stratification scale was used. Purposes: The aim of this study was to validate a Risk Stratification Scale and thromboprophylaxis protocol by means of comparing it with a similar scale, developed and published by Sarig. Material and Methods: We have compared: The distribution, by risk groups, obtained through the application of the two scales on pregnant women followed at Centro Hospitalar São João, Porto, Portugal, consultation; the sensibility and specificity for each one of the scales (DeLong scale, applied to Receiver Operating Characteristic) curves; the outcomes in pregnancies followed in Hospital São João, Porto, Portugal Results: According to our Hema-Obs risk stratification scale, 29% were allocated to low-risk, 47% to high-risk and 24% to very-high-risk groups. According to Galit Sarig risk stratification scale, 24% were considered low-risk, 53% moderate, 16% high-risk and 7% as very high-risk group. In our study we observed 9% of spontaneous abortions, in comparison with 18% in the Galit Sarig cohort. From the application of Receiver Operating Characteristic curve to both risk stratification scales, the results of the calculated areas were 58,8% to our Hema-Obs risk stratification scale and 38,7% to Galit Sarig risk stratification scale, with a Delong test significancie of p = 0.0006. Conclusions: We concluded that Hema-Obs risk stratification scale is an effective support for clinical monitoring of therapeutic strategies.

Download Full-text

Orale Antikoagulation bei chronischer Beinischämie

VASA ◽

10.1024/0301-1526.30.2.83 ◽

2001 ◽

Vol 30 (2) ◽

pp. 83-88

Author(s):

U. Mueller-Kolck

Keyword(s):

Oral Anticoagulation ◽

Clinical Decision Making ◽

Risk Model ◽

Anticoagulation Therapy ◽

Limb Ischemia ◽

Risk Groups ◽

Clinical Decision ◽

Arterial Disease ◽

Bleeding Complications ◽

Minor Bleeding

This review article summarizes clinical data on adjuvant long-term oral anticoagulation therapy (OAC) of peripheral arterial disease (PAD). It analyzes the underlying risk model of oral anticoagulation. Definitions of runoff patterns, of major and minor bleeding complications, of predictors of major bleedings as well as a classification of patient risk groups are described. The indication for OAC treatment of chronic limb ischemia remains still due to an individual decision. Clinical decision making is facilitated by the risk model. Improved oral anticoagulation control results in fewer bleeding complications. Studies on patient weekly self-testing and self-dosing which support this hypothesis are reviewed in the context of adjuvant OAC therapy.

Download Full-text