scholarly journals Validation of parsimonious prognostic models for patients infected with COVID-19

2021 ◽  
Vol 28 (1) ◽  
pp. e100267
Author(s):  
Keerthi Harish ◽  
Ben Zhang ◽  
Peter Stella ◽  
Kevin Hauck ◽  
Marwa M Moussa ◽  
...  

ObjectivesPredictive studies play important roles in the development of models informing care for patients with COVID-19. Our concern is that studies producing ill-performing models may lead to inappropriate clinical decision-making. Thus, our objective is to summarise and characterise performance of prognostic models for COVID-19 on external data.MethodsWe performed a validation of parsimonious prognostic models for patients with COVID-19 from a literature search for published and preprint articles. Ten models meeting inclusion criteria were either (a) externally validated with our data against the model variables and weights or (b) rebuilt using original features if no weights were provided. Nine studies had internally or externally validated models on cohorts of between 18 and 320 inpatients with COVID-19. One model used cross-validation. Our external validation cohort consisted of 4444 patients with COVID-19 hospitalised between 1 March and 27 May 2020.ResultsMost models failed validation when applied to our institution’s data. Included studies reported an average validation area under the receiver–operator curve (AUROC) of 0.828. Models applied with reported features averaged an AUROC of 0.66 when validated on our data. Models rebuilt with the same features averaged an AUROC of 0.755 when validated on our data. In both cases, models did not validate against their studies’ reported AUROC values.DiscussionPublished and preprint prognostic models for patients infected with COVID-19 performed substantially worse when applied to external data. Further inquiry is required to elucidate mechanisms underlying performance deviations.ConclusionsClinicians should employ caution when applying models for clinical prediction without careful validation on local data.

2019 ◽  
Vol 39 (11) ◽  
Author(s):  
Shaonan Fan ◽  
Ting Li ◽  
Ping Zhou ◽  
Qiliang Peng ◽  
Yaqun Zhu

Abstract Purpose: Nomogram is a widely used tool that precisely predicts individualized cancer prognoses. We aimed to develop and validate a reliable nomogram including serum tumor biomarkers to predict individual overall survival (OS) for patients with resected rectal cancer (RC) and compare the predictive value with the American Joint Committee on Cancer (AJCC) stages. Patients and methods: We analyzed 520 patients who were diagnosed with non-metastatic rectal cancer as training cohort. External validation was performed in a cohort of 11851 patients from the Surveillance, Epidemiology, and End Results (SEER) database. Independent prognostic factors were identified and integrated to build a nomogram using the Cox proportional hazard regression model. The nomogram was evaluated by Harrell’s concordance index (C-index) and calibration plots in both training and validation cohort. Results: The calibration curves for probability of 1-, 3-, and 5-year OS in both cohorts showed favorable accordance between the nomogram prediction and the actual observation. The C-indices of the nomograms to predict OS were 0.71 in training cohort and 0.69 in the SEER cohort, which were higher than that of the seventh edition American Joint Committee on Cancer TNM staging system for predicting OS (training cohort, 0.71 vs. 0.58, respectively; P-value < 0.001; validation cohort, 0.69 vs. 0.57, respectively; P-value < 0.001). Conclusion: We developed and validated a novel nomogram based on CEA and other factors for predicting OS in patients with resected RC, which could assist clinical decision making and improvement of prognosis prediction for individual RC patients after surgery.


2017 ◽  
Vol 63 (2) ◽  
pp. 121-125 ◽  
Author(s):  
Adrian C Traeger ◽  
Markus Hübscher ◽  
James H McAuley

Author(s):  
Zaki Arshad ◽  
Henry David Maughan ◽  
Karadi Hari Sunil Kumar ◽  
Matthew Pettit ◽  
Arvind Arora ◽  
...  

Abstract Purpose The aim of this study was investigate the relationship between version and torsional abnormalities of the acetabulum, femur and tibia in patients with symptomatic FAI. Methods A systematic review was performed according to PRISMA guidelines using the EMBASE, MEDLINE, PubMed and Cochrane databases. Original research articles evaluating the described version and torsional parameters in FAI were included. The MINORS criteria were used to appraise study quality and risk of bias. Mean version and torsion values were displayed using forest plots and the estimated proportion of hips displaying abnormalities in version/torsion were calculated. Results A total of 1206 articles were identified from the initial search, with 43 articles, involving 8861 hips, meeting the inclusion criteria. All studies evaluating femoral or acetabular version in FAI reported ‘normal’ mean version values (10–25 °). However, distribution analysis revealed that an estimated 31% and 51% of patients with FAI displayed abnormal central acetabular and femoral version, respectively. Conclusion Up to 51% of patients presenting with symptomatic FAI show an abnormal femoral version, whilst up to 31% demonstrate abnormal acetabular version. This high percentage of version abnormalities highlights the importance of evaluating these parameters routinely during assessment of patients with FAI, to guide clinical decision-making. Level of evidence IV.


2020 ◽  
Author(s):  
Dennis Shung ◽  
Cynthia Tsay ◽  
Loren Laine ◽  
Prem Thomas ◽  
Caitlin Partridge ◽  
...  

Background and AimGuidelines recommend risk stratification scores in patients presenting with gastrointestinal bleeding (GIB), but such scores are uncommonly employed in practice. Automation and deployment of risk stratification scores in real time within electronic health records (EHRs) would overcome a major impediment. This requires an automated mechanism to accurately identify (“phenotype”) patients with GIB at the time of presentation. The goal is to identify patients with acute GIB by developing and evaluating EHR-based phenotyping algorithms for emergency department (ED) patients.MethodsWe specified criteria using structured data elements to create rules for identifying patients, and also developed a natural-language-processing (NLP)-based algorithm for automated phenotyping of patients, tested them with tenfold cross-validation (n=7144) and external validation (n=2988), and compared them with the standard method for encoding patient conditions in the EHR, Systematized Nomenclature of Medicine (SNOMED). The gold standard for GIB diagnosis was independent dual manual review of medical records. The primary outcome was positive predictive value (PPV).ResultsA decision rule using GIB-specific terms from ED triage and from ED review-of-systems assessment performed better than SNOMED on internal validation (PPV=91% [90%-93%] vs. 74% [71%-76%], P<0.001) and external validation (PPV=85% [84%-87%] vs. 69% [67%-71%], P<0.001). The NLP algorithm (external validation PPV=80% [79-82%]) was not superior to the structured-datafields decision rule.ConclusionsAn automated decision rule employing GIB-specific triage and review-of-systems terms can be used to trigger EHR-based deployment of risk stratification models to guide clinical decision-making in real time for patients with acute GIB presenting to the ED.


2020 ◽  
Vol 10 (1) ◽  
Author(s):  
David J. Altschul ◽  
Santiago R. Unda ◽  
Joshua Benton ◽  
Rafael de la Garza Ramos ◽  
Phillip Cezayirli ◽  
...  

Abstract COVID-19 is commonly mild and self-limiting, but in a considerable portion of patients the disease is severe and fatal. Determining which patients are at high risk of severe illness or mortality is essential for appropriate clinical decision making. We propose a novel severity score specifically for COVID-19 to help predict disease severity and mortality. 4711 patients with confirmed SARS-CoV-2 infection were included. We derived a risk model using the first half of the cohort (n = 2355 patients) by logistic regression and bootstrapping methods. The discriminative power of the risk model was assessed by calculating the area under the receiver operating characteristic curves (AUC). The severity score was validated in a second half of 2356 patients. Mortality incidence was 26.4% in the derivation cohort and 22.4% in the validation cohort. A COVID-19 severity score ranging from 0 to 10, consisting of age, oxygen saturation, mean arterial pressure, blood urea nitrogen, C-Reactive protein, and the international normalized ratio was developed. A ROC curve analysis was performed in the derivation cohort achieved an AUC of 0.824 (95% CI 0.814–0.851) and an AUC of 0.798 (95% CI 0.789–0.818) in the validation cohort. Furthermore, based on the risk categorization the probability of mortality was 11.8%, 39% and 78% for patient with low (0–3), moderate (4–6) and high (7–10) COVID-19 severity score. This developed and validated novel COVID-19 severity score will aid physicians in predicting mortality during surge periods.


BMJ Open ◽  
2019 ◽  
Vol 9 (12) ◽  
pp. e033374 ◽  
Author(s):  
Daniela Balzi ◽  
Giulia Carreras ◽  
Francesco Tonarelli ◽  
Luca Degli Esposti ◽  
Paola Michelozzi ◽  
...  

ObjectiveIdentification of older patients at risk, among those accessing the emergency department (ED), may support clinical decision-making. To this purpose, we developed and validated the Dynamic Silver Code (DSC), a score based on real-time linkage of administrative data.Design and settingThe ‘Silver Code National Project (SCNP)’, a non-concurrent cohort study, was used for retrospective development and internal validation of the DSC. External validation was obtained in the ‘Anziani in DEA (AIDEA)’ concurrent cohort study, where the DSC was generated by the software routinely used in the ED.ParticipantsThe SCNP contained 281 321 records of 180 079 residents aged 75+ years from Tuscany and Lazio, Italy, admitted via the ED to Internal Medicine or Geriatrics units. The AIDEA study enrolled 4425 subjects aged 75+ years (5217 records) accessing two EDs in the area of Florence, Italy.InterventionsNone.Outcome measuresPrimary outcome: 1-year mortality. Secondary outcomes: 7 and 30-day mortality and 1-year recurrent ED visits.ResultsAdvancing age, male gender, previous hospital admission, discharge diagnosis, time from discharge and polypharmacy predicted 1-year mortality and contributed to the DSC in the development subsample of the SCNP cohort. Based on score quartiles, participants were classified into low, medium, high and very high-risk classes. In the SCNP validation sample, mortality increased progressively from 144 to 367 per 1000 person-years, across DSC classes, with HR (95% CI) of 1.92 (1.85 to 1.99), 2.71 (2.61 to 2.81) and 5.40 (5.21 to 5.59) in class II, III and IV, respectively versus class I (p<0.001). Findings were similar in AIDEA, where the DSC predicted also recurrent ED visits in 1 year. In both databases, the DSC predicted 7 and 30-day mortality.ConclusionsThe DSC, based on administrative data available in real time, predicts prognosis of older patients and might improve their management in the ED.


2019 ◽  
Vol 33 (9) ◽  
pp. 1146-1157 ◽  
Author(s):  
Annie Pettifer ◽  
Katherine Froggatt ◽  
Sean Hughes

Background:Addressing the concerns of family members is an important aspect of palliative and end-of-life care. One aspect that commonly causes family caregivers concern is the decline of patients’ oral fluid intake in the last few days of life.Aim:To map the narratives in which family members’ experiences of witnessing the diminishing drinking of a dying relative have been researched, review the findings within each narrative and consider directions for future research.Design:An adapted meta-narrative review approach.Data Sources:The Cumulative Index of Nursing and Applied Health Literature, Medline, PsycINFO, Psycharticles and Scopus databases were searched for relevant research published between January 1982 and December 2017. Quality was assessed using the Quality Assessment and Review Instrument.Results:A total of 22 papers met the inclusion criteria. No study focused specifically on the experiences of family members when witnessing the diminishing drinking of dying relatives. However, research about diminishing drinking was identified within studies broadly focusing on cancer cachexia, clinical decision-making about hydration and/or nutrition and support in a hospice context. The research indicates that family members’ experiences of diminishing drinking vary with their views about the significance of drinking, dying well and their expectations of themselves and healthcare professionals.Conclusion:While some understanding of the topic can be inferred from research in related areas, there is a paucity of information specifically about family members’ experiences when witnessing the diminishing drinking of a dying relative.


Gut ◽  
2020 ◽  
pp. gutjnl-2019-319926 ◽  
Author(s):  
Waku Hatta ◽  
Yosuke Tsuji ◽  
Toshiyuki Yoshio ◽  
Naomi Kakushima ◽  
Shu Hoteya ◽  
...  

ObjectiveBleeding after endoscopic submucosal dissection (ESD) for early gastric cancer (EGC) is a frequent adverse event after ESD. We aimed to develop and externally validate a clinically useful prediction model (BEST-J score: Bleeding after ESD Trend from Japan) for bleeding after ESD for EGC.DesignThis retrospective study enrolled patients who underwent ESD for EGC. Patients in the derivation cohort (n=8291) were recruited from 25 institutions, and patients in the external validation cohort (n=2029) were recruited from eight institutions in other areas. In the derivation cohort, weighted points were assigned to predictors of bleeding determined in the multivariate logistic regression analysis and a prediction model was established. External validation of the model was conducted to analyse discrimination and calibration.ResultsA prediction model comprised 10 variables (warfarin, direct oral anticoagulant, chronic kidney disease with haemodialysis, P2Y12 receptor antagonist, aspirin, cilostazol, tumour size >30 mm, lower-third in tumour location, presence of multiple tumours and interruption of each kind of antithrombotic agents). The rates of bleeding after ESD at low-risk (0 to 1 points), intermediate-risk (2 points), high-risk (3 to 4 points) and very high-risk (≥5 points) were 2.8%, 6.1%, 11.4% and 29.7%, respectively. In the external validation cohort, the model showed moderately good discrimination, with a c-statistic of 0.70 (95% CI, 0.64 to 0.76), and good calibration (calibration-in-the-large, 0.05; calibration slope, 1.01).ConclusionsIn this nationwide multicentre study, we derived and externally validated a prediction model for bleeding after ESD. This model may be a good clinical decision-making support tool for ESD in patients with EGC.


Hematology ◽  
2018 ◽  
Vol 2018 (1) ◽  
pp. 110-117 ◽  
Author(s):  
Michele Ciboddo ◽  
Ann Mullally

Abstract Now that the spectrum of somatic mutations that initiate, propagate, and drive the progression of myeloproliferative neoplasms (MPNs) has largely been defined, recent efforts have focused on integrating this information into clinical decision making. In this regard, the greatest progress has been made in myelofibrosis, in which high-molecular-risk mutations have been identified and incorporated into prognostic models to help guide treatment decisions. In this chapter, we focus on advances in 4 main areas: (1) What are the MPN phenotypic driver mutations? (2) What constitutes high molecular risk in MPN (focusing on ASXL1)? (3) How do we risk-stratify patients with MPN? And (4) What is the significance of molecular genetics for MPN treatment? Although substantial progress has been made, we still have an incomplete understanding of the molecular basis for phenotypic diversity in MPN, and few rationally designed therapeutic approaches to target high-risk mutations are available. Ongoing research efforts in these areas are critical to understanding the biological consequences of genetic heterogeneity in MPN and to improving outcomes for patients.


Sign in / Sign up

Export Citation Format

Share Document