P3429A novel prediction model for mortality after cardiac surgery using institutional case volume

2019 ◽  
Vol 40 (Supplement_1) ◽  
Author(s):  
S Lee ◽  
J B Park ◽  
Y J Cho ◽  
H G Ryu ◽  
E J Jang
Keyword(s):  
Cardiac Surgery ◽  
Hospital Mortality ◽  
Risk Prediction ◽  
Prediction Models ◽  
Validation Cohort ◽  
Mortality Data ◽  
P Value ◽  
Case Volume ◽  
Risk Prediction Models ◽  
Derivation Cohort

Abstract Purpose A number of risk prediction models have been developed to identify short term mortality after cardiovascular surgery. Most models include patient characteristics, laboratory data, and type of surgery, but no consideration for the amount of surgical experience. With numerous reports on the impact of case volume on patient outcome after high risk procedures, we attempted to develop a risk prediction models for in-hospital and 1-year mortality that takes institutional case volume into account. Methods We identified adult patients who underwent cardiac surgery from January 2008 to December 2017 from the National Health Insurance Service (NHIS) database by searching for patients with procedure codes of coronary artery bypass grafting, valve surgery, and surgery on thoracic aorta during the hospitalization. Study subjects were randomly assigned to either the derivation cohort or the validation cohort. In-hospital mortality and 1-year mortality data were collected using the NHIS database. Risk prediction models were developed from the derivation cohort using Cox proportional hazards regression. The prediction performances of models were evaluated in the validation cohort. Results The models developed in this study demonstrated fair discrimination for derivation cohort (N=22,004, c-statistics, 0.75 for in-hospital mortality; 0.73 for 1-year mortality) and acceptable calibration in the validation cohort. (N=22,003, Hosmer-Lemeshow χ2-test, P=0.08 and 0.16, respectively). Case volume was the key factor of mortality prediction models after cardiac surgery. (50≤ x <100 case per year. 100≤ x <200 case per year, ≥200 case per year are correlated with OR 3.29, 2.49, 1.85 in in-hospital mortality, 2.76, 1.99, 1.69 in 1-year mortality respectively, P value <0.001.) Annual case volume as risk factor Variables In-hospital mortality 1-year mortality OR (95% CI) p-value OR (95% CI) p-value Annual case-volume (reference: ≥200) – – 100–200 1.69 (1.48, 1.93) <0.001 1.85 (1.58, 2.18) <0.001 50–100 1.99 (1.75, 2.25) <0.001 2.49 (2.15, 2.89) <0.001 <50 2.76 (2.44, 3.11) <0.001 3.29 (2.85, 3.79) <0.001 OR: Odds ratio; CI: confidence interval; Ref: Reference. Discrimination and calibration Conclusion We developed and validated new risk prediction models for in-hospital and 1-year mortality after cardiac surgery using the NHIS database. These models may provide useful guides to predict mortality risks of patients with basic information and without laboratory findings.

scholarly journals Development and validation of two SCORE-based cardiovascular risk prediction models for Eastern Europe: a multicohort study

2020 ◽  
Vol 41 (35) ◽  
pp. 3325-3333 ◽  
Author(s):  
Taavi Tillmann ◽  
Kristi Läll ◽  
Oliver Dukes ◽  
Giovanni Veronesi ◽  
Hynek Pikhart ◽  
...  
Keyword(s):  
Risk Factors ◽  
Prediction Models ◽  
Validation Cohort ◽  
European Countries ◽  
Eastern European ◽  
Risk Prediction Models ◽  
Derivation Cohort ◽  
Score Model ◽  
Eastern European Countries

Abstract Aims Cardiovascular disease (CVD) risk prediction models are used in Western European countries, but less so in Eastern European countries where rates of CVD can be two to four times higher. We recalibrated the SCORE prediction model for three Eastern European countries and evaluated the impact of adding seven behavioural and psychosocial risk factors to the model. Methods and results We developed and validated models using data from the prospective HAPIEE cohort study with 14 598 participants from Russia, Poland, and the Czech Republic (derivation cohort, median follow-up 7.2 years, 338 fatal CVD cases) and Estonian Biobank data with 4632 participants (validation cohort, median follow-up 8.3 years, 91 fatal CVD cases). The first model (recalibrated SCORE) used the same risk factors as in the SCORE model. The second model (HAPIEE SCORE) added education, employment, marital status, depression, body mass index, physical inactivity, and antihypertensive use. Discrimination of the original SCORE model (C-statistic 0.78 in the derivation and 0.83 in the validation cohorts) was improved in recalibrated SCORE (0.82 and 0.85) and HAPIEE SCORE (0.84 and 0.87) models. After dichotomizing risk at the clinically meaningful threshold of 5%, and when comparing the final HAPIEE SCORE model against the original SCORE model, the net reclassification improvement was 0.07 [95% confidence interval (CI) 0.02–0.11] in the derivation cohort and 0.14 (95% CI 0.04–0.25) in the validation cohort. Conclusion Our recalibrated SCORE may be more appropriate than the conventional SCORE for some Eastern European populations. The addition of seven quick, non-invasive, and cheap predictors further improved prediction accuracy.

scholarly journals Generalizability Challenges of Mortality Risk Prediction Models: A Retrospective Analysis on a Multi-center Database

2021 ◽  
Author(s):  
Harvineet Singh ◽  
Vishwali Mhasawade ◽  
Rumi Chunara
Keyword(s):  
Hospital Mortality ◽  
Risk Prediction ◽  
Performance Metrics ◽  
Prediction Models ◽  
Characteristic Curve ◽  
False Negative ◽  
Model Performance ◽  
Clinical Practices ◽  
Cross Sectional ◽  
Risk Prediction Models

Importance: Modern predictive models require large amounts of data for training and evaluation which can result in building models that are specific to certain locations, populations in them and clinical practices. Yet, best practices and guidelines for clinical risk prediction models have not yet considered such challenges to generalizability. Objectives: To investigate changes in measures of predictive discrimination, calibration, and algorithmic fairness when transferring models for predicting in-hospital mortality across ICUs in different populations. Also, to study the reasons for the lack of generalizability in these measures. Design, Setting, and Participants: In this multi-center cross-sectional study, electronic health records from 179 hospitals across the US with 70,126 hospitalizations were analyzed. Time of data collection ranged from 2014 to 2015. Main Outcomes and Measures: The main outcome is in-hospital mortality. Generalization gap, defined as difference between model performance metrics across hospitals, is computed for discrimination and calibration metrics, namely area under the receiver operating characteristic curve (AUC) and calibration slope. To assess model performance by race variable, we report differences in false negative rates across groups. Data were also analyzed using a causal discovery algorithm "Fast Causal Inference" (FCI) that infers paths of causal influence while identifying potential influences associated with unmeasured variables. Results: In-hospital mortality rates differed in the range of 3.9%-9.3% (1st-3rd quartile) across hospitals. When transferring models across hospitals, AUC at the test hospital ranged from 0.777 to 0.832 (1st to 3rd quartile; median 0.801); calibration slope from 0.725 to 0.983 (1st to 3rd quartile; median 0.853); and disparity in false negative rates from 0.046 to 0.168 (1st to 3rd quartile; median 0.092). When transferring models across geographies, AUC ranged from 0.795 to 0.813 (1st to 3rd quartile; median 0.804); calibration slope from 0.904 to 1.018 (1st to 3rd quartile; median 0.968); and disparity in false negative rates from 0.018 to 0.074 (1st to 3rd quartile; median 0.040). Distribution of all variable types (demography, vitals, and labs) differed significantly across hospitals and regions. Shifts in the race variable distribution and some clinical (vitals, labs and surgery) variables by hospital or region. Race variable also mediates differences in the relationship between clinical variables and mortality, by hospital/region. Conclusions and Relevance: Group-specific metrics should be assessed during generalizability checks to identify potential harms to the groups. In order to develop methods to improve and guarantee performance of prediction models in new environments for groups and individuals, better understanding and provenance of health processes as well as data generating processes by sub-group are needed to identify and mitigate sources of variation.

scholarly journals An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: national validation cohort study in England

2021 ◽  
Author(s):  
Vahe Nafilyan ◽  
Ben Humberstone ◽  
Nisha Mehta ◽  
Ian Diamond ◽  
Carol Coupland ◽  
...  
Keyword(s):  
Risk Prediction ◽  
Prediction Models ◽  
Validation Cohort ◽  
Population Based ◽  
Prediction Algorithm ◽  
Risk Prediction Models ◽  
Men And Women ◽  
Time Period ◽  
Time Periods ◽  
First Time

SUMMARYBackgroundTo externally validate a risk prediction algorithm (QCovid) to estimate mortality outcomes from COVID-19 in adults in England.MethodsPopulation-based cohort study using the ONS Public Health Linked Data Asset, a cohort based on the 2011 Census linked to Hospital Episode Statistics, the General Practice Extraction Service Data for pandemic planning and research, radiotherapy and systemic chemotherapy records. The primary outcome was time to COVID-19 death, defined as confirmed or suspected COVID-19 death as per death certification. Two time periods were used: (a) 24th January to 30th April 2020; and (b) 1st May to 28th July 2020. We evaluated the performance of the QCovid algorithms using measures of discrimination and calibration for each validation time period.FindingsThe study comprises 34,897,648 adults aged 19-100 years resident in England. There were 26,985 COVID-19 deaths during the first time-period and 13,177 during the second. The algorithms had good calibration in the validation cohort in both time periods with close correspondence of observed and predicted risks. They explained 77.1% (95% CI: 76.9% to 77.4%) of the variation in time to death in men in the first time-period (R2); the D statistic was 3.76 (95% CI: 3.73 to 3.79); Harrell’s C was 0.935 (0.933 to 0.937). Similar results were obtained for women, and in the second time-period. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths in the first time period was 65.9% for men and 71.7% for women. People in the top 20% of predicted risks of death accounted for 90.8% of all COVID-19 deaths for men and 93.0% for women.InterpretationThe QCovid population-based risk algorithm performed well, showing very high levels of discrimination for COVID-19 deaths in men and women for both time periods. It has the potential to be dynamically updated as the pandemic evolves and therefore, has potential use in guiding national policy.FundingNational Institute of Health ResearchRESEARCH IN CONTEXTEvidence before this studyPublic policy measures and clinical risk assessment relevant to COVID-19 need to be aided by rigorously developed and validated risk prediction models. A recent living systematic review of published risk prediction models for COVID-19 found most models are subject to a high risk of bias with optimistic reported performance, raising concern that these models may be unreliable when applied in practice. A population-based risk prediction model, QCovid risk prediction algorithm, has recently been developed to identify adults at high risk of serious COVID-19 outcomes, which overcome many of the limitations of previous tools.Added value of this studyCommissioned by the Chief Medical Officer for England, we validated the novel clinical risk prediction model (QCovid) to identify risks of short-term severe outcomes due to COVID-19. We used national linked datasets from general practice, death registry and hospital episode data for a population-representative sample of over 34 million adults. The risk models have excellent discrimination in men and women (Harrell’s C statistic>0.9) and are well calibrated. QCovid represents a new, evidence-based opportunity for population risk-stratification.Implications of all the available evidenceQCovid has the potential to support public health policy, from enabling shared decision making between clinicians and patients in relation to health and work risks, to targeted recruitment for clinical trials, and prioritisation of vaccination, for example.

scholarly journals Predicting Acute Kidney Injury after Cardiac Surgery by Machine Learning Approaches

2020 ◽  
Vol 9 (6) ◽  
pp. 1767 ◽  
Author(s):  
Charat Thongprayoon ◽  
Panupong Hansrivijit ◽  
Tarun Bathini ◽  
Saraschandra Vallabhajosyula ◽  
Poemlarp Mekraksakit ◽  
...  
Keyword(s):  
Machine Learning ◽  
Cardiac Surgery ◽  
Risk Prediction ◽  
Prediction Models ◽  
Clinical Medicine ◽  
Kidney Injury ◽  
Risk Scores ◽  
Learning Approaches ◽  
Risk Prediction Models ◽  
General Utility

Cardiac surgery-associated AKI (CSA-AKI) is common after cardiac surgery and has an adverse impact on short- and long-term mortality. Early identification of patients at high risk of CSA-AKI by applying risk prediction models allows clinicians to closely monitor these patients and initiate effective preventive and therapeutic approaches to lessen the incidence of AKI. Several risk prediction models and risk assessment scores have been developed for CSA-AKI. However, the definition of AKI and the variables utilized in these risk scores differ, making general utility complex. Recently, the utility of artificial intelligence coupled with machine learning, has generated much interest and many studies in clinical medicine, including CSA-AKI. In this article, we discussed the evolution of models established by machine learning approaches to predict CSA-AKI.

scholarly journals 1657 Systematic Review and Meta-Analysis of Mortality Risk Prediction Models in Adult Cardiac Surgery

2021 ◽  
Vol 108 (Supplement_6) ◽  
Author(s):  
S Sinha ◽  
A Dimagli ◽  
L Dixon ◽  
M Gaudino ◽  
M Caputo ◽  
...  
Keyword(s):  
Cardiac Surgery ◽  
Risk Prediction ◽  
Mortality Risk ◽  
Prediction Models ◽  
Meta Analysis ◽  
Group Analysis ◽  
Risk Prediction Models ◽  
Geographical Variations ◽  
Significant Difference ◽  
Adult Cardiac Surgery

Abstract Background The most used mortality risk prediction models in cardiac surgery are the European System for Cardiac Operative Risk Evaluation(EuroSCORE)(ES) and Society of Thoracic Surgeons(STS) score. There is no agreement on which score should be considered more accurate nor which score should be utilised in each population sub-group. We sought to provide a thorough quantitative assessment of these 2 models. Method We performed a systematic literature review and captured information on discrimination, as quantified by the area under the receiver operator curve(AUC), and calibration, as quantified by the ratio of observed-to-expected mortality(O:E). We performed random effects meta-analysis of the performance of the individual models as well as pairwise comparisons and sub-group analysis by procedure type, time and continent. Results The ES2(AUC 0.783[95%CI 0.765-0.800];O:E 1.102[95%CI 0.943-1.289]) and STS(AUC 0.757[95%CI 0.727-0.785];O:E 1.111[95%CI 0.853-1.447]) both showed good overall discrimination and calibration. There was no significant difference in the discrimination of the two models(Difference in AUC -0.016; 95%CI -0.034 to -0.002;p0.09). However, the calibration of ES2 showed significant geographical variations(p &lt; 0.001) and a trend towards miscalibration with time(p0.0057). This was not seen with STS. Conclusions ES2 and STS are both reliable predictors of short-term mortality following adult cardiac surgery in the populations from which they were derived. STS may have broader applications when comparing outcomes across continents and time periods as compared to ES2.

Sign in / Sign up

Export Citation Format

Share Document