Derivation and validation of a prognostic model for predicting in-hospital mortality in patients admitted with COVID-19 in Wuhan, China: the PLANS (platelet lymphocyte age neutrophil sex) model

Abstract Background Previous published prognostic models for COVID-19 patients have been suggested to be prone to bias due to unrepresentativeness of patient population, lack of external validation, inappropriate statistical analyses, or poor reporting. A high-quality and easy-to-use prognostic model to predict in-hospital mortality for COVID-19 patients could support physicians to make better clinical decisions. Methods Fine-Gray models were used to derive a prognostic model to predict in-hospital mortality (treating discharged alive from hospital as the competing event) in COVID-19 patients using two retrospective cohorts (n = 1008) in Wuhan, China from January 1 to February 10, 2020. The proposed model was internally evaluated by bootstrap approach and externally evaluated in an external cohort (n = 1031). Results The derivation cohort was a case-mix of mild-to-severe hospitalized COVID-19 patients (43.6% females, median age 55). The final model (PLANS), including five predictor variables of platelet count, lymphocyte count, age, neutrophil count, and sex, had an excellent predictive performance (optimism-adjusted C-index: 0.85, 95% CI: 0.83 to 0.87; averaged calibration slope: 0.95, 95% CI: 0.82 to 1.08). Internal validation showed little overfitting. External validation using an independent cohort (47.8% female, median age 63) demonstrated excellent predictive performance (C-index: 0.87, 95% CI: 0.85 to 0.89; calibration slope: 1.02, 95% CI: 0.92 to 1.12). The averaged predicted cumulative incidence curves were close to the observed cumulative incidence curves in patients with different risk profiles. Conclusions The PLANS model based on five routinely collected predictors would assist clinicians in better triaging patients and allocating healthcare resources to reduce COVID-19 fatality.

Download Full-text

Derivation and validation of a prognostic model for predicting in-hospital mortality in patients admitted with COVID-19 in Wuhan, China: the PLANS (Platelet Lymphocyte Age Neutrophil Sex) model

10.1101/2020.05.13.20100370 ◽

2020 ◽

Author(s):

Jiong Li ◽

Yuntao Chen ◽

Shujing Chen ◽

Sihua Wang ◽

Dingyu Zhang ◽

...

Keyword(s):

Hospital Mortality ◽

Clinical Characteristics ◽

Prognostic Model ◽

External Validation ◽

Predictive Performance ◽

Predictor Variables ◽

Survival Curves ◽

Internal Validation ◽

Derivation Cohort ◽

Union Hospital

OBJECTIVE To develop and validate a prognostic model for in-hospital mortality in COVID-19 patients using routinely collected demographic and clinical characteristics. DESIGN Multicenter, retrospective cohort study. SETTING Jinyintan Hospital, Union Hospital, and Tongji Hosptial in Wuhan, China. PARTICIPANTS A pooled derivation cohort of 1008 COVID-19 patients from Jinyintan Hospital, Union Hospital in Wuhan and an external validation cohort of 1031 patients from Tongji Hospital in Wuhan, China. MAIN OUTCOME MEASURES Outcome of interest was in-hospital mortality, treating discharged alive from hospital as the competing event. Fine-Gray models, using backward elimination for inclusion of predictor variables and allowing non-linear effects of continuous variables, were used to derive a prognostic model for predicting in-hospital mortality among COVID-19 patients. Internal validation was implemented to check model overfitting using bootstrap approach. External validation to a separate hospital was implemented to evaluate the generalizability of the model. RESULTS The derivation cohort was a case-mix of mild-to-severe hospitalized COVID-19 patients (n=1008, 43.6% females, median age 55). The final model (PLANS), including five predictor variables of platelet count, lymphocyte count, age, neutrophil count, and sex, had an excellent predictive performance (optimism-adjusted C-index: 0.85, 95% CI: 0.83 to 0.87; averaged calibration slope: 0.95, 95% CI: 0.82 to 1.08). Internal validation showed little overfitting. External validation using an independent cohort (n=1031, 47.8% female, median age 63) demonstrated excellent predictive performance (C-index: 0.87, 95% CI: 0.85 to 0.89; calibration slope: 1.02, 95% CI: 0.92 to 1.12). The averaged predicted survival curves were close to the observed survival curves across patients with different risk profiles. CONCLUSIONS The PLANS model based on the five routinely collected demographic and clinical characteristics (platelet count, lymphocyte count, age, neutrophil count, and sex) showed excellent discriminative and calibration accuracy in predicting in-hospital mortality in COVID-19 patients. This prognostic model would assist clinicians in better triaging patients and allocating healthcare resources to reduce COVID-19 fatality.

Download Full-text

Prognostic model research

Prognosis Research in Health Care ◽

10.1093/med/9780198796619.003.0008 ◽

2019 ◽

pp. 139-187

Author(s):

Richard D Riley ◽

Karel GM Moons ◽

Thomas PA Debray ◽

Kym IE Snell ◽

Ewout W Steyerberg ◽

...

Keyword(s):

Decision Making ◽

Prognostic Model ◽

Model Updating ◽

External Validation ◽

Predictive Performance ◽

Prognostic Models ◽

Net Benefit ◽

Internal Validation ◽

Model Research ◽

Future Outcomes

Prognostic models combine multiple prognostic factors to estimate the risk of future outcomes in individuals with a particular disease or health condition. A useful model provides accurate predictions to support decision making by individuals and caregivers. This chapter describes the three phases of prognostic model research development (including internal validation), external validation (including model updating), and impact on decision making and individual health outcomes. Methodology is detailed for each phase, including the need for large representative datasets, methods to avoid or reduce overfitting and optimism, and the use of both discrimination and calibration to assess a model’s predictive performance. TRIPOD reporting guidelines are introduced. Emphasis is also given to the application of models in practice, including linking the model to clinical decisions using risk thresholds, and evaluating this using measures of net benefit, decision curves, cost-effectiveness analyses, and impact studies (such as randomized trials) to evaluate the effectiveness of models in improving outcomes.

Download Full-text

Development and Validation of a Prognostic Model for Transplant-Associated Thrombotic Microangiopathy Following Allogeneic Hematopoietic Stem Cell Transplantation

Blood ◽

10.1182/blood-2020-138408 ◽

2020 ◽

Vol 136 (Supplement 1) ◽

pp. 16-17

Author(s):

Peng Zhao ◽

Ye-Jun Wu ◽

Qing-Yuan Qu ◽

Shan Chong ◽

Xiao-Wan Sun ◽

...

Keyword(s):

Overall Survival ◽

Prognostic Model ◽

Validation Cohort ◽

Risk Group ◽

External Validation ◽

Internal Validation ◽

Hematopoietic Stem ◽

Derivation Cohort ◽

Peking University ◽

Score Model

Introduction Transplant-associated thrombotic microangiopathy (TA-TMA) is a potentially life-threatening complication of allogeneic hematopoietic stem cell transplantation (allo-HSCT), which can result in multiorgan injury and increased risk for mortality. Renewed interest has emerged in the prognostication of TA-TMA with the development of novel diagnostic and management algorithms. Our previous study reported an adverse outcome in patients with TA-TMA and concomitant acute graft-versus-host disease (Eur J Haematol, 2018). However, information on markers for the early identification of severe cases remains limited. Therefore, this study is concentrated on the development and validation of a prognostic model for TA-TMA, which might facilitate risk stratification and contribute to individualized management. Methods Patients receiving allo-HSCT in Peking University People's Hospital with 1) a diagnosis of microangiopathic hemolytic anemia (MAHA) or 2) evidence of microangiopathy were retrospectively identified from 2010 to 2018. The diagnosis of TA-TMA was reviewed according to the Overall-TMA criteria (Transplantation, 2010). Patients without fulfillment of the diagnostic criteria or complicated with other causes of MAHA were excluded from analysis. Prognostic factors for TA-TMA were determined among patients receiving HSCT between 2010 and 2014 (derivation cohort). Candidate predictors (univariate P < 0.1) were included in the multivariate analysis using a backward stepwise logistic regression model. A risk score model was then established according to the regression coefficient of each independent prognostic factor. The performance of this predictive model was evaluated through internal validation (bootstrap method with 1000 repetitions) and external temporal validation performed on data from those who received HSCT between 2015 and 2018 (validation cohort). Results 5337 patients underwent allo-HSCT at Peking University Institute of Hematology from 2010 to 2018. A total of 1255 patients with a diagnosis of MAHA and/or evidence of microangiopathy were retrospectively identified, among whom 493 patients met the inclusion criteria for this analysis (269 in the derivation cohort and 224 in the validation cohort). The median age at the time of TA-TMA diagnosis was 28 (IQR: 17-41) years. The median duration from the time of transplantation to the diagnosis of TA-TMA was 63 (IQR: 38-121) days. The 6-month overall survival rate was 42.2% (208/493), and the 1-year overall survival rate was 45.0% (222/493). In the derivation cohort, patient age (≥35 years), anemia (hemoglobin <70 g/L), severe thrombocytopenia (platelet count <15,000/μL), elevated lactic dehydrogenase (serum LDH >800 U/L) and elevated total bilirubin (TBIL >1.5*ULN) were identified by multivariate analysis as independent prognostic factors for the 6-month outcome of TA-TMA. A risk score model was constructed according to the regression coefficients (Table 1), and patients were stratified into a low-risk group (0-1 points), an intermediate-risk group (2-4 points) and a high-risk group (5-6 points). The Kaplan-Meier estimations of overall survival separated well between these risk groups (Figure 1). The prognostic model showed significant discriminatory capacity, with a cross-validated c-index of 0.770 (95%CI, 0.714-0.826) in the internal validation and 0.768 (95%CI, 0.707-0.829) in the external validation cohort. The calibration plots also indicated a good correlation between model-predicted and observed probabilities. Conclusions A prognostic model for TA-TMA incorporating several baseline laboratory factors was developed and evaluated, which demonstrated significant predictive capacity through internal and external validation. This predictive model might facilitate prognostication of TA-TMA and contribute to early identification of patients at higher risk for adverse outcomes. Further study may focus on whether these high-risk patients could benefit from early application of specific management. Disclosures No relevant conflicts of interest to declare.

Download Full-text

Early prediction of noninvasive ventilation failure in COPD patients: derivation, internal validation, and external validation of a simple risk score

Annals of Intensive Care ◽

10.1186/s13613-019-0585-9 ◽

2019 ◽

Vol 9 (1) ◽

Cited By ~ 7

Author(s):

Jun Duan ◽

Shengyu Wang ◽

Ping Liu ◽

Xiaoli Han ◽

Yao Tian ◽

...

Keyword(s):

Hospital Mortality ◽

Noninvasive Ventilation ◽

Scoring System ◽

External Validation ◽

Chronic Obstructive ◽

Internal Validation ◽

Obstructive Pulmonary Disease ◽

Derivation Cohort ◽

Copd Patients ◽

Early Intubation

Abstract Background Early identification of noninvasive ventilation (NIV) failure is a promising strategy for reducing mortality in chronic obstructive pulmonary disease (COPD) patients. However, a risk-scoring system is lacking. Methods To develop a scale to predict NIV failure, 500 COPD patients were enrolled in a derivation cohort. Heart rate, acidosis (assessed by pH), consciousness (assessed by Glasgow coma score), oxygenation, and respiratory rate (HACOR) were entered into the scoring system. Another two groups of 323 and 395 patients were enrolled to internally and externally validate the scale, respectively. NIV failure was defined as intubation or death during NIV. Results Using HACOR score collected at 1–2 h of NIV to predict NIV failure, the area under the receiver operating characteristic curves (AUC) was 0.90, 0.89, and 0.71 for the derivation, internal-validation, and external-validation cohorts, respectively. For the prediction of early NIV failure in these three cohorts, the AUC was 0.91, 0.96, and 0.83, respectively. In all patients with HACOR score > 5, the NIV failure rate was 50.2%. In these patients, early intubation (< 48 h) was associated with decreased hospital mortality (unadjusted odds ratio = 0.15, 95% confidence interval 0.05–0.39, p < 0.01). Conclusions HACOR scores exhibited good predictive power for NIV failure in COPD patients, particularly for the prediction of early NIV failure (< 48 h). In high-risk patients, early intubation was associated with decreased hospital mortality.

Download Full-text

Development and validation of a prediction model for fat mass in children and adolescents: meta-analysis using individual participant data

BMJ ◽

10.1136/bmj.l4293 ◽

2019 ◽

pp. l4293 ◽

Cited By ~ 3

Author(s):

Mohammed T Hudda ◽

Mary S Fewtrell ◽

Dalia Haroun ◽

Sooky Lum ◽

Jane E Williams ◽

...

Keyword(s):

Confidence Interval ◽

Prediction Model ◽

Fat Mass ◽

Meta Analysis ◽

External Validation ◽

Predictive Performance ◽

Individual Participant Data ◽

Internal Validation ◽

Individual Participant ◽

Complex Forms

Abstract Objectives To develop and validate a prediction model for fat mass in children aged 4-15 years using routinely available risk factors of height, weight, and demographic information without the need for more complex forms of assessment. Design Individual participant data meta-analysis. Setting Four population based cross sectional studies and a fifth study for external validation, United Kingdom. Participants A pooled derivation dataset (four studies) of 2375 children and an external validation dataset of 176 children with complete data on anthropometric measurements and deuterium dilution assessments of fat mass. Main outcome measure Multivariable linear regression analysis, using backwards selection for inclusion of predictor variables and allowing non-linear relations, was used to develop a prediction model for fat-free mass (and subsequently fat mass by subtracting resulting estimates from weight) based on the four studies. Internal validation and then internal-external cross validation were used to examine overfitting and generalisability of the model’s predictive performance within the four development studies; external validation followed using the fifth dataset. Results Model derivation was based on a multi-ethnic population of 2375 children (47.8% boys, n=1136) aged 4-15 years. The final model containing predictor variables of height, weight, age, sex, and ethnicity had extremely high predictive ability (optimism adjusted R 2 : 94.8%, 95% confidence interval 94.4% to 95.2%) with excellent calibration of observed and predicted values. The internal validation showed minimal overfitting and good model generalisability, with excellent calibration and predictive performance. External validation in 176 children aged 11-12 years showed promising generalisability of the model (R 2 : 90.0%, 95% confidence interval 87.2% to 92.8%) with good calibration of observed and predicted fat mass (slope: 1.02, 95% confidence interval 0.97 to 1.07). The mean difference between observed and predicted fat mass was −1.29 kg (95% confidence interval −1.62 to −0.96 kg). Conclusion The developed model accurately predicted levels of fat mass in children aged 4-15 years. The prediction model is based on simple anthropometric measures without the need for more complex forms of assessment and could improve the accuracy of assessments for body fatness in children (compared with those provided by body mass index) for effective surveillance, prevention, and management of clinical and public health obesity.

Download Full-text

Prognostic prediction model for glioblastoma: a metabolic gene signature and independent external validation

10.21203/rs.3.rs-18209/v1 ◽

2020 ◽

Author(s):

Chuxiang Lei ◽

Wenlin Chen ◽

Yuekun Wang ◽

Binghao Zhao ◽

Penghao Liu ◽

...

Keyword(s):

Prognostic Model ◽

Validation Cohort ◽

External Validation ◽

Cox Proportional Hazards ◽

The Cancer Genome Atlas ◽

Prognostic Models ◽

Normal Brain ◽

Protective Effects ◽

Lasso Regression ◽

Metabolic Genes

Abstract Background. Glioblastoma (GBM) is the most common primary malignant intracranial tumor and is closely related to metabolic alterations. However, few accepted prognostic models are currently available, especially models based on metabolic genes. Methods . Transcriptome data were obtained for all patients diagnosed with GBM from the Gene Expression Omnibus (GEO) (training cohort, n=369) and The Cancer Genome Atlas (TCGA) (validation cohort, n=152) with the following variables: age at diagnosis, sex, follow-up and overall survival (OS). Metabolic genes according to Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways were filtered, and a Lasso regression model was constructed. Survival was assessed by univariate or multivariate Cox proportional hazards regression and Kaplan-Meier analysis, and we also conducted an independent external validation to examine the model. Results. There were 341 metabolic genes that showed significant differences between normal brain tissues and GBM tissues in both the training and validation cohorts, among which 56 genes were significantly correlated with the OS of patients. Lasso regression revealed that the metabolic prognostic model was composed of 18 genes, including COX10 , COMT , and GPX2 , with protective effects, as well as OCRL and RRM2 , with unfavorable effects. Patients classified as high-risk by the risk score from this model had markedly shorter OS than low-risk patients ( P <0.0001), and this significant result was also observed in the independent external validation cohort ( P <0.001). Conclusions . The prognosis of GBM was dramatically related to metabolic pathways, and our metabolic prognostic model had high accuracy and application value in predicting the OS of GBM patients. Background. Glioblastoma (GBM) is the most common primary malignant intracranial tumor and is closely related to metabolic alterations. However, few accepted prognostic models are currently available, especially models based on metabolic genes. Methods . Transcriptome data were obtained for all patients diagnosed with GBM from the Gene Expression Omnibus (GEO) (training cohort, n=369) and The Cancer Genome Atlas (TCGA) (validation cohort, n=152) with the following variables: age at diagnosis, sex, follow-up and overall survival (OS). Metabolic genes according to Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways were filtered, and a Lasso regression model was constructed. Survival was assessed by univariate or multivariate Cox proportional hazards regression and Kaplan-Meier analysis, and we also conducted an independent external validation to examine the model. Results. There were 341 metabolic genes that showed significant differences between normal brain tissues and GBM tissues in both the training and validation cohorts, among which 56 genes were significantly correlated with the OS of patients. Lasso regression revealed that the metabolic prognostic model was composed of 18 genes, including COX10 , COMT , and GPX2 , with protective effects, as well as OCRL and RRM2 , with unfavorable effects. Patients classified as high-risk by the risk score from this model had markedly shorter OS than low-risk patients ( P <0.0001), and this significant result was also observed in the independent external validation cohort ( P <0.001).Conclusions . The prognosis of GBM was dramatically related to metabolic pathways, and our metabolic prognostic model had high accuracy and application value in predicting the OS of GBM patients.

Download Full-text

Development and external validation of a prognostic tool for COVID-19 critical disease

PLoS ONE ◽

10.1371/journal.pone.0242953 ◽

2020 ◽

Vol 15 (12) ◽

pp. e0242953

Author(s):

Daniel S. Chow ◽

Justin Glavis-Bloom ◽

Jennifer E. Soun ◽

Brent Weinberg ◽

Theresa Berens Loveless ◽

...

Keyword(s):

Critical Care ◽

Prognostic Model ◽

Validation Cohort ◽

External Validation ◽

Vital Signs ◽

Clinical Tool ◽

Derivation Cohort ◽

Laboratory Values ◽

Increased Risk ◽

Care Capacity

Background The rapid spread of coronavirus disease 2019 (COVID-19) revealed significant constraints in critical care capacity. In anticipation of subsequent waves, reliable prediction of disease severity is essential for critical care capacity management and may enable earlier targeted interventions to improve patient outcomes. The purpose of this study is to develop and externally validate a prognostic model/clinical tool for predicting COVID-19 critical disease at presentation to medical care. Methods This is a retrospective study of a prognostic model for the prediction of COVID-19 critical disease where critical disease was defined as ICU admission, ventilation, and/or death. The derivation cohort was used to develop a multivariable logistic regression model. Covariates included patient comorbidities, presenting vital signs, and laboratory values. Model performance was assessed on the validation cohort by concordance statistics. The model was developed with consecutive patients with COVID-19 who presented to University of California Irvine Medical Center in Orange County, California. External validation was performed with a random sample of patients with COVID-19 at Emory Healthcare in Atlanta, Georgia. Results Of a total 3208 patients tested in the derivation cohort, 9% (299/3028) were positive for COVID-19. Clinical data including past medical history and presenting laboratory values were available for 29% (87/299) of patients (median age, 48 years [range, 21–88 years]; 64% [36/55] male). The most common comorbidities included obesity (37%, 31/87), hypertension (37%, 32/87), and diabetes (24%, 24/87). Critical disease was present in 24% (21/87). After backward stepwise selection, the following factors were associated with greatest increased risk of critical disease: number of comorbidities, body mass index, respiratory rate, white blood cell count, % lymphocytes, serum creatinine, lactate dehydrogenase, high sensitivity troponin I, ferritin, procalcitonin, and C-reactive protein. Of a total of 40 patients in the validation cohort (median age, 60 years [range, 27–88 years]; 55% [22/40] male), critical disease was present in 65% (26/40). Model discrimination in the validation cohort was high (concordance statistic: 0.94, 95% confidence interval 0.87–1.01). A web-based tool was developed to enable clinicians to input patient data and view likelihood of critical disease. Conclusions and relevance We present a model which accurately predicted COVID-19 critical disease risk using comorbidities and presenting vital signs and laboratory values, on derivation and validation cohorts from two different institutions. If further validated on additional cohorts of patients, this model/clinical tool may provide useful prognostication of critical care needs.

Download Full-text

Development and validation of nomograms for prediction of overall survival and cancer-specific survival of patients of colorectal cancer

Japanese Journal of Clinical Oncology ◽

10.1093/jjco/hyz182 ◽

2019 ◽

Vol 50 (3) ◽

pp. 261-269

Author(s):

Jieyun Zhang ◽

Yue Yang ◽

Xiaojian Fu ◽

Weijian Guo

Keyword(s):

Colorectal Cancer ◽

Overall Survival ◽

Prognostic Factors ◽

Prognostic Model ◽

Cox Regression ◽

External Validation ◽

Cancer Prognosis ◽

Cancer Specific Survival ◽

Internal Validation ◽

Development And Validation

Abstract Purpose Nomograms are intuitive tools for individualized cancer prognosis. We sought to develop a clinical nomogram for prediction of overall survival and cancer-specific survival for patients with colorectal cancer. Methods Patients with colorectal cancer diagnosed between 1988 and 2006 and those who underwent surgery were retrieved from the Surveillance, Epidemiology, and End Results database and randomly divided into the training (n = 119 797) and validation (n = 119 797) cohorts. Log-rank and multivariate Cox regression analyses were used in our analysis. To find out death from other cancer causes and non-cancer causes, a competing-risks model was used, based on which we integrated these significant prognostic factors into nomograms and subjected the nomograms to bootstrap internal validation and to external validation. Results The 1-, 3-, 5- and 10-year probabilities of overall survival in patients of colorectal cancer after surgery intervention were 83.04, 65.54, 54.79 and 38.62%, respectively. The 1-, 3-, 5- and 10-year cancer-specific survival was 87.36, 73.44, 66.22 and 59.11%, respectively. Nine independent prognostic factors for overall survival and nine independent prognostic factors for cancer specific survival were included to build the nomograms. Internal and external validation CI indexes of overall survival were 0.722 and 0.721, and those of cancer-specific survival were 0.765 and 0.766, which was satisfactory. Conclusions Nomograms for prediction of overall survival and cancer-specific survival of patients with colorectal cancer. Performance of the model was excellent. This practical prognostic model may help clinicians in decision-making and design of clinical studies.

Download Full-text

A Prediction Model for Severe AKI in Critically Ill Adults That Incorporates Clinical and Biomarker Data

Clinical Journal of the American Society of Nephrology ◽

10.2215/cjn.04100318 ◽

2019 ◽

Vol 14 (4) ◽

pp. 506-514 ◽

Cited By ~ 7

Author(s):

Pavan Kumar Bhatraju ◽

Leila R. Zelnick ◽

Ronit Katz ◽

Carmen Mikacenic ◽

Susanna Kosamo ◽

...

Keyword(s):

Critically Ill ◽

Critically Ill Patients ◽

Prediction Models ◽

Risk Model ◽

External Validation ◽

Variable Model ◽

Internal Validation ◽

Predictive Values ◽

Derivation Cohort ◽

C Statistic

Background and objectivesCritically ill patients with worsening AKI are at high risk for poor outcomes. Predicting which patients will experience progression of AKI remains elusive. We sought to develop and validate a risk model for predicting severe AKI within 72 hours after intensive care unit admission.Design, setting, participants, & measurementsWe applied least absolute shrinkage and selection operator regression methodology to two prospectively enrolled, critically ill cohorts of patients who met criteria for the systemic inflammatory response syndrome, enrolled within 24–48 hours after hospital admission. The risk models were derived and internally validated in 1075 patients and externally validated in 262 patients. Demographics and laboratory and plasma biomarkers of inflammation or endothelial dysfunction were used in the prediction models. Severe AKI was defined as Kidney Disease Improving Global Outcomes (KDIGO) stage 2 or 3.ResultsSevere AKI developed in 62 (8%) patients in the derivation, 26 (8%) patients in the internal validation, and 15 (6%) patients in the external validation cohorts. In the derivation cohort, a three-variable model (age, cirrhosis, and soluble TNF receptor-1 concentrations [ACT]) had a c-statistic of 0.95 (95% confidence interval [95% CI], 0.91 to 0.97). The ACT model performed well in the internal (c-statistic, 0.90; 95% CI, 0.82 to 0.96) and external (c-statistic, 0.93; 95% CI, 0.89 to 0.97) validation cohorts. The ACT model had moderate positive predictive values (0.50–0.95) and high negative predictive values (0.94–0.95) for severe AKI in all three cohorts.ConclusionsACT is a simple, robust model that could be applied to improve risk prognostication and better target clinical trial enrollment in critically ill patients with AKI.

Download Full-text

Empirical evidence of the impact of study characteristics on the performance of prediction models: a meta-epidemiological study

BMJ Open ◽

10.1136/bmjopen-2018-026160 ◽

2019 ◽

Vol 9 (4) ◽

pp. e026160 ◽

Cited By ~ 4

Author(s):

Johanna A A G Damen ◽

Thomas P A Debray ◽

Romin Pajouheshnia ◽

Johannes B Reitsma ◽

Rob J P M Scholten ◽

...

Keyword(s):

Performance Measures ◽

Systematic Reviews ◽

Validation Studies ◽

Prognostic Model ◽

External Validation ◽

Model Performance ◽

Population Characteristics ◽

Prognostic Models ◽

C Statistic ◽

Study Characteristics

ObjectivesTo empirically assess the relation between study characteristics and prognostic model performance in external validation studies of multivariable prognostic models.DesignMeta-epidemiological study.Data sources and study selectionOn 16 October 2018, we searched electronic databases for systematic reviews of prognostic models. Reviews from non-overlapping clinical fields were selected if they reported common performance measures (either the concordance (c)-statistic or the ratio of observed over expected number of events (OE ratio)) from 10 or more validations of the same prognostic model.Data extraction and analysesStudy design features, population characteristics, methods of predictor and outcome assessment, and the aforementioned performance measures were extracted from the included external validation studies. Random effects meta-regression was used to quantify the association between the study characteristics and model performance.ResultsWe included 10 systematic reviews, describing a total of 224 external validations, of which 221 reported c-statistics and 124 OE ratios. Associations between study characteristics and model performance were heterogeneous across systematic reviews. C-statistics were most associated with variation in population characteristics, outcome definitions and measurement and predictor substitution. For example, validations with eligibility criteria comparable to the development study were associated with higher c-statistics compared with narrower criteria (difference in logit c-statistic 0.21(95% CI 0.07 to 0.35), similar to an increase from 0.70 to 0.74). Using a case-control design was associated with higher OE ratios, compared with using data from a cohort (difference in log OE ratio 0.97(95% CI 0.38 to 1.55), similar to an increase in OE ratio from 1.00 to 2.63).ConclusionsVariation in performance of prognostic models across studies is mainly associated with variation in case-mix, study designs, outcome definitions and measurement methods and predictor substitution. Researchers developing and validating prognostic models should realise the potential influence of these study characteristics on the predictive performance of prognostic models.

Download Full-text