A systematic review and external validation of stroke prediction models demonstrates poor performance in dialysis patients

Objective. To assess the performance of prognostic models in predicting mortality or clinical deterioration among patients with COVID-19, both hospitalized and non-hospitalized Methods. We conducted a systematic review of the literature until March 8, 2021. We included models for the prediction of mortality or clinical deterioration in COVID-19 with external validation. We used the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and the GRADEpro Guideline Development Tool (GDT) to assess the evidence obtained. Results. We reviewed 33 cohort studies. Two studies had a low risk of bias, four unclear risks, and 27 with a high risk of bias due to participant selection and analysis. For the outcome of mortality, the QCOVID model had excellent prediction with high certainty of evidence but was specific for use in England. The COVID Outcome Prediction in the Emergency Department (COPE) model, the 4C Mortality Score, the Age, BUN, number of comorbidities, CRP, SpO2/FiO2 ratio, platelet count, heart rate (ABC2-SPH) risk score, the Confusion Urea Respiration Blood Pressure (CURB-65) severity score, the Rapid Emergency Medicine Score (REMS), and the Risk Stratification in the Emergency Department in Acutely Ill Older Patients (RISE UP) score had fair to good prediction of death among inpatients, while the quick Sepsis-related Organ Failure Assessment (qSOFA) score had poor to fair prediction. The certainty of evidence for these models was very low to low. For the outcome of clinical deterioration, the 4C Deterioration Score had fair prediction, the National Early Warning Score 2 (NEWS2) score poor to good, and the Modified Early Warning Score (MEWS) had poor prediction. The certainty of evidence for these three models was also very low to low. None of these models had been validated in the Philippine setting. Conclusion. The QCOVID, COPE, ABC2-SPH, 4C, CURB-65, REMS, RISE-UP models for prediction of mortality and the 4C Deterioration and NEWS2 models for prediction of clinical deterioration are potentially useful but need to be validated among patients with COVID-19 of varying severity in the Philippine setting.

Download Full-text

P–419 On prognosis after unexplained recurrent pregnancy losses (RPL); a systematic review and external validation of clinical prediction models

Human Reproduction ◽

10.1093/humrep/deab130.418 ◽

2021 ◽

Vol 36 (Supplement_1) ◽

Author(s):

A Youssef

Keyword(s):

Systematic Review ◽

Supportive Care ◽

Prediction Model ◽

Pregnancy Outcome ◽

Prediction Models ◽

External Validation ◽

Model Development ◽

Predictive Performance ◽

Pregnancy Rates ◽

Cochrane Library

Abstract Study question Which models that predict pregnancy outcome in couples with unexplained RPL exist and what is the performance of the most used model? Summary answer We identified seven prediction models; none followed the recommended prediction model development steps. Moreover, the most used model showed poor predictive performance. What is known already RPL remains unexplained in 50–75% of couples For these couples, there is no effective treatment option and clinical management rests on supportive care. Essential part of supportive care consists of counselling on the prognosis of subsequent pregnancies. Indeed, multiple prediction models exist, however the quality and validity of these models varies. In addition, the prediction model developed by Brigham et al is the most widely used model, but has never been externally validated. Study design, size, duration We performed a systematic review to identify prediction models for pregnancy outcome after unexplained RPL. In addition we performed an external validation of the Brigham model in a retrospective cohort, consisting of 668 couples with unexplained RPL that visited our RPL clinic between 2004 and 2019. Participants/materials, setting, methods A systematic search was performed in December 2020 in Pubmed, Embase, Web of Science and Cochrane library to identify relevant studies. Eligible studies were selected and assessed according to the TRIPOD) guidelines, covering topics on model performance and validation statement. The performance of predicting live birth in the Brigham model was evaluated through calibration and discrimination, in which the observed pregnancy rates were compared to the predicted pregnancy rates. Main results and the role of chance Seven models were compared and assessed according to the TRIPOD statement. This resulted in two studies of low, three of moderate and two of above average reporting quality. These studies did not follow the recommended steps for model development and did not calculate a sample size. Furthermore, the predictive performance of neither of these models was internally- or externally validated. We performed an external validation of Brigham model. Calibration showed overestimation of the model and too extreme predictions, with a negative calibration intercept of –0.52 (CI 95% –0.68 – –0.36), with a calibration slope of 0.39 (CI 95% 0.07 – 0.71). The discriminative ability of the model was very low with a concordance statistic of 0.55 (CI 95% 0.50 – 0.59). Limitations, reasons for caution None of the studies are specifically named prediction models, therefore models may have been missed in the selection process. The external validation cohort used a retrospective design, in which only the first pregnancy after intake was registered. Follow-up time was not limited, which is important in counselling unexplained RPL couples. Wider implications of the findings: Currently, there are no suitable models that predict on pregnancy outcome after RPL. Moreover, we are in need of a model with several variables such that prognosis is individualized, and factors from both the female as the male to enable a couple specific prognosis. Trial registration number Not applicable

Download Full-text

A systematic review of the quality of clinical prediction models in in vitro fertilisation

Human Reproduction ◽

10.1093/humrep/dez258 ◽

2020 ◽

Vol 35 (1) ◽

pp. 100-116 ◽

Cited By ~ 8

Author(s):

M B Ratna ◽

S Bhattacharya ◽

B Abdulrahim ◽

D J McLernon

Keyword(s):

Systematic Review ◽

Live Birth ◽

Prediction Models ◽

Conflicts Of Interest ◽

External Validation ◽

Post Treatment ◽

Treatment Model ◽

Clinical Prediction ◽

Clinical Prediction Models

Abstract STUDY QUESTION What are the best-quality clinical prediction models in IVF (including ICSI) treatment to inform clinicians and their patients of their chance of success? SUMMARY ANSWER The review recommends the McLernon post-treatment model for predicting the cumulative chance of live birth over and up to six complete cycles of IVF. WHAT IS KNOWN ALREADY Prediction models in IVF have not found widespread use in routine clinical practice. This could be due to their limited predictive accuracy and clinical utility. A previous systematic review of IVF prediction models, published a decade ago and which has never been updated, did not assess the methodological quality of existing models nor provided recommendations for the best-quality models for use in clinical practice. STUDY DESIGN, SIZE, DURATION The electronic databases OVID MEDLINE, OVID EMBASE and Cochrane library were searched systematically for primary articles published from 1978 to January 2019 using search terms on the development and/or validation (internal and external) of models in predicting pregnancy or live birth. No language or any other restrictions were applied. PARTICIPANTS/MATERIALS, SETTING, METHODS The PRISMA flowchart was used for the inclusion of studies after screening. All studies reporting on the development and/or validation of IVF prediction models were included. Articles reporting on women who had any treatment elements involving donor eggs or sperm and surrogacy were excluded. The CHARMS checklist was used to extract and critically appraise the methodological quality of the included articles. We evaluated models’ performance by assessing their c-statistics and plots of calibration in studies and assessed correct reporting by calculating the percentage of the TRIPOD 22 checklist items met in each study. MAIN RESULTS AND THE ROLE OF CHANCE We identified 33 publications reporting on 35 prediction models. Seventeen articles had been published since the last systematic review. The quality of models has improved over time with regard to clinical relevance, methodological rigour and utility. The percentage of TRIPOD score for all included studies ranged from 29 to 95%, and the c-statistics of all externally validated studies ranged between 0.55 and 0.77. Most of the models predicted the chance of pregnancy/live birth for a single fresh cycle. Six models aimed to predict the chance of pregnancy/live birth per individual treatment cycle, and three predicted more clinically relevant outcomes such as cumulative pregnancy/live birth. The McLernon (pre- and post-treatment) models predict the cumulative chance of live birth over multiple complete cycles of IVF per woman where a complete cycle includes all fresh and frozen embryo transfers from the same episode of ovarian stimulation. McLernon models were developed using national UK data and had the highest TRIPOD score, and the post-treatment model performed best on external validation. LIMITATIONS, REASONS FOR CAUTION To assess the reporting quality of all included studies, we used the TRIPOD checklist, but many of the earlier IVF prediction models were developed and validated before the formal TRIPOD reporting was published in 2015. It should also be noted that two of the authors of this systematic review are authors of the McLernon model article. However, we feel we have conducted our review and made our recommendations using a fair and transparent systematic approach. WIDER IMPLICATIONS OF THE FINDINGS This study provides a comprehensive picture of the evolving quality of IVF prediction models. Clinicians should use the most appropriate model to suit their patients’ needs. We recommend the McLernon post-treatment model as a counselling tool to inform couples of their predicted chance of success over and up to six complete cycles. However, it requires further external validation to assess applicability in countries with different IVF practices and policies. STUDY FUNDING/COMPETING INTEREST(S) The study was funded by the Elphinstone Scholarship Scheme and the Assisted Reproduction Unit, University of Aberdeen. Both D.J.M. and S.B. are authors of the McLernon model article and S.B. is Editor in Chief of Human Reproduction Open. They have completed and submitted the ICMJE forms for Disclosure of potential Conflicts of Interest. The other co-authors have no conflicts of interest to declare. REGISTRATION NUMBER N/A

Download Full-text

Prediction models for intracranial hemorrhage or major bleeding in patients on antiplatelet therapy: a systematic review and external validation study

Journal of Thrombosis and Haemostasis ◽

10.1111/jth.13196 ◽

2016 ◽

Vol 14 (1) ◽

pp. 167-174 ◽

Cited By ~ 6

Author(s):

N. A. Hilkens ◽

A. Algra ◽

J. P. Greving

Keyword(s):

Systematic Review ◽

Antiplatelet Therapy ◽

Intracranial Hemorrhage ◽

Major Bleeding ◽

Validation Study ◽

Prediction Models ◽

External Validation ◽

External Validation Study

Download Full-text

Comparison of prognostic models to predict the occurrence of colorectal cancer in asymptomatic individuals: a systematic literature review and external validation in the EPIC and UK Biobank prospective cohort studies

Gut ◽

10.1136/gutjnl-2017-315730 ◽

2018 ◽

Vol 68 (4) ◽

pp. 672-683 ◽

Cited By ~ 6

Author(s):

Todd Smith ◽

David C Muller ◽

Karel G M Moons ◽

Amanda J Cross ◽

Mattias Johansson ◽

...

Keyword(s):

Colorectal Cancer ◽

Systematic Review ◽

Prediction Models ◽

Large Population ◽

External Validation ◽

Population Based ◽

Prognostic Models ◽

Uk Biobank ◽

Asymptomatic Individuals ◽

The Uk

ObjectiveTo systematically identify and validate published colorectal cancer risk prediction models that do not require invasive testing in two large population-based prospective cohorts.DesignModels were identified through an update of a published systematic review and validated in the European Prospective Investigation into Cancer and Nutrition (EPIC) and the UK Biobank. The performance of the models to predict the occurrence of colorectal cancer within 5 or 10 years after study enrolment was assessed by discrimination (C-statistic) and calibration (plots of observed vs predicted probability).ResultsThe systematic review and its update identified 16 models from 8 publications (8 colorectal, 5 colon and 3 rectal). The number of participants included in each model validation ranged from 41 587 to 396 515, and the number of cases ranged from 115 to 1781. Eligible and ineligible participants across the models were largely comparable. Calibration of the models, where assessable, was very good and further improved by recalibration. The C-statistics of the models were largely similar between validation cohorts with the highest values achieved being 0.70 (95% CI 0.68 to 0.72) in the UK Biobank and 0.71 (95% CI 0.67 to 0.74) in EPIC.ConclusionSeveral of these non-invasive models exhibited good calibration and discrimination within both external validation populations and are therefore potentially suitable candidates for the facilitation of risk stratification in population-based colorectal screening programmes. Future work should both evaluate this potential, through modelling and impact studies, and ascertain if further enhancement in their performance can be obtained.

Download Full-text

P454 Prediction of endoscopic activity in patients with Crohn’s disease: systematic review and external validation of published prediction models

Journal of Crohn s and Colitis ◽

10.1093/ecco-jcc/jjy222.578 ◽

2019 ◽

Vol 13 (Supplement_1) ◽

pp. S338-S339

Author(s):

E C Brand ◽

S G Elias ◽

I M Minderhoud ◽

J J van der Veen ◽

F Baert ◽

...

Keyword(s):

Systematic Review ◽

Crohn’S Disease ◽

Crohn's Disease ◽

Prediction Models ◽

External Validation

Download Full-text

Mo1816 – Prediction of Endoscopic Activity in Patients with Crohn’s Disease – Systematic Review and External Validation of Published Prediction Models

Gastroenterology ◽

10.1016/s0016-5085(19)39083-3 ◽

2019 ◽

Vol 156 (6) ◽

pp. S-848

Author(s):

Eelco C. Brand ◽

Sjoerd G. Elias ◽

Itta M. Minderhoud ◽

Julius J. van der Veen ◽

Filip J. Baert ◽

...

Keyword(s):

Systematic Review ◽

Crohn’S Disease ◽

Crohn's Disease ◽

Prediction Models ◽

External Validation

Download Full-text

Systematic Review and External Validation of Prediction Models Based on Symptoms and Biomarkers for Identifying Endoscopic Activity in Crohn's Disease

Clinical Gastroenterology and Hepatology ◽

10.1016/j.cgh.2019.12.014 ◽

2020 ◽

Vol 18 (8) ◽

pp. 1704-1718 ◽

Cited By ~ 2

Author(s):

Eelco C. Brand ◽

Sjoerd G. Elias ◽

Itta M. Minderhoud ◽

Julius J. van der Veen ◽

Filip J. Baert ◽

...

Keyword(s):

Systematic Review ◽

Crohn’S Disease ◽

Crohn's Disease ◽

Prediction Models ◽

External Validation

Download Full-text

Prognostic prediction models for pregnancy complications in women with gestational diabetes: a protocol for systematic review, critical appraisal and meta-analysis

Systematic Reviews ◽

10.1186/s13643-019-1151-0 ◽

2019 ◽

Vol 8 (1) ◽

Cited By ~ 7

Author(s):

Shamil D. Cooray ◽

Jacqueline A. Boyle ◽

Georgia Soldatos ◽

Lihini A. Wijeyaratne ◽

Helena J. Teede

Keyword(s):

Systematic Review ◽

Decision Making ◽

Gestational Diabetes ◽

Pregnancy Complications ◽

Prediction Models ◽

Absolute Risk ◽

Meta Analysis ◽

External Validation ◽

The Absolute ◽

Prognostic Prediction

Abstract Background Gestational diabetes (GDM) is increasingly common and has significant implications during pregnancy and for the long-term health of the mother and offspring. However, it is a heterogeneous condition with inter-related factors including ethnicity, body mass index and gestational weight gain significantly modifying the absolute risk of complications at an individual level. Predicting the risk of pregnancy complications for an individual woman with GDM presents a useful adjunct to therapeutic decision-making and patient education. Diagnostic prediction models for GDM are prevalent. In contrast, prediction models for risk of complications in those with GDM are relatively novel. This study will systematically review published prognostic prediction models for pregnancy complications in women with GDM, describe their characteristics, compare performance and assess methodological quality and applicability. Methods Studies will be identified by searching MEDLINE and Embase electronic databases. Title and abstract screening, full-text review and data extraction will be completed independently by two reviewers. The included studies will be systematically assessed for risk of bias and applicability using appropriate tools designed for prediction modelling studies. Extracted data will be tabulated to facilitate qualitative comparison of published prediction models. Quantitative data on predictive performance of these models will be synthesised with meta-analyses if appropriate. Discussion This review will identify and summarise all published prognostic prediction models for pregnancy complications in women with GDM. We will compare model performance across different settings and populations with meta-analysis if appropriate. This work will guide subsequent phases in the prognosis research framework: further model development, external validation and model updating, and impact assessment. The ultimate model will estimate the absolute risk of pregnancy complications for women with GDM and will be implemented into routine care as an evidence-based GDM complication risk prediction model. It is anticipated to offer value to women and their clinicians with individualised risk assessment and may assist decision-making. Ultimately, this systematic review is an important step towards a personalised risk-stratified model-of-care for GDM to allow preventative and therapeutic interventions for the maximal benefit to women and their offspring, whilst sparing expense and harm for those at low risk. Systematic review registration PROSPERO registration number CRD42019115223

Download Full-text