scholarly journals Facilitating safe discharge through predicting disease progression in moderate COVID-19: a prospective cohort study to develop and validate a clinical prediction model in resource-limited settings

2021 ◽  
Author(s):  
Arjun Chandna ◽  
Raman Mahajan ◽  
Priyanka Gautam ◽  
Lazaro Mwandigha ◽  
Karthik Gunasekaran ◽  
...  
Keyword(s):  
Prediction Model ◽  
Prediction Models ◽  
Validation Cohort ◽  
External Validation ◽  
Clinical Parameters ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Resource Limited Settings ◽  
Biochemical Biomarkers ◽  
Resource Limited

ABSTRACTBackgroundIn locations where few people have received COVID-19 vaccines, health systems remain vulnerable to surges in SARS-CoV-2 infections. Tools to identify patients suitable for community-based management are urgently needed.MethodsWe prospectively recruited adults presenting to two hospitals in India with moderate symptoms of laboratory-confirmed COVID-19 in order to develop and validate a clinical prediction model to rule-out progression to supplemental oxygen requirement. The primary outcome was defined as any of the following: SpO2 < 94%; respiratory rate > 30 bpm; SpO2/FiO2 < 400; or death. We specified a priori that each model would contain three clinical parameters (age, sex and SpO2) and one of seven shortlisted biochemical biomarkers measurable using near-patient tests (CRP, D-dimer, IL-6, NLR, PCT, sTREM-1 or suPAR), to ensure the models would be suitable for resource-limited settings. We evaluated discrimination, calibration and clinical utility of the models in a temporal external validation cohort.Findings426 participants were recruited, of whom 89 (21·0%) met the primary outcome. 257 participants comprised the development cohort and 166 comprised the validation cohort. The three models containing NLR, suPAR or IL-6 demonstrated promising discrimination (c-statistics: 0·72 to 0·74) and calibration (calibration slopes: 1·01 to 1·05) in the validation cohort, and provided greater utility than a model containing the clinical parameters alone.InterpretationWe present three clinical prediction models that could help clinicians identify patients with moderate COVID-19 suitable for community-based management. The models are readily implementable and of particular relevance for locations with limited resources.FundingMédecins Sans Frontières, India.RESEARCH IN CONTEXTEvidence before this studyA living systematic review by Wynants et al. identified 137 COVID-19 prediction models, 47 of which were derived to predict whether patients with COVID-19 will have an adverse outcome. Most lacked external validation, relied on retrospective data, did not focus on patients with moderate disease, were at high risk of bias, and were not practical for use in resource-limited settings. To identify promising biochemical biomarkers which may have been evaluated independently of a prediction model and therefore not captured by this review, we searched PubMed on 1 June 2020 using synonyms of “SARS-CoV-2” AND [“biomarker” OR “prognosis”]. We identified 1,214 studies evaluating biochemical biomarkers of potential value in the prognostication of COVID-19 illness. In consultation with FIND (Geneva, Switzerland) we shortlisted seven candidates for evaluation in this study, all of which are measurable using near-patient tests which are either currently available or in late-stage development.Added value of this studyWe followed the TRIPOD guidelines to develop and validate three promising clinical prediction models to help clinicians identify which patients presenting with moderate COVID-19 can be safely managed in the community. Each model contains three easily ascertained clinical parameters (age, sex, and SpO2) and one biochemical biomarker (NLR, suPAR or IL-6), and would be practical for implementation in high-patient-throughput low resource settings. The models showed promising discrimination and calibration in the validation cohort. The inclusion of a biomarker test improved prognostication compared to a model containing the clinical parameters alone, and extended the range of contexts in which such a tool might provide utility to include situations when bed pressures are less critical, for example at earlier points in a COVID-19 surge.Implications of all the available evidencePrognostic models should be developed for clearly-defined clinical use-cases. We report the development and temporal validation of three clinical prediction models to rule-out progression to supplemental oxygen requirement amongst patients presenting with moderate COVID-19. The models are readily implementable and should prove useful in triage and resource allocation. We provide our full models to enable independent validation.

Statistical Development and Validation of Clinical Prediction Models

2021 ◽  
Author(s):  
Steven J. Staffa ◽  
David Zurakowski
Keyword(s):  
Prediction Model ◽  
Prediction Models ◽  
External Validation ◽  
Model Development ◽  
Predictive Ability ◽  
Binary Outcome ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Surgery Research ◽  
Development And Validation

Summary Clinical prediction models in anesthesia and surgery research have many clinical applications including preoperative risk stratification with implications for clinical utility in decision-making, resource utilization, and costs. It is imperative that predictive algorithms and multivariable models are validated in a suitable and comprehensive way in order to establish the robustness of the model in terms of accuracy, predictive ability, reliability, and generalizability. The purpose of this article is to educate anesthesia researchers at an introductory level on important statistical concepts involved with development and validation of multivariable prediction models for a binary outcome. Methods covered include assessments of discrimination and calibration through internal and external validation. An anesthesia research publication is examined to illustrate the process and presentation of multivariable prediction model development and validation for a binary outcome. Properly assessing the statistical and clinical validity of a multivariable prediction model is essential for reassuring the generalizability and reproducibility of the published tool.

scholarly journals Prognostic Prediction Models for Patients with Low Back Pain: Systematic Review Protocol

2020 ◽  
Author(s):  
Fernanda Gonçalves Silva ◽  
Leonardo Oliveira Pena Costa ◽  
Mark J Hancock ◽  
Gabriele Alves Palomo ◽  
Luciola da Cunha Menezes Costa ◽  
...  
Keyword(s):  
Systematic Review ◽  
Low Back Pain ◽  
Back Pain ◽  
Prediction Model ◽  
Prediction Models ◽  
External Validation ◽  
Risk Of Bias ◽  
Low Back ◽  
Clinical Prediction ◽  
Clinical Prediction Models

Abstract Background: The prognosis of acute low back pain is generally favourable in terms of pain and disability; however, outcomes vary substantially between individual patients. Clinical prediction models help in estimating the likelihood of an outcome at a certain time point. There are existing clinical prediction models focused on prognosis for patients with low back pain. To date, there is only one previous systematic review summarising the discrimination of validated clinical prediction models to identify the prognosis in patients with low back pain of less than 3 months duration. The aim of this systematic review is to identify existing developed and/or validated clinical prediction models on prognosis of patients with low back pain of less than 3 months duration, and to summarise their performance in terms of discrimination and calibration. Methods: MEDLINE, Embase and CINAHL databases will be searched, from the inception of these databases until January 2020. Eligibility criteria will be: (1) prognostic model development studies with or without external validation, or prognostic external validation studies with or without model updating; (2) with adults aged 18 or over, with ‘recent onset’ low back pain (i.e. less than 3 months duration), with or without leg pain; (3) outcomes of pain, disability, sick leave or days absent from work or return to work status, and self-reported recovery; and (4) study with a follow-up of at least 12 weeks duration. The risk of bias of the included studies will be assessed by the Prediction model Risk Of Bias ASsessment Tool, and the overall quality of evidence will be rated using the Hierarchy of Evidence for Clinical Prediction Rules. Discussion: This systematic review will identify, appraise, and summarize evidence on the performance of existing prediction models for prognosis of low back pain, and may help clinicians to choose the best option of prediction model to better inform patients about their likely prognosis. Systematic review registration: PROSPERO reference number CRD42020160988

scholarly journals A systematic review of the quality of clinical prediction models in in vitro fertilisation

2020 ◽  
Vol 35 (1) ◽  
pp. 100-116 ◽  
Author(s):  
M B Ratna ◽  
S Bhattacharya ◽  
B Abdulrahim ◽  
D J McLernon
Keyword(s):  
Systematic Review ◽  
Live Birth ◽  
Prediction Models ◽  
Conflicts Of Interest ◽  
External Validation ◽  
Post Treatment ◽  
Treatment Model ◽  
Clinical Prediction ◽  
Clinical Prediction Models

Abstract STUDY QUESTION What are the best-quality clinical prediction models in IVF (including ICSI) treatment to inform clinicians and their patients of their chance of success? SUMMARY ANSWER The review recommends the McLernon post-treatment model for predicting the cumulative chance of live birth over and up to six complete cycles of IVF. WHAT IS KNOWN ALREADY Prediction models in IVF have not found widespread use in routine clinical practice. This could be due to their limited predictive accuracy and clinical utility. A previous systematic review of IVF prediction models, published a decade ago and which has never been updated, did not assess the methodological quality of existing models nor provided recommendations for the best-quality models for use in clinical practice. STUDY DESIGN, SIZE, DURATION The electronic databases OVID MEDLINE, OVID EMBASE and Cochrane library were searched systematically for primary articles published from 1978 to January 2019 using search terms on the development and/or validation (internal and external) of models in predicting pregnancy or live birth. No language or any other restrictions were applied. PARTICIPANTS/MATERIALS, SETTING, METHODS The PRISMA flowchart was used for the inclusion of studies after screening. All studies reporting on the development and/or validation of IVF prediction models were included. Articles reporting on women who had any treatment elements involving donor eggs or sperm and surrogacy were excluded. The CHARMS checklist was used to extract and critically appraise the methodological quality of the included articles. We evaluated models’ performance by assessing their c-statistics and plots of calibration in studies and assessed correct reporting by calculating the percentage of the TRIPOD 22 checklist items met in each study. MAIN RESULTS AND THE ROLE OF CHANCE We identified 33 publications reporting on 35 prediction models. Seventeen articles had been published since the last systematic review. The quality of models has improved over time with regard to clinical relevance, methodological rigour and utility. The percentage of TRIPOD score for all included studies ranged from 29 to 95%, and the c-statistics of all externally validated studies ranged between 0.55 and 0.77. Most of the models predicted the chance of pregnancy/live birth for a single fresh cycle. Six models aimed to predict the chance of pregnancy/live birth per individual treatment cycle, and three predicted more clinically relevant outcomes such as cumulative pregnancy/live birth. The McLernon (pre- and post-treatment) models predict the cumulative chance of live birth over multiple complete cycles of IVF per woman where a complete cycle includes all fresh and frozen embryo transfers from the same episode of ovarian stimulation. McLernon models were developed using national UK data and had the highest TRIPOD score, and the post-treatment model performed best on external validation. LIMITATIONS, REASONS FOR CAUTION To assess the reporting quality of all included studies, we used the TRIPOD checklist, but many of the earlier IVF prediction models were developed and validated before the formal TRIPOD reporting was published in 2015. It should also be noted that two of the authors of this systematic review are authors of the McLernon model article. However, we feel we have conducted our review and made our recommendations using a fair and transparent systematic approach. WIDER IMPLICATIONS OF THE FINDINGS This study provides a comprehensive picture of the evolving quality of IVF prediction models. Clinicians should use the most appropriate model to suit their patients’ needs. We recommend the McLernon post-treatment model as a counselling tool to inform couples of their predicted chance of success over and up to six complete cycles. However, it requires further external validation to assess applicability in countries with different IVF practices and policies. STUDY FUNDING/COMPETING INTEREST(S) The study was funded by the Elphinstone Scholarship Scheme and the Assisted Reproduction Unit, University of Aberdeen. Both D.J.M. and S.B. are authors of the McLernon model article and S.B. is Editor in Chief of Human Reproduction Open. They have completed and submitted the ICMJE forms for Disclosure of potential Conflicts of Interest. The other co-authors have no conflicts of interest to declare. REGISTRATION NUMBER N/A

scholarly journals Predictors of disease severity in children presenting from the community with febrile illnesses: a systematic review of prognostic studies

2021 ◽  
Vol 6 (1) ◽  
pp. e003451
Author(s):  
Arjun Chandna ◽  
Rainer Tan ◽  
Michael Carter ◽  
Ann Van Den Bruel ◽  
Jan Verbakel ◽  
...  
Keyword(s):  
At Risk ◽  
Prognostic Factors ◽  
Prediction Model ◽  
Disease Severity ◽  
Prediction Models ◽  
Severe Disease ◽  
Risk Of Bias ◽  
Clinical Prediction ◽  
Children At Risk ◽  
Clinical Prediction Models

IntroductionEarly identification of children at risk of severe febrile illness can optimise referral, admission and treatment decisions, particularly in resource-limited settings. We aimed to identify prognostic clinical and laboratory factors that predict progression to severe disease in febrile children presenting from the community.MethodsWe systematically reviewed publications retrieved from MEDLINE, Web of Science and Embase between 31 May 1999 and 30 April 2020, supplemented by hand search of reference lists and consultation with an expert Technical Advisory Panel. Studies evaluating prognostic factors or clinical prediction models in children presenting from the community with febrile illnesses were eligible. The primary outcome was any objective measure of disease severity ascertained within 30 days of enrolment. We calculated unadjusted likelihood ratios (LRs) for comparison of prognostic factors, and compared clinical prediction models using the area under the receiver operating characteristic curves (AUROCs). Risk of bias and applicability of studies were assessed using the Prediction Model Risk of Bias Assessment Tool and the Quality In Prognosis Studies tool.ResultsOf 5949 articles identified, 18 studies evaluating 200 prognostic factors and 25 clinical prediction models in 24 530 children were included. Heterogeneity between studies precluded formal meta-analysis. Malnutrition (positive LR range 1.56–11.13), hypoxia (2.10–8.11), altered consciousness (1.24–14.02), and markers of acidosis (1.36–7.71) and poor peripheral perfusion (1.78–17.38) were the most common predictors of severe disease. Clinical prediction model performance varied widely (AUROC range 0.49–0.97). Concerns regarding applicability were identified and most studies were at high risk of bias.ConclusionsFew studies address this important public health question. We identified prognostic factors from a wide range of geographic contexts that can help clinicians assess febrile children at risk of progressing to severe disease. Multicentre studies that include outpatients are required to explore generalisability and develop data-driven tools to support patient prioritisation and triage at the community level.PROSPERO registration numberCRD42019140542.

scholarly journals Machine learning-based clinical outcome prediction in surgery for acromegaly

Endocrine ◽  
2021 ◽  
Author(s):  
Olivier Zanier ◽  
Matteo Zoli ◽  
Victor E. Staartjes ◽  
Federica Guaraldi ◽  
Sofia Asioli ◽  
...  
Keyword(s):  
Machine Learning ◽  
Transsphenoidal Surgery ◽  
Prediction Models ◽  
External Validation ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Total Resection ◽  
Prior Surgery ◽  
Using Data ◽  
Csf Leaks

Abstract Purpose Biochemical remission (BR), gross total resection (GTR), and intraoperative cerebrospinal fluid (CSF) leaks are important metrics in transsphenoidal surgery for acromegaly, and prediction of their likelihood using machine learning would be clinically advantageous. We aim to develop and externally validate clinical prediction models for outcomes after transsphenoidal surgery for acromegaly. Methods Using data from two registries, we develop and externally validate machine learning models for GTR, BR, and CSF leaks after endoscopic transsphenoidal surgery in acromegalic patients. For the model development a registry from Bologna, Italy was used. External validation was then performed using data from Zurich, Switzerland. Gender, age, prior surgery, as well as Hardy and Knosp classification were used as input features. Discrimination and calibration metrics were assessed. Results The derivation cohort consisted of 307 patients (43.3% male; mean [SD] age, 47.2 [12.7] years). GTR was achieved in 226 (73.6%) and BR in 245 (79.8%) patients. In the external validation cohort with 46 patients, 31 (75.6%) achieved GTR and 31 (77.5%) achieved BR. Area under the curve (AUC) at external validation was 0.75 (95% confidence interval: 0.59–0.88) for GTR, 0.63 (0.40–0.82) for BR, as well as 0.77 (0.62–0.91) for intraoperative CSF leaks. While prior surgery was the most important variable for prediction of GTR, age, and Hardy grading contributed most to the predictions of BR and CSF leaks, respectively. Conclusions Gross total resection, biochemical remission, and CSF leaks remain hard to predict, but machine learning offers potential in helping to tailor surgical therapy. We demonstrate the feasibility of developing and externally validating clinical prediction models for these outcomes after surgery for acromegaly and lay the groundwork for development of a multicenter model with more robust generalization.

Abstract 17185: Modest Performance of Heart Failure Clinical Prediction Models: A Systematic External Validation Study

Circulation ◽  
2018 ◽  
Vol 138 (Suppl_1) ◽  
Author(s):  
Jenica N Upshaw ◽  
Jason Nelson ◽  
Benjamin Wessler ◽  
Benjamin Koethe ◽  
Christine Lundquist ◽  
...  
Keyword(s):  
Heart Failure ◽  
Prediction Models ◽  
External Validation ◽  
Significant Heterogeneity ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Derivation Cohort ◽  
C Statistic ◽  
Improved Model ◽  
Relative Change

Introduction: Most heart failure (HF) clinical prediction models (CPMs] have not been independently externally validated. We sought to test the performance of HF models in a diverse population using a systematic approach. Methods: A systematic review identified CPMs predicting outcomes for patients with HF. Individual patient data from 5 large publicaly available clinical trials enrolling patients with chronic HF were matched to published CPMs based on similarity in populations and available outcome and predictor variables in the clinical trial databases. CPM performance was evaluated for discrimination (c-statistic, % relative change in c-statistic) and calibration (Harrell’s E and E 90 , the mean and the 90% quantile of the error distribution from the smoothed loess observed value) for the original and recalibrated models. Results: Out of 135 HF CPMs reviewed, we identified 45 CPM-trial pairs including 13 unique CPMs. The outcome was mortality for all of the models with a trial match. During external validations, median c-statistic was 0.595 (IQR 0.563 to 0.630) with a median relative decrease in the c-statistic of -57 % (IQR, -49% to -71%) compared to the c-statistic reported in the derivation cohort. Overall, the median Harrell’s E was 0.09 (IQR, 0.04 to 0.135) and E 90 was 0.11 (IQR, 0.07 to 0.21). Recalibration of the intercept and slope led to substantially improved calibration with median change in Harrell’s E of -35% [IQR 0 to -75%] for the intercept and -56% [IQR -17% to -75%] for the intercept and slope. Refitting model covariates improved the median c-statistic by 38% to 0.629 [IQR 0.613 to 0.649]. Conclusion: For HF CPMs, independent external validations demonstrate that CPMs perform significantly worse than originally presented; however with significant heterogeneity. Recalibration of the intercept and slope improved model calibration. These results underscore the need to carefully consider the derivation cohort characteristics when using published CPMs.

scholarly journals A review of statistical updating methods for clinical prediction models

2016 ◽  
Vol 27 (1) ◽  
pp. 185-197 ◽  
Author(s):  
Ting-Li Su ◽  
Thomas Jaki ◽  
Graeme L Hickey ◽  
Iain Buchan ◽  
Matthew Sperrin
Keyword(s):  
Prediction Model ◽  
Prediction Models ◽  
Healthcare Outcomes ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Clinical Prediction Model ◽  
The United Kingdom ◽  
Single Strategy ◽  
Using Data ◽  
Dynamic Updating

A clinical prediction model is a tool for predicting healthcare outcomes, usually within a specific population and context. A common approach is to develop a new clinical prediction model for each population and context; however, this wastes potentially useful historical information. A better approach is to update or incorporate the existing clinical prediction models already developed for use in similar contexts or populations. In addition, clinical prediction models commonly become miscalibrated over time, and need replacing or updating. In this article, we review a range of approaches for re-using and updating clinical prediction models; these fall in into three main categories: simple coefficient updating, combining multiple previous clinical prediction models in a meta-model and dynamic updating of models. We evaluated the performance (discrimination and calibration) of the different strategies using data on mortality following cardiac surgery in the United Kingdom: We found that no single strategy performed sufficiently well to be used to the exclusion of the others. In conclusion, useful tools exist for updating existing clinical prediction models to a new population or context, and these should be implemented rather than developing a new clinical prediction model from scratch, using a breadth of complementary statistical methods.

scholarly journals External validation of six existing multivariable clinical prediction models for short-term mortality in patients undergoing lung resection

2020 ◽  
Author(s):  
Marcus Taylor ◽  
Bartłomiej Szafron ◽  
Glen P Martin ◽  
Udo Abah ◽  
Matthew Smith ◽  
...  
Keyword(s):  
Receiver Operating Characteristic ◽  
Operating Characteristic ◽  
Prediction Models ◽  
Characteristic Curve ◽  
Lung Resection ◽  
External Validation ◽  
Clinical Prediction ◽  
National Guidelines ◽  
Clinical Prediction Models ◽  
Receiver Operating

Abstract OBJECTIVES National guidelines advocate the use of clinical prediction models to estimate perioperative mortality for patients undergoing lung resection. Several models have been developed that may potentially be useful but contemporary external validation studies are lacking. The aim of this study was to validate existing models in a multicentre patient cohort. METHODS The Thoracoscore, Modified Thoracoscore, Eurolung, Modified Eurolung, European Society Objective Score and Brunelli models were validated using a database of 6600 patients who underwent lung resection between 2012 and 2018. Models were validated for in-hospital or 30-day mortality (depending on intended outcome of each model) and also for 90-day mortality. Model calibration (calibration intercept, calibration slope, observed to expected ratio and calibration plots) and discrimination (area under receiver operating characteristic curve) were assessed as measures of model performance. RESULTS Mean age was 66.8 years (±10.9 years) and 49.7% (n = 3281) of patients were male. In-hospital, 30-day, perioperative (in-hospital or 30-day) and 90-day mortality were 1.5% (n = 99), 1.4% (n = 93), 1.8% (n = 121) and 3.1% (n = 204), respectively. Model area under the receiver operating characteristic curves ranged from 0.67 to 0.73. Calibration was inadequate in five models and mortality was significantly overestimated in five models. No model was able to adequately predict 90-day mortality. CONCLUSIONS Five of the validated models were poorly calibrated and had inadequate discriminatory ability. The modified Eurolung model demonstrated adequate statistical performance but lacked clinical validity. Development of accurate models that can be used to estimate the contemporary risk of lung resection is required.

Sign in / Sign up

Export Citation Format

Share Document