Prediction Models for Type 2 Diabetes Risk in the General Population: A Systematic Review of Observational Studies

Objectives: This study aimed to provide an overview of prediction models of undiagnosed type 2 diabetes mellitus (U-T2DM) or the incident T2DM (I-T2DM) using the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) checklist and the prediction model risk of the bias assessment tool (PROBAST). Data Sources: Both PUBMED and EMBASE databases were searched to guarantee adequate and efficient coverage. Study Selection: Articles published between December 2011 and October 2019 were considered. Data Extraction: For each article, information on model development requirements, discrimination measures, calibration, overall performance, clinical usefulness, overfitting, and risk of bias (ROB) was reported. Results: The median (interquartile range; IQR) number of the 46 study populations for model development was 5711 (1971 - 27426) and 2457 (2060 - 6995) individuals for I-T2DM and U-T2DM, respectively. The most common reported predictors were age and body mass index, and only the Qrisk-2017 study included social factors (e.g., Townsend score). Univariable analysis was reported in 46% of the studies, and the variable selection procedure was not clear in 17.4% of them. Moreover, internal and external validation was reported in 43% the studies, while over 63% of them reported calibration. The median (IQR) of AUC for I-T2DM models was 0.78 (0.74 - 0.82); the corresponding value for studies derived before October 2011 was 0.80 (0.77 - 0.83). The highest discrimination index was reported for Qrisk-2017 with C-statistics of 0.89 for women and 0.87 for men. Low ROB for I-T2DM and U-T2DM was assessed at 18% and 41%, respectively. Conclusions: Among prediction models, an intermediate to poor quality were reassessed in several aspects of model development and validation, even though there was a comprehensive protocol. Generally, despite its new risk factors or new methodological aspects, the newly developed model did not increase our capability in screening/predicting T2DM, mainly in the analysis part. It was due to the lack of external validation of the prediction models.

Download Full-text

P–419 On prognosis after unexplained recurrent pregnancy losses (RPL); a systematic review and external validation of clinical prediction models

Human Reproduction ◽

10.1093/humrep/deab130.418 ◽

2021 ◽

Vol 36 (Supplement_1) ◽

Author(s):

A Youssef

Keyword(s):

Systematic Review ◽

Supportive Care ◽

Prediction Model ◽

Pregnancy Outcome ◽

Prediction Models ◽

External Validation ◽

Model Development ◽

Predictive Performance ◽

Pregnancy Rates ◽

Cochrane Library

Abstract Study question Which models that predict pregnancy outcome in couples with unexplained RPL exist and what is the performance of the most used model? Summary answer We identified seven prediction models; none followed the recommended prediction model development steps. Moreover, the most used model showed poor predictive performance. What is known already RPL remains unexplained in 50–75% of couples For these couples, there is no effective treatment option and clinical management rests on supportive care. Essential part of supportive care consists of counselling on the prognosis of subsequent pregnancies. Indeed, multiple prediction models exist, however the quality and validity of these models varies. In addition, the prediction model developed by Brigham et al is the most widely used model, but has never been externally validated. Study design, size, duration We performed a systematic review to identify prediction models for pregnancy outcome after unexplained RPL. In addition we performed an external validation of the Brigham model in a retrospective cohort, consisting of 668 couples with unexplained RPL that visited our RPL clinic between 2004 and 2019. Participants/materials, setting, methods A systematic search was performed in December 2020 in Pubmed, Embase, Web of Science and Cochrane library to identify relevant studies. Eligible studies were selected and assessed according to the TRIPOD) guidelines, covering topics on model performance and validation statement. The performance of predicting live birth in the Brigham model was evaluated through calibration and discrimination, in which the observed pregnancy rates were compared to the predicted pregnancy rates. Main results and the role of chance Seven models were compared and assessed according to the TRIPOD statement. This resulted in two studies of low, three of moderate and two of above average reporting quality. These studies did not follow the recommended steps for model development and did not calculate a sample size. Furthermore, the predictive performance of neither of these models was internally- or externally validated. We performed an external validation of Brigham model. Calibration showed overestimation of the model and too extreme predictions, with a negative calibration intercept of –0.52 (CI 95% –0.68 – –0.36), with a calibration slope of 0.39 (CI 95% 0.07 – 0.71). The discriminative ability of the model was very low with a concordance statistic of 0.55 (CI 95% 0.50 – 0.59). Limitations, reasons for caution None of the studies are specifically named prediction models, therefore models may have been missed in the selection process. The external validation cohort used a retrospective design, in which only the first pregnancy after intake was registered. Follow-up time was not limited, which is important in counselling unexplained RPL couples. Wider implications of the findings: Currently, there are no suitable models that predict on pregnancy outcome after RPL. Moreover, we are in need of a model with several variables such that prognosis is individualized, and factors from both the female as the male to enable a couple specific prognosis. Trial registration number Not applicable

Download Full-text

External validation of the KORA S4/F4 prediction models for the risk of developing type 2 diabetes in older adults: the PREVEND study

European Journal of Epidemiology ◽

10.1007/s10654-011-9648-4 ◽

2012 ◽

Vol 27 (1) ◽

pp. 47-52 ◽

Cited By ~ 12

Author(s):

Ali Abbasi ◽

Eva Corpeleijn ◽

Linda M. Peelen ◽

Ron T. Gansevoort ◽

Paul E. de Jong ◽

...

Keyword(s):

Type 2 Diabetes ◽

Older Adults ◽

Prediction Models ◽

External Validation

Download Full-text

Systematic review of prediction models for pulmonary tuberculosis treatment outcomes in adults

BMJ Open ◽

10.1136/bmjopen-2020-044687 ◽

2021 ◽

Vol 11 (3) ◽

pp. e044687

Author(s):

Lauren S. Peetluk ◽

Felipe M. Ridolfi ◽

Peter F. Rebeiro ◽

Dandan Liu ◽

Valeria C Rolla ◽

...

Keyword(s):

Systematic Review ◽

Treatment Outcomes ◽

Prediction Models ◽

Assessment Tool ◽

Data Extraction ◽

External Validation ◽

Included Study ◽

Risk Of Bias ◽

Pulmonary Tb ◽

Tb Treatment

ObjectiveTo systematically review and critically evaluate prediction models developed to predict tuberculosis (TB) treatment outcomes among adults with pulmonary TB.DesignSystematic review.Data sourcesPubMed, Embase, Web of Science and Google Scholar were searched for studies published from 1 January 1995 to 9 January 2020.Study selection and data extractionStudies that developed a model to predict pulmonary TB treatment outcomes were included. Study screening, data extraction and quality assessment were conducted independently by two reviewers. Study quality was evaluated using the Prediction model Risk Of Bias Assessment Tool. Data were synthesised with narrative review and in tables and figures.Results14 739 articles were identified, 536 underwent full-text review and 33 studies presenting 37 prediction models were included. Model outcomes included death (n=16, 43%), treatment failure (n=6, 16%), default (n=6, 16%) or a composite outcome (n=9, 25%). Most models (n=30, 81%) measured discrimination (median c-statistic=0.75; IQR: 0.68–0.84), and 17 (46%) reported calibration, often the Hosmer-Lemeshow test (n=13). Nineteen (51%) models were internally validated, and six (16%) were externally validated. Eighteen (54%) studies mentioned missing data, and of those, half (n=9) used complete case analysis. The most common predictors included age, sex, extrapulmonary TB, body mass index, chest X-ray results, previous TB and HIV. Risk of bias varied across studies, but all studies had high risk of bias in their analysis.ConclusionsTB outcome prediction models are heterogeneous with disparate outcome definitions, predictors and methodology. We do not recommend applying any in clinical settings without external validation, and encourage future researchers adhere to guidelines for developing and reporting of prediction models.Trial registrationThe study was registered on the international prospective register of systematic reviews PROSPERO (CRD42020155782)

Download Full-text

Prediction of Type 2 Diabetes Based on Machine Learning Algorithm

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18063317 ◽

2021 ◽

Vol 18 (6) ◽

pp. 3317

Author(s):

Henock M. Deberneh ◽

Intaek Kim

Keyword(s):

Machine Learning ◽

Type 2 Diabetes ◽

Prediction Model ◽

Prediction Models ◽

Machine Learning Algorithms ◽

Superior Performance ◽

Recursive Feature Elimination ◽

Medical Institute ◽

Support Vector

Prediction of type 2 diabetes (T2D) occurrence allows a person at risk to take actions that can prevent onset or delay the progression of the disease. In this study, we developed a machine learning (ML) model to predict T2D occurrence in the following year (Y + 1) using variables in the current year (Y). The dataset for this study was collected at a private medical institute as electronic health records from 2013 to 2018. To construct the prediction model, key features were first selected using ANOVA tests, chi-squared tests, and recursive feature elimination methods. The resultant features were fasting plasma glucose (FPG), HbA1c, triglycerides, BMI, gamma-GTP, age, uric acid, sex, smoking, drinking, physical activity, and family history. We then employed logistic regression, random forest, support vector machine, XGBoost, and ensemble machine learning algorithms based on these variables to predict the outcome as normal (non-diabetic), prediabetes, or diabetes. Based on the experimental results, the performance of the prediction model proved to be reasonably good at forecasting the occurrence of T2D in the Korean population. The model can provide clinicians and patients with valuable predictive information on the likelihood of developing T2D. The cross-validation (CV) results showed that the ensemble models had a superior performance to that of the single models. The CV performance of the prediction models was improved by incorporating more medical history from the dataset.

Download Full-text

Prognostic Models for Predicting Remission of Diabetes Following Bariatric Surgery: A Systematic Review and Meta-analysis

10.2337/figshare.15173232 ◽

2021 ◽

Author(s):

Pushpa Singh ◽

Nicola J Adderley ◽

Jonathan Hazlehurst ◽

Malcolm Price ◽

Abd A Tahrani ◽

...

Keyword(s):

Systematic Review ◽

Bariatric Surgery ◽

Prediction Models ◽

Data Extraction ◽

External Validation ◽

Model Development ◽

Diabetes Remission ◽

Development Studies ◽

Cochrane Central Register

Background Remission of type 2 diabetes following bariatric surgery is well established but identifying patients who will go into remission is challenging. Purpose To perform a systematic review of currently available diabetes remission prediction models, compare their performance, and evaluate their applicability in clinical settings. Data sources A comprehensive systematic literature search of MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE and Cochrane Central Register of Controlled Trials was undertaken. The search was restricted to studies published in the last 15 years and in the English language. Study selection and data extraction All studies developing or validating a prediction model for diabetes remission in adults after bariatric surgery were included. The search identified 4165 references of which 38 were included for data extraction. We identified 16 model development and 22 validation studies. Data synthesis Of the 16 model development studies, 11 developed scoring systems and 5 proposed logistic regression models. In model development studies, 10 models showed excellent discrimination with area under curve (AUC) ≥ 0.800. Two of these prediction models, ABCD and DiaRem, were widely externally validated in different populations, a variety of bariatric procedures, and for both short- and long-term diabetes remission. Newer prediction models showed excellent discrimination in test studies, but external validation was limited. Limitations and Conclusions Amongst the prediction models identified, the ABCD and DiaRem models were the most widely validated and showed acceptable to excellent discrimination. More studies validating newer models and focusing on long-term diabetes remission are needed.

Download Full-text

Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19

10.1101/2020.03.28.20045997 ◽

2020 ◽

Cited By ~ 18

Author(s):

Jianfeng Xie ◽

Daniel Hungerford ◽

Hui Chen ◽

Simon T Abrams ◽

Shusheng Li ◽

...

Keyword(s):

Lactate Dehydrogenase ◽

Prediction Model ◽

Lymphocyte Count ◽

Prediction Models ◽

External Validation ◽

Model Development ◽

Added Value ◽

Clinical Prediction ◽

Healthcare Resources ◽

D Dimer

SummaryBackgroundCOVID-19 pandemic has developed rapidly and the ability to stratify the most vulnerable patients is vital. However, routinely used severity scoring systems are often low on diagnosis, even in non-survivors. Therefore, clinical prediction models for mortality are urgently required.MethodsWe developed and internally validated a multivariable logistic regression model to predict inpatient mortality in COVID-19 positive patients using data collected retrospectively from Tongji Hospital, Wuhan (299 patients). External validation was conducted using a retrospective cohort from Jinyintan Hospital, Wuhan (145 patients). Nine variables commonly measured in these acute settings were considered for model development, including age, biomarkers and comorbidities. Backwards stepwise selection and bootstrap resampling were used for model development and internal validation. We assessed discrimination via the C statistic, and calibration using calibration-in-the-large, calibration slopes and plots.FindingsThe final model included age, lymphocyte count, lactate dehydrogenase and SpO2 as independent predictors of mortality. Discrimination of the model was excellent in both internal (c=0·89) and external (c=0·98) validation. Internal calibration was excellent (calibration slope=1). External validation showed some over-prediction of risk in low-risk individuals and under-prediction of risk in high-risk individuals prior to recalibration. Recalibration of the intercept and slope led to excellent performance of the model in independent data.InterpretationCOVID-19 is a new disease and behaves differently from common critical illnesses. This study provides a new prediction model to identify patients with lethal COVID-19. Its practical reliance on commonly available parameters should improve usage of limited healthcare resources and patient survival rate.FundingThis study was supported by following funding: Key Research and Development Plan of Jiangsu Province (BE2018743 and BE2019749), National Institute for Health Research (NIHR) (PDF-2018-11-ST2-006), British Heart Foundation (BHF) (PG/16/65/32313) and Liverpool University Hospitals NHS Foundation Trust in UK.Research in contextEvidence before this studySince the outbreak of COVID-19, there has been a pressing need for development of a prognostic tool that is easy for clinicians to use. Recently, a Lancet publication showed that in a cohort of 191 patients with COVID-19, age, SOFA score and D-dimer measurements were associated with mortality. No other publication involving prognostic factors or models has been identified to date.Added value of this studyIn our cohorts of 444 patients from two hospitals, SOFA scores were low in the majority of patients on admission. The relevance of D-dimer could not be verified, as it is not included in routine laboratory tests. In this study, we have established a multivariable clinical prediction model using a development cohort of 299 patients from one hospital. After backwards selection, four variables, including age, lymphocyte count, lactate dehydrogenase and SpO2 remained in the model to predict mortality. This has been validated internally and externally with a cohort of 145 patients from a different hospital. Discrimination of the model was excellent in both internal (c=0·89) and external (c=0·98) validation. Calibration plots showed excellent agreement between predicted and observed probabilities of mortality after recalibration of the model to account for underlying differences in the risk profile of the datasets. This demonstrated that the model is able to make reliable predictions in patients from different hospitals. In addition, these variables agree with pathological mechanisms and the model is easy to use in all types of clinical settings.Implication of all the available evidenceAfter further external validation in different countries the model will enable better risk stratification and more targeted management of patients with COVID-19. With the nomogram, this model that is based on readily available parameters can help clinicians to stratify COVID-19 patients on diagnosis to use limited healthcare resources effectively and improve patient outcome.

Download Full-text

Evaluating Methodological Quality of Prognostic Prediction Models on Patient Reported Outcome Measurements after Total Hip Replacement and Total Knee Replacement Surgery: a systematic review protocol

10.21203/rs.3.rs-903247/v1 ◽

2021 ◽

Author(s):

Wei-Ju Chang ◽

Justine Naylor ◽

Pragadesh Natarajan ◽

Spiro Menounos ◽

Masiath Monuja ◽

...

Keyword(s):

Systematic Review ◽

Knee Osteoarthritis ◽

Prediction Model ◽

Methodological Quality ◽

Prediction Models ◽

External Validation ◽

Model Development ◽

Patient Reported ◽

Modelling Studies

Abstract Background Prediction models for poor patient-reported surgical outcomes after total hip replacement (THR) and total knee replacement (TKR) may provide a method for improving appropriate surgical care for hip and knee osteoarthritis. There are concerns about methodological issues and the risk of bias of studies producing prediction models. A critical evaluation of the methodological quality of prediction modelling studies in THR and TKR is needed to ensure their clinical usefulness. This systematic review aims to: 1) evaluate and report the quality of risk stratification and prediction modelling studies that predict patient-reported outcomes after THR and TKR; 2) identify areas of methodological deficit and provide recommendations for future research; and 3) synthesise the evidence on prediction models associated with post-operative patient-reported outcomes after THR and TKR surgeries. Methods MEDLINE, EMBASE and CINAHL electronic databases will be searched to identify relevant studies. Title and abstract and full-text screening will be performed by two independent reviewers. We will include: 1) prediction model development studies without external validation; 2) prediction model development studies with external validation of independent data; 3) external model validation studies; and 4) studies updating a previously developed prediction model. Data extraction spreadsheets will be developed based on the CHARMS checklist and TRIPOD statement and piloted on two relevant studies. Study quality and risk of bias will be assessed using the PROBAST tool. Prediction models will be summarised qualitatively. Meta-analyses on the predictive performance of included models will be conducted if appropriate. Discussion This systematic review will evaluate the methodological quality and usefulness of prediction models for poor outcomes after THR or TKR. This information is essential to provide evidence-based healthcare for end-stage hip and knee osteoarthritis. Findings of this review will contribute to the identification of key areas for improvement in conducting prognostic research in this field and facilitate the progress in evidence-based tailored treatments for hip and knee osteoarthritis. Systematic review registration: Submitted to PROSPERO on 30 August 2021.

Download Full-text

Prediction performance of a cardiovascular risk assessment tool using Stanford EHR data repository

10.1101/648956 ◽

2019 ◽

Author(s):

Mehrdad Rezaee ◽

Arsia Takeh ◽

Igor Putrenko ◽

Andrea Ganna ◽

Erik Ingelsson

Keyword(s):

Type 2 Diabetes ◽

Risk Assessment ◽

Prediction Models ◽

Assessment Tool ◽

Discrimination Performance ◽

Data Repository ◽

Good Prediction ◽

Effective Prevention ◽

Medicine Research

AbstractBackgroundStratification of individuals for their risk to develop cardiovascular diseases can be used for effective prevention and intervention. A significant amount of information for risk assessment can be obtained through repurposing electronic health records (EHR). The objective of this study is to derive and assess the performance of prediction models for cardiovascular outcomes by using EHR-derived data.MethodsWe used the Stanford Medicine Research Data Repository (STARR) data from 2000-2017, containing over 2.1 million patients. A subset of 762,372 individuals with complete International Classification of Diseases (ICD) data was used to fit Cox proportional hazard models for prediction of six cardiovascular-related diseases and type 2 diabetes.ResultsThe derived prediction models indicated consistent high discrimination performance (C-index) for all diseases examined: coronary artery disease (0.85), hypertension (0.82), type 2 diabetes (0.77), stroke (0.76), atrial fibrillation (0.82) and abdominal aortic aneurysm (0.77). Lower prediction abilities were observed for deep vein thrombosis (0.67). These results were consistent across age groups and maintained good prediction abilities among individuals with pre-existing diabetes or hypertension. Assessment of model calibration is ongoing.ConclusionsWe proposed new prediction models for the seven diseases using ICD codes derived from EHR data. EHR data can be used for health risk assessment, but challenges related to data quality and model generalizability and calibration remain to be solved.

Download Full-text

Statistical Development and Validation of Clinical Prediction Models

Anesthesiology ◽

10.1097/aln.0000000000003871 ◽

2021 ◽

Author(s):

Steven J. Staffa ◽

David Zurakowski

Keyword(s):

Prediction Model ◽

Prediction Models ◽

External Validation ◽

Model Development ◽

Predictive Ability ◽

Binary Outcome ◽

Clinical Prediction ◽

Clinical Prediction Models ◽

Surgery Research ◽

Development And Validation

Summary Clinical prediction models in anesthesia and surgery research have many clinical applications including preoperative risk stratification with implications for clinical utility in decision-making, resource utilization, and costs. It is imperative that predictive algorithms and multivariable models are validated in a suitable and comprehensive way in order to establish the robustness of the model in terms of accuracy, predictive ability, reliability, and generalizability. The purpose of this article is to educate anesthesia researchers at an introductory level on important statistical concepts involved with development and validation of multivariable prediction models for a binary outcome. Methods covered include assessments of discrimination and calibration through internal and external validation. An anesthesia research publication is examined to illustrate the process and presentation of multivariable prediction model development and validation for a binary outcome. Properly assessing the statistical and clinical validity of a multivariable prediction model is essential for reassuring the generalizability and reproducibility of the published tool.

Download Full-text