Evaluation of Electronic Health Record-Based Suicide Risk Prediction Models on Contemporary Data

2021 ◽  
Vol 12 (04) ◽  
pp. 778-787
Author(s):  
Rod L. Walker ◽  
Susan M. Shortreed ◽  
Rebecca A. Ziebell ◽  
Eric Johnson ◽  
Jennifer M. Boggs ◽  
...  
Keyword(s):  
Mental Health ◽  
Suicide Attempt ◽  
Health System ◽  
Health Systems ◽  
Risk Prediction ◽  
Suicide Risk ◽  
Prediction Models ◽  
Model Development ◽  
Risk Prediction Models ◽  
Suicide Death

Abstract Background Suicide risk prediction models have been developed by using information from patients' electronic health records (EHR), but the time elapsed between model development and health system implementation is often substantial. Temporal changes in health systems and EHR coding practices necessitate the evaluation of such models in more contemporary data. Objectives A set of published suicide risk prediction models developed by using EHR data from 2009 to 2015 across seven health systems reported c-statistics of 0.85 for suicide attempt and 0.83 to 0.86 for suicide death. Our objective was to evaluate these models' performance with contemporary data (2014–2017) from these systems. Methods We evaluated performance using mental health visits (6,832,439 to mental health specialty providers and 3,987,078 to general medical providers) from 2014 to 2017 made by 1,799,765 patients aged 13+ across the health systems. No visits in our evaluation were used in the previous model development. Outcomes were suicide attempt (health system records) and suicide death (state death certificates) within 90 days following a visit. We assessed calibration and computed c-statistics with 95% confidence intervals (CI) and cut-point specific estimates of sensitivity, specificity, and positive/negative predictive value. Results Models were well calibrated; 46% of suicide attempts and 35% of suicide deaths in the mental health specialty sample were preceded by a visit (within 90 days) with a risk score in the top 5%. In the general medical sample, 53% of attempts and 35% of deaths were preceded by such a visit. Among these two samples, respectively, c-statistics were 0.862 (95% CI: 0.860–0.864) and 0.864 (95% CI: 0.860–0.869) for suicide attempt, and 0.806 (95% CI: 0.790–0.822) and 0.804 (95% CI: 0.782–0.829) for suicide death. Conclusion Performance of the risk prediction models in this contemporary sample was similar to historical estimates for suicide attempt but modestly lower for suicide death. These published models can inform clinical practice and patient care today.

scholarly journals Risk prediction for malignant intraductal papillary mucinous neoplasm of the pancreas: logistic regression versus machine learning

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Jae Seung Kang ◽  
Chanhee Lee ◽  
Wookyeong Song ◽  
Wonho Choo ◽  
Seungyeoun Lee ◽  
...  
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Risk Prediction ◽  
Prediction Models ◽  
Model Development ◽  
Test Set ◽  
Risk Prediction Models ◽  
Mucinous Neoplasms ◽  
Tumour Location ◽  
Tenfold Cross Validation

AbstractMost models for predicting malignant pancreatic intraductal papillary mucinous neoplasms were developed based on logistic regression (LR) analysis. Our study aimed to develop risk prediction models using machine learning (ML) and LR techniques and compare their performances. This was a multinational, multi-institutional, retrospective study. Clinical variables including age, sex, main duct diameter, cyst size, mural nodule, and tumour location were factors considered for model development (MD). After the division into a MD set and a test set (2:1), the best ML and LR models were developed by training with the MD set using a tenfold cross validation. The test area under the receiver operating curves (AUCs) of the two models were calculated using an independent test set. A total of 3,708 patients were included. The stacked ensemble algorithm in the ML model and variable combinations containing all variables in the LR model were the most chosen during 200 repetitions. After 200 repetitions, the mean AUCs of the ML and LR models were comparable (0.725 vs. 0.725). The performances of the ML and LR models were comparable. The LR model was more practical than ML counterpart, because of its convenience in clinical use and simple interpretability.

scholarly journals Risk Prediction Models for Melanoma: A Systematic Review on the Heterogeneity in Model Development and Validation

2020 ◽  
Vol 17 (21) ◽  
pp. 7919
Author(s):  
Isabelle Kaiser ◽  
Annette B. Pfahlberg ◽  
Wolfgang Uter ◽  
Markus V. Heppt ◽  
Marit B. Veierød ◽  
...  
Keyword(s):  
Risk Factors ◽  
Risk Factor ◽  
Risk Prediction ◽  
Prediction Models ◽  
External Validation ◽  
Model Development ◽  
Risk Prediction Models ◽  
Screening Programs ◽  
Development And Validation ◽  
Selection And Assessment

The rising incidence of cutaneous melanoma over the past few decades has prompted substantial efforts to develop risk prediction models identifying people at high risk of developing melanoma to facilitate targeted screening programs. We review these models, regarding study characteristics, differences in risk factor selection and assessment, evaluation, and validation methods. Our systematic literature search revealed 40 studies comprising 46 different risk prediction models eligible for the review. Altogether, 35 different risk factors were part of the models with nevi being the most common one (n = 35, 78%); little consistency in other risk factors was observed. Results of an internal validation were reported for less than half of the studies (n = 18, 45%), and only 6 performed external validation. In terms of model performance, 29 studies assessed the discriminative ability of their models; other performance measures, e.g., regarding calibration or clinical usefulness, were rarely reported. Due to the substantial heterogeneity in risk factor selection and assessment as well as methodologic aspects of model development, direct comparisons between models are hardly possible. Uniform methodologic standards for the development and validation of risk prediction models for melanoma and reporting standards for the accompanying publications are necessary and need to be obligatory for that reason.

scholarly journals Rare Germline Pathogenic Variants Identified by Multigene Panel Testing and the Risk of Aggressive Prostate Cancer

Cancers ◽  
2021 ◽  
Vol 13 (7) ◽  
pp. 1495
Author(s):  
Tú Nguyen-Dumont ◽  
James G. Dowty ◽  
Robert J. MacInnis ◽  
Jason A. Steen ◽  
Moeen Riaz ◽  
...  
Keyword(s):  
Prostate Cancer ◽  
Risk Prediction ◽  
Prediction Models ◽  
Risk Prediction Models ◽  
Aggressive Prostate Cancer ◽  
Panel Testing ◽  
Pathogenic Variants ◽  
Increased Risk ◽  
Predisposition Genes ◽  
Rare Genetic Variants

While gene panel sequencing is becoming widely used for cancer risk prediction, its clinical utility with respect to predicting aggressive prostate cancer (PrCa) is limited by our current understanding of the genetic risk factors associated with predisposition to this potentially lethal disease phenotype. This study included 837 men diagnosed with aggressive PrCa and 7261 controls (unaffected men and men who did not meet criteria for aggressive PrCa). Rare germline pathogenic variants (including likely pathogenic variants) were identified by targeted sequencing of 26 known or putative cancer predisposition genes. We found that 85 (10%) men with aggressive PrCa and 265 (4%) controls carried a pathogenic variant (p < 0.0001). Aggressive PrCa odds ratios (ORs) were estimated using unconditional logistic regression. Increased risk of aggressive PrCa (OR (95% confidence interval)) was identified for pathogenic variants in BRCA2 (5.8 (2.7–12.4)), BRCA1 (5.5 (1.8–16.6)), and ATM (3.8 (1.6–9.1)). Our study provides further evidence that rare germline pathogenic variants in these genes are associated with increased risk of this aggressive, clinically relevant subset of PrCa. These rare genetic variants could be incorporated into risk prediction models to improve their precision to identify men at highest risk of aggressive prostate cancer and be used to identify men with newly diagnosed prostate cancer who require urgent treatment.

scholarly journals Risk Prediction of Barrett’s Esophagus in a Taiwanese Health Examination Center Based on Regression Models

2021 ◽  
Vol 18 (10) ◽  
pp. 5332
Author(s):  
Po-Hsiang Lin ◽  
Jer-Guang Hsieh ◽  
Hsien-Chung Yu ◽  
Jyh-Horng Jeng ◽  
Chiao-Lin Hsu ◽  
...  
Keyword(s):  
Barrett’S Esophagus ◽  
Risk Prediction ◽  
Barrett's Esophagus ◽  
Gastrointestinal Endoscopy ◽  
Prediction Models ◽  
Upper Gastrointestinal Endoscopy ◽  
Health Examination ◽  
Endoscopic Screening ◽  
Risk Prediction Models ◽  
Upper Gastrointestinal

Determining the target population for the screening of Barrett’s esophagus (BE), a precancerous condition of esophageal adenocarcinoma, remains a challenge in Asia. The aim of our study was to develop risk prediction models for BE using logistic regression (LR) and artificial neural network (ANN) methods. Their predictive performances were compared. We retrospectively analyzed 9646 adults aged ≥20 years undergoing upper gastrointestinal endoscopy at a health examinations center in Taiwan. Evaluated by using 10-fold cross-validation, both models exhibited good discriminative power, with comparable area under curve (AUC) for the LR and ANN models (Both AUC were 0.702). Our risk prediction models for BE were developed from individuals with or without clinical indications of upper gastrointestinal endoscopy. The models have the potential to serve as a practical tool for identifying high-risk individuals of BE among the general population for endoscopic screening.

scholarly journals Evaluating risk prediction models for adults with heart failure: A systematic literature review

PLoS ONE ◽  
2020 ◽  
Vol 15 (1) ◽  
pp. e0224135 ◽  
Author(s):  
Gian Luca Di Tanna ◽  
Heidi Wirtz ◽  
Karen L. Burrows ◽  
Gary Globe
Keyword(s):  
Heart Failure ◽  
Literature Review ◽  
Risk Prediction ◽  
Systematic Literature Review ◽  
Prediction Models ◽  
Risk Prediction Models
Sign in / Sign up

Export Citation Format

Share Document