scholarly journals Temporally-Informed Random Forests for Suicide Risk Prediction

2021 ◽  
Author(s):  
Ilkin Bayramli ◽  
Victor Castro ◽  
Yuval Barak-Corrren ◽  
Emily Masden ◽  
Matthew Nock ◽  
...  
Keyword(s):  
Random Forest ◽  
Risk Prediction ◽  
Random Forests ◽  
Suicide Risk ◽  
Prediction Models ◽  
Predictive Performance ◽  
Temporal Information ◽  
Risk Detection ◽  
Temporal Variables ◽  
Algorithmic Approaches

Background. Suicide is one of the leading causes of death worldwide, yet clinicians find it difficult to reliably identify individuals at high risk for suicide. Algorithmic approaches for suicide risk detection have been developed in recent years, mostly based on data from electronics health records (EHRs). These models typically do not optimally exploit the valuable temporal information inherent in these longitudinal data. Methods. We propose a temporally enhanced variant of the Random Forest model - Omni-Temporal Balanced Random Forests (OTBRFs) - that incorporates temporal information in every tree within the forest. We develop and validate this model using longitudinal EHRs and clinician notes from the Mass General Brigham Health System recorded between 1998 and 2018, and compare its performance to a baseline Naive Bayes Classifier and two standard versions of Balanced Random Forests. Results. Temporal variables were found to be associated with suicide risk. RF models were more accurate than Naive Bayesian classifiers at predicting suicide risk in advance (AUC=0.824 vs. 0.754 respectively). The OT-BRF model performed best among all RF approaches (0.339 sensitivity at 95% specificity), compared to 0.290 and 0.286 for the other two RF models. Temporal variables were assigned high importance by the models that incorporated them. Discussion. We demonstrate that temporal variables have an important role to play in suicide risk detection, and that requiring their inclusion in all random forest trees leads to increased predictive performance. Integrating temporal information into risk prediction models helps the models interpret patient data in temporal context, improving predictive performance.

scholarly journals Development of Machine Learning Models to Predict Probabilities and Types of Stroke at Prehospital Stage: the Japan Urgent Stroke Triage Score Using Machine Learning (JUST-ML)

2021 ◽  
Author(s):  
Kazutaka Uchida ◽  
Junichi Kouno ◽  
Shinichi Yoshimura ◽  
Norito Kinjo ◽  
Fumihiro Sakakibara ◽  
...  
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Random Forests ◽  
Prediction Models ◽  
Characteristic Curve ◽  
Predictive Performance ◽  
Vessel Occlusion ◽  
Predictive Values ◽  
Training Cohort ◽  
Sensitivity Specificity

AbstractIn conjunction with recent advancements in machine learning (ML), such technologies have been applied in various fields owing to their high predictive performance. We tried to develop prehospital stroke scale with ML. We conducted multi-center retrospective and prospective cohort study. The training cohort had eight centers in Japan from June 2015 to March 2018, and the test cohort had 13 centers from April 2019 to March 2020. We use the three different ML algorithms (logistic regression, random forests, XGBoost) to develop models. Main outcomes were large vessel occlusion (LVO), intracranial hemorrhage (ICH), subarachnoid hemorrhage (SAH), and cerebral infarction (CI) other than LVO. The predictive abilities were validated in the test cohort with accuracy, positive predictive value, sensitivity, specificity, area under the receiver operating characteristic curve (AUC), and F score. The training cohort included 3178 patients with 337 LVO, 487 ICH, 131 SAH, and 676 CI cases, and the test cohort included 3127 patients with 183 LVO, 372 ICH, 90 SAH, and 577 CI cases. The overall accuracies were 0.65, and the positive predictive values, sensitivities, specificities, AUCs, and F scores were stable in the test cohort. The classification abilities were also fair for all ML models. The AUCs for LVO of logistic regression, random forests, and XGBoost were 0.89, 0.89, and 0.88, respectively, in the test cohort, and these values were higher than the previously reported prediction models for LVO. The ML models developed to predict the probability and types of stroke at the prehospital stage had superior predictive abilities.

scholarly journals Improving Outcome Predictions for Patients Receiving Mechanical Circulatory Support by Optimizing Imputation of Missing Values

2021 ◽  
Author(s):  
Byron C. Jaeger ◽  
Ryan Cantor ◽  
Venkata Sthanam ◽  
Rongbing Xie ◽  
James K. Kirklin ◽  
...  
Keyword(s):  
Multiple Imputation ◽  
Risk Prediction ◽  
Random Forests ◽  
Missing Values ◽  
Prediction Models ◽  
Model Performance ◽  
Circulatory Support ◽  
Risk Prediction Models ◽  
Prognostic Accuracy ◽  
The Mean

Background: Risk prediction models play an important role in clinical decision making. When developing risk prediction models, practitioners often impute missing values to the mean. We evaluated the impact of applying other strategies to impute missing values on the prognostic accuracy of downstream risk prediction models, that is, models fitted to the imputed data. A secondary objective was to compare the accuracy of imputation methods based on artificially induced missing values. To complete these objectives, we used data from the Interagency Registry for Mechanically Assisted Circulatory Support. Methods: We applied 12 imputation strategies in combination with 2 different modeling strategies for mortality and transplant risk prediction following surgery to receive mechanical circulatory support. Model performance was evaluated using Monte-Carlo cross-validation and measured based on outcomes 6 months following surgery using the scaled Brier score, concordance index, and calibration error. We used Bayesian hierarchical models to compare model performance. Results: Multiple imputation with random forests emerged as a robust strategy to impute missing values, increasing model concordance by 0.0030 (25th–75th percentile: 0.0008–0.0052) compared with imputation to the mean for mortality risk prediction using a downstream proportional hazards model. The posterior probability that single and multiple imputation using random forests would improve concordance versus mean imputation was 0.464 and >0.999, respectively. Conclusions: Selecting an optimal strategy to impute missing values such as random forests and applying multiple imputation can improve the prognostic accuracy of downstream risk prediction models.

Implementing machine learning algorithms for suicide risk prediction in clinical practice: A focus group study

2021 ◽  
Author(s):  
Kate Bentley ◽  
Kelly Zuromski ◽  
Rebecca Fortgang ◽  
Emily Madsen ◽  
Daniel Kessler ◽  
...  
Keyword(s):  
Machine Learning ◽  
Risk Assessment ◽  
Clinical Practice ◽  
Risk Prediction ◽  
Suicide Risk ◽  
Prediction Models ◽  
Clinical Decision ◽  
Machine Learning Algorithms ◽  
Suicide Risk Assessment ◽  
Focus Group Study

Background: Interest in developing machine learning algorithms that use electronic health record data to predict patients’ risk of suicidal behavior has recently proliferated. Whether and how such models might be implemented and useful in clinical practice, however, remains unknown. In order to ultimately make automated suicide risk prediction algorithms useful in practice, and thus better prevent patient suicides, it is critical to partner with key stakeholders (including the frontline providers who will be using such tools) at each stage of the implementation process.Objective: The aim of this focus group study was to inform ongoing and future efforts to deploy suicide risk prediction models in clinical practice. The specific goals were to better understand hospital providers’ current practices for assessing and managing suicide risk; determine providers’ perspectives on using automated suicide risk prediction algorithms; and identify barriers, facilitators, recommendations, and factors to consider for initiatives in this area. Methods: We conducted 10 two-hour focus groups with a total of 40 providers from psychiatry, internal medicine and primary care, emergency medicine, and obstetrics and gynecology departments within an urban academic medical center. Audio recordings of open-ended group discussions were transcribed and coded for relevant and recurrent themes by two independent study staff members. All coded text was reviewed and discrepancies resolved in consensus meetings with doctoral-level staff. Results: Though most providers reported using standardized suicide risk assessment tools in their clinical practices, existing tools were commonly described as unhelpful and providers indicated dissatisfaction with current suicide risk assessment methods. Overall, providers’ general attitudes toward the practical use of automated suicide risk prediction models and corresponding clinical decision support tools were positive. Providers were especially interested in the potential to identify high-risk patients who might be missed by traditional screening methods. Some expressed skepticism about the potential usefulness of these models in routine care; specific barriers included concerns about liability, alert fatigue, and increased demand on the healthcare system. Key facilitators included presenting specific patient-level features contributing to risk scores, emphasizing changes in risk over time, and developing systematic clinical workflows and provider trainings. Participants also recommended considering risk-prediction windows, timing of alerts, who will have access to model predictions, and variability across treatment settings.Conclusions: Providers were dissatisfied with current suicide risk assessment methods and open to the use of a machine learning-based risk prediction system to inform clinical decision-making. They also raised multiple concerns about potential barriers to the usefulness of this approach and suggested several possible facilitators. Future efforts in this area will benefit from incorporating systematic qualitative feedback from providers, patients, administrators, and payers on the use of new methods in routine care, especially given the complex, sensitive, and unfortunately still stigmatized nature of suicide risk.

scholarly journals Urban Crime Risk Prediction Using Point of Interest Data

2020 ◽  
Vol 9 (7) ◽  
pp. 459
Author(s):  
Paweł Cichosz
Keyword(s):  
Risk Prediction ◽  
Urban Areas ◽  
Prediction Models ◽  
Learning Algorithms ◽  
Predictive Performance ◽  
Machine Learning Algorithms ◽  
Geographical Information ◽  
Risk Prediction Models ◽  
Point Of Interest ◽  
Crime Risk

Geographical information systems have found successful applications to prediction and decision-making in several areas of vital importance to contemporary society. This article demonstrates how they can be combined with machine learning algorithms to create crime prediction models for urban areas. Selected point of interest (POI) layers from OpenStreetMap are used to derive attributes describing micro-areas, which are assigned crime risk classes based on police crime records. POI attributes then serve as input attributes for learning crime risk prediction models with classification learning algorithms. The experimental results obtained for four UK urban areas suggest that POI attributes have high predictive utility. Classification models using these attributes, without any form of location identification, exhibit good predictive performance when applied to new, previously unseen micro-areas. This makes them capable of crime risk prediction for newly developed or dynamically changing neighborhoods. The high dimensionality of the model input space can be considerably reduced without predictive performance loss by attribute selection or principal component analysis. Models trained on data from one area achieve a good level of prediction quality when applied to another area, which makes it possible to transfer or combine crime risk prediction models across different urban areas.

scholarly journals Evaluation of Feature Selection Techniques for Breast Cancer Risk Prediction

2021 ◽  
Vol 18 (20) ◽  
pp. 10670
Author(s):  
Nahúm Cueto López ◽  
María Teresa García-Ordás ◽  
Facundo Vitelli-Storelli ◽  
Pablo Fernández-Navarro ◽  
Camilo Palazuelos ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Feature Selection ◽  
Random Forest ◽  
Cancer Risk ◽  
Risk Prediction ◽  
Prediction Models ◽  
Public Health Problem ◽  
Healthy Population ◽  
Major Public Health Problem ◽  
The Stability

This study evaluates several feature ranking techniques together with some classifiers based on machine learning to identify relevant factors regarding the probability of contracting breast cancer and improve the performance of risk prediction models for breast cancer in a healthy population. The dataset with 919 cases and 946 controls comes from the MCC-Spain study and includes only environmental and genetic features. Breast cancer is a major public health problem. Our aim is to analyze which factors in the cancer risk prediction model are the most important for breast cancer prediction. Likewise, quantifying the stability of feature selection methods becomes essential before trying to gain insight into the data. This paper assesses several feature selection algorithms in terms of performance for a set of predictive models. Furthermore, their robustness is quantified to analyze both the similarity between the feature selection rankings and their own stability. The ranking provided by the SVM-RFE approach leads to the best performance in terms of the area under the ROC curve (AUC) metric. Top-47 ranked features obtained with this approach fed to the Logistic Regression classifier achieve an AUC = 0.616. This means an improvement of 5.8% in comparison with the full feature set. Furthermore, the SVM-RFE ranking technique turned out to be highly stable (as well as Random Forest), whereas relief and the wrapper approaches are quite unstable. This study demonstrates that the stability and performance of the model should be studied together as Random Forest and SVM-RFE turned out to be the most stable algorithms, but in terms of model performance SVM-RFE outperforms Random Forest.

Evaluation of Electronic Health Record-Based Suicide Risk Prediction Models on Contemporary Data

2021 ◽  
Vol 12 (04) ◽  
pp. 778-787
Author(s):  
Rod L. Walker ◽  
Susan M. Shortreed ◽  
Rebecca A. Ziebell ◽  
Eric Johnson ◽  
Jennifer M. Boggs ◽  
...  
Keyword(s):  
Mental Health ◽  
Suicide Attempt ◽  
Health System ◽  
Health Systems ◽  
Risk Prediction ◽  
Suicide Risk ◽  
Prediction Models ◽  
Model Development ◽  
Risk Prediction Models ◽  
Suicide Death

Abstract Background Suicide risk prediction models have been developed by using information from patients' electronic health records (EHR), but the time elapsed between model development and health system implementation is often substantial. Temporal changes in health systems and EHR coding practices necessitate the evaluation of such models in more contemporary data. Objectives A set of published suicide risk prediction models developed by using EHR data from 2009 to 2015 across seven health systems reported c-statistics of 0.85 for suicide attempt and 0.83 to 0.86 for suicide death. Our objective was to evaluate these models' performance with contemporary data (2014–2017) from these systems. Methods We evaluated performance using mental health visits (6,832,439 to mental health specialty providers and 3,987,078 to general medical providers) from 2014 to 2017 made by 1,799,765 patients aged 13+ across the health systems. No visits in our evaluation were used in the previous model development. Outcomes were suicide attempt (health system records) and suicide death (state death certificates) within 90 days following a visit. We assessed calibration and computed c-statistics with 95% confidence intervals (CI) and cut-point specific estimates of sensitivity, specificity, and positive/negative predictive value. Results Models were well calibrated; 46% of suicide attempts and 35% of suicide deaths in the mental health specialty sample were preceded by a visit (within 90 days) with a risk score in the top 5%. In the general medical sample, 53% of attempts and 35% of deaths were preceded by such a visit. Among these two samples, respectively, c-statistics were 0.862 (95% CI: 0.860–0.864) and 0.864 (95% CI: 0.860–0.869) for suicide attempt, and 0.806 (95% CI: 0.790–0.822) and 0.804 (95% CI: 0.782–0.829) for suicide death. Conclusion Performance of the risk prediction models in this contemporary sample was similar to historical estimates for suicide attempt but modestly lower for suicide death. These published models can inform clinical practice and patient care today.

scholarly journals Systematic review of risk prediction scores for surgical site infection or periprosthetic joint infection following joint arthroplasty

2017 ◽  
Vol 145 (9) ◽  
pp. 1738-1749 ◽  
Author(s):  
S. K. KUNUTSOR ◽  
M. R. WHITEHOUSE ◽  
A. W. BLOM ◽  
A. D. BESWICK
Keyword(s):  
Systematic Review ◽  
Risk Prediction ◽  
Prediction Models ◽  
Joint Infection ◽  
External Validation ◽  
Surgical Site Infections ◽  
Predictive Performance ◽  
Risk Scores ◽  
Accurate Identification ◽  
Discriminative Ability

SUMMARYAccurate identification of individuals at high risk of surgical site infections (SSIs) or periprosthetic joint infections (PJIs) influences clinical decisions and development of preventive strategies. We aimed to determine progress in the development and validation of risk prediction models for SSI or PJI using a systematic review. We searched for studies that have developed or validated a risk prediction tool for SSI or PJI following joint replacement in MEDLINE, EMBASE, Web of Science and Cochrane databases; trial registers and reference lists of studies up to September 2016. Nine studies describing 16 risk scores for SSI or PJI were identified. The number of component variables in a risk score ranged from 4 to 45. The C-index ranged from 0·56 to 0·74, with only three risk scores reporting a discriminative ability of >0·70. Five risk scores were validated internally. The National Healthcare Safety Network SSIs risk models for hip and knee arthroplasties (HPRO and KPRO) were the only scores to be externally validated. Except for HPRO which shows some promise for use in a clinical setting (based on predictive performance and external validation), none of the identified risk scores can be considered ready for use. Further research is urgently warranted within the field.

scholarly journals Prediction of arrhythmia after intervention in children with atrial septal defect based on random forest

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Hongxiao Sun ◽  
Yuhai Liu ◽  
Bo Song ◽  
Xiaowen Cui ◽  
Gang Luo ◽  
...  
Keyword(s):  
Random Forest ◽  
Prediction Model ◽  
Atrial Septal Defect ◽  
Risk Prediction ◽  
Area Under Curve ◽  
Septal Defect ◽  
Prediction Models ◽  
Postoperative Arrhythmia ◽  
Risk Prediction Models ◽  
Interventional Closure

Abstract Background Using random forest to predict arrhythmia after intervention in children with atrial septal defect. Methods We constructed a prediction model of complications after interventional closure for children with atrial septal defect. The model was based on random forest, and it solved the need for postoperative arrhythmia risk prediction and assisted clinicians and patients’ families to make preoperative decisions. Results Available risk prediction models provided patients with specific risk factor assessments, we used Synthetic Minority Oversampling Technique algorithm and random forest machine learning to propose a prediction model, and got a prediction accuracy of 94.65 % and an Area Under Curve value of 0.8956. Conclusions Our study was based on the model constructed by random forest, which can effectively predict the complications of arrhythmia after interventional closure in children with atrial septal defect.

scholarly journals Ensemble Learning Models for Food Safety Risk Prediction

2021 ◽  
Vol 13 (21) ◽  
pp. 12291
Author(s):  
Li-Ya Wu ◽  
Sung-Shun Weng
Keyword(s):  
Ensemble Learning ◽  
Risk Prediction ◽  
Prediction Models ◽  
Confusion Matrix ◽  
Area Under The Curve ◽  
Predictive Performance ◽  
Learning Models ◽  
Hit Rate ◽  
Online Implementation ◽  
Food Safety Risk

Ensemble learning was adopted to design risk prediction models with the aim of improving border inspection methods for food imported into Taiwan. Specifically, we constructed a set of prediction models to enhance the hit rate of non-conforming products, thus strengthening the border control of food products to safeguard public health. Using five algorithms, we developed models to provide recommendations for the risk assessment of each imported food batch. The models were evaluated by constructing a confusion matrix to calculate predictive performance indicators, including the positive prediction value (PPV), recall, harmonic mean of PPV and recall (F1 score), and area under the curve. Our results showed that ensemble learning achieved better and more stable prediction results than any single algorithm. When the results of comparable data periods were examined, the non-conformity hit rate was found to increase significantly after online implementation of the ensemble learning models, indicating that ensemble learning was effective at risk prediction. In addition to enhancing the inspection hit rate of non-conforming food, the results of this study can serve as a reference for the improvement of existing random inspection methods, thus strengthening capabilities in food risk management.

Sign in / Sign up

Export Citation Format

Share Document