Prediction of COVID-19 severity using laboratory findings on admission: informative values, thresholds, ML model performance

BackgroundDespite the necessity, there is no reliable biomarker to predict disease severity and prognosis of patients with COVID-19. The currently published prediction models are not fully applicable to clinical use.ObjectivesTo identify predictive biomarkers of COVID-19 severity and to justify their threshold values for the stratification of the risk of deterioration that would require transferring to the intensive care unit (ICU).MethodsThe study cohort (560 subjects) included all consecutive patients admitted to Dubai Mediclinic Parkview Hospital from February to May 2020 with COVID-19 confirmed by the PCR. The challenge of finding the cut-off thresholds was the unbalanced dataset (eg, the disproportion in the number of 72 patients admitted to ICU vs 488 non-severe cases). Therefore, we customised supervised machine learning (ML) algorithm in terms of threshold value used to predict worsening.ResultsWith the default thresholds returned by the ML estimator, the performance of the models was low. It was improved by setting the cut-off level to the 25th percentile for lymphocyte count and the 75th percentile for other features. The study justified the following threshold values of the laboratory tests done on admission: lymphocyte count <2.59×109/L, and the upper levels for total bilirubin 11.9 μmol/L, alanine aminotransferase 43 U/L, aspartate aminotransferase 32 U/L, D-dimer 0.7 mg/L, activated partial thromboplastin time (aPTT) 39.9 s, creatine kinase 247 U/L, C reactive protein (CRP) 14.3 mg/L, lactate dehydrogenase 246 U/L, troponin 0.037 ng/mL, ferritin 498 ng/mL and fibrinogen 446 mg/dL.ConclusionThe performance of the neural network trained with top valuable tests (aPTT, CRP and fibrinogen) is admissible (area under the curve (AUC) 0.86; 95% CI 0.486 to 0.884; p<0.001) and comparable with the model trained with all the tests (AUC 0.90; 95% CI 0.812 to 0.902; p<0.001). Free online tool at https://med-predict.com illustrates the study results.

Download Full-text

Classification models using circulating neutrophil transcripts can detect unruptured intracranial aneurysm

Journal of Translational Medicine ◽

10.1186/s12967-020-02550-2 ◽

2020 ◽

Vol 18 (1) ◽

Author(s):

Kerry E. Poppenberg ◽

Vincent M. Tutino ◽

Lu Li ◽

Muhammad Waqas ◽

Armond June ◽

...

Keyword(s):

Machine Learning ◽

Random Forest ◽

Prediction Models ◽

Model Performance ◽

Supervised Machine Learning ◽

Support Vector ◽

Learning Methods ◽

Training Cohort ◽

Network Analyses ◽

Machine Learning Methods

Abstract Background Intracranial aneurysms (IAs) are dangerous because of their potential to rupture. We previously found significant RNA expression differences in circulating neutrophils between patients with and without unruptured IAs and trained machine learning models to predict presence of IA using 40 neutrophil transcriptomes. Here, we aim to develop a predictive model for unruptured IA using neutrophil transcriptomes from a larger population and more robust machine learning methods. Methods Neutrophil RNA extracted from the blood of 134 patients (55 with IA, 79 IA-free controls) was subjected to next-generation RNA sequencing. In a randomly-selected training cohort (n = 94), the Least Absolute Shrinkage and Selection Operator (LASSO) selected transcripts, from which we constructed prediction models via 4 well-established supervised machine-learning algorithms (K-Nearest Neighbors, Random Forest, and Support Vector Machines with Gaussian and cubic kernels). We tested the models in the remaining samples (n = 40) and assessed model performance by receiver-operating-characteristic (ROC) curves. Real-time quantitative polymerase chain reaction (RT-qPCR) of 9 IA-associated genes was used to verify gene expression in a subset of 49 neutrophil RNA samples. We also examined the potential influence of demographics and comorbidities on model prediction. Results Feature selection using LASSO in the training cohort identified 37 IA-associated transcripts. Models trained using these transcripts had a maximum accuracy of 90% in the testing cohort. The testing performance across all methods had an average area under ROC curve (AUC) = 0.97, an improvement over our previous models. The Random Forest model performed best across both training and testing cohorts. RT-qPCR confirmed expression differences in 7 of 9 genes tested. Gene ontology and IPA network analyses performed on the 37 model genes reflected dysregulated inflammation, cell signaling, and apoptosis processes. In our data, demographics and comorbidities did not affect model performance. Conclusions We improved upon our previous IA prediction models based on circulating neutrophil transcriptomes by increasing sample size and by implementing LASSO and more robust machine learning methods. Future studies are needed to validate these models in larger cohorts and further investigate effect of covariates.

Download Full-text

Predicting poor outcome in patients with suspected COVID-19 presenting to the Emergency Department (COVERED) – Development, internal and external validation of a prediction model

Acute Medicine Journal ◽

10.52964/amja.0836 ◽

2021 ◽

Vol 20 (1) ◽

pp. 4-14

Author(s):

K. Azijli ◽

◽

A.W.E. Lieveld ◽

S.F.B. van der Horst ◽

N. de Graaf ◽

...

Keyword(s):

Emergency Department ◽

Prediction Model ◽

Clinical Decision Making ◽

Prediction Models ◽

External Validation ◽

Model Performance ◽

Area Under The Curve ◽

Brier Score ◽

Multi Centre Study ◽

Poor Outcome

Background: A recent systematic review recommends against the use of any of the current COVID-19 prediction models in clinical practice. To enable clinicians to appropriately profile and treat suspected COVID-19 patients at the emergency department (ED), externally validated models that predict poor outcome are desperately needed. Objective: Our aims were to identify predictors of poor outcome, defined as mortality or ICU admission within 30 days, in patients presenting to the ED with a clinical suspicion of COVID-19, and to develop and externally validate a prediction model for poor outcome. Methods: In this prospective, multi-centre study, we enrolled suspected COVID-19 patients presenting at the EDs of two hospitals in the Netherlands. We used backward logistic regression to develop a prediction model. We used the area under the curve (AUC), Brier score and pseudo-R2 to assess model performance. The model was externally validated in an Italian cohort. Results: We included 1193 patients between March 12 and May 27 2020, of whom 196 (16.4%) had a poor outcome. We identified 10 predictors of poor outcome: current malignancy (OR 2.774; 95%CI 1.682-4.576), systolic blood pressure (OR 0.981; 95%CI 0.964-0.998), heart rate (OR 1.001; 95%CI 0.97-1.028), respiratory rate (OR 1.078; 95%CI 1.046-1.111), oxygen saturation (OR 0.899; 95%CI 0.850-0.952), body temperature (OR 0.505; 95%CI 0.359-0.710), serum urea (OR 1.404; 95%CI 1.198-1.645), C-reactive protein (OR 1.013; 95%CI 1.001-1.024), lactate dehydrogenase (OR 1.007; 95%CI 1.002-1.013) and SARS-CoV-2 PCR result (OR 2.456; 95%CI 1.526-3.953). The AUC was 0.86 (95%CI 0.83-0.89), with a Brier score of 0.32 and, and R2 of 0.41. The AUC in the external validation in 500 patients was 0.70 (95%CI 0.65-0.75). Conclusion: The COVERED risk score showed excellent discriminatory ability, also in external validation. It may aid clinical decision making, and improve triage at the ED in health care environments with high patient throughputs.

Download Full-text

Algorithms for Supervised Machine Learning-Based Structural Performance Evaluation Framework

Volume 2A: Structures, Safety, and Reliability ◽

10.1115/omae2020-18947 ◽

2020 ◽

Author(s):

Xiaowei Wang ◽

YeongAe Heo

Keyword(s):

Machine Learning ◽

Prediction Models ◽

Numerical Models ◽

Evaluation Framework ◽

Supervised Machine Learning ◽

System Level ◽

Structural Performance ◽

Fe Model ◽

Study Results

Abstract Machine learning (ML) approaches have gained increasing attention in the structural engineering field to evaluate structural performance using structural health monitoring (SHM) data. Supervised ML approaches can accelerate the learning process by using labeled training datasets to map an input to output dataset. But, SHM data are not informative to drive a mapping function to determine the real-world performance of large-scale complex structures in particular for future events. To leverage a framework for evaluating the system-level structural performance, this study couples supervised ML approaches with an advanced finite element (FE) model considering pre- and post-event model validation and updating. A well-instrumented system experiencing multiple seismic events is employed as a case study to demonstrate the proposed framework. An FE model of the instrumented system is created and validated using pre-event SHM datasets. Numerical data obtained from the FE model are used for datasets to develop ML prediction models, which are then validated by a post-event SHM dataset. Eight popular ML algorithms are examined and compared to shed light on the effectiveness of the ML algorithms for the proposed framework. The case study results indicate that the Random Forests and Neural Network algorithms provide better estimation for the structural system. The results also imply the need of post-event updating for numerical models used in the case study.

Download Full-text

Prognostic Value of Preoperative Absolute Lymphocyte Count in Children With Tetralogy of Fallot

Journal of the American Heart Association ◽

10.1161/jaha.120.019098 ◽

2021 ◽

Author(s):

Xie Wu ◽

Qipeng Luo ◽

Zhanhao Su ◽

Yinan Li ◽

Hongbai Wang ◽

...

Keyword(s):

Tetralogy Of Fallot ◽

Lymphocyte Count ◽

Poor Prognosis ◽

Prognostic Value ◽

Low Cost ◽

Characteristic Curve ◽

Model Performance ◽

Area Under The Curve ◽

Absolute Lymphocyte Count ◽

Reverse Causality

Background Tetralogy of Fallot (TOF) is the most common cyanotic congenital heart disease. Absolute lymphocyte count (ALC) is a low‐cost and easy‐to‐obtain inflammatory indicator; however, its association with the prognosis of patients with TOF remains unknown. This study aimed to determine the prognostic value of preoperative ALC in children with TOF. Methods and Results This retrospective study included 707 patients aged <6 years who underwent corrective operations for TOF between January 2016 and December 2018 in Fuwai Hospital, China. The end points were mortality, extracorporeal membrane oxygenation placement, postoperative hospital stay >30 days, and severe postoperative complications; patients were grouped on the basis of prognosis: poor prognosis (n=76) and good prognosis (n=631). Univariable and multivariable logistic regression analyses were performed to identify the independent risk factors for poor prognosis, on which a risk scoring system was based. The receiver operating characteristic curve was used to assess model performance. Using another model without ALC, the effect of the addition of ALC was assessed. Results suggested that ALC was an independent factor with a cutoff point of 4.36×10 9 /L. The addition of ALC improved the area under the curve from 0.771 to 0.781 ( P <0.001). To avoid reverse causality and further control for confounding factors, the patients were further divided on the basis of ALC level, and a propensity score matching was performed; 117 paired patients were identified for further analysis. Low ALC levels had an odds ratio of 3.500 (95% CI, 1.413–8.672). Conclusions Low preoperative ALC represents an independent predictor of poor prognosis in children with TOF.

Download Full-text

Predicting inhospital admission at the emergency department: a systematic review

Emergency Medicine Journal ◽

10.1136/emermed-2020-210902 ◽

2021 ◽

pp. emermed-2020-210902

Author(s):

Anniek Brink ◽

Jelmer Alsma ◽

Lodewijk AAM van Attekum ◽

Wichor M Bramer ◽

Robert Zietse ◽

...

Keyword(s):

Systematic Review ◽

Prediction Models ◽

Data Extraction ◽

Model Development ◽

Model Performance ◽

Area Under The Curve ◽

Registration Number ◽

Modelling Studies ◽

Assess Quality ◽

The Uk

BackgroundED crowding has potential detrimental consequences for both patient care and staff. Advancing disposition can reduce crowding. This may be achieved by using prediction models for admission. This systematic review aims to present an overview of prediction models for admission at the ED. Furthermore, we aimed to identify the best prediction tool based on its performance, validation, calibration and clinical usability.MethodsWe included observational studies published in Embase.com, Medline Ovid, Cochrane CENTRAL, Web of Science Core Collection or Google scholar, in which admission models were developed or validated in a general medical population in European EDs including the UK. We used the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist to assess quality of model development. Model performance was presented as discrimination and calibration. The search was performed on 11 October 2020.ResultsIn total, 18 539 articles were identified. We included 11 studies, describing 16 different models, comprising the development of 9 models and 12 external validations of 11 models. The risk of bias of the development studies was considered low to medium. Discrimination, as represented by the area under the curve ranged from 0.630 to 0.878. Calibration was assessed in seven models and was strong. The best performing models are the models of Lucke et al and Cameron et al. These models combine clinical applicability, by inclusion of readily available parameters, and appropriate discrimination, calibration and validation.ConclusionNone of the models are yet implemented in EDs. Further research is needed to assess the applicability and implementation of the best performing models in the ED.Systematic review registration numberPROSPERO CRD42017057975.

Download Full-text

Evaluating the Impact of Covariate Lookback Times on Performance of Patient-Level Prediction Models

10.21203/rs.3.rs-227607/v1 ◽

2021 ◽

Author(s):

Jill Hardin ◽

Jenna M. Reps

Keyword(s):

Gastrointestinal Bleeding ◽

Prediction Models ◽

External Validation ◽

Model Performance ◽

Area Under The Curve ◽

Logistic Models ◽

Optimal Model ◽

Discriminative Performance ◽

Patient Level ◽

The Impact

Abstract Background: The goal of our study is to provide guidance for deciding which length of lookback to implement when engineering features to use when developing predictive models using observational healthcare data. Using a longer lookback for feature engineering gives more insight about patients but increases the issue of left-censoring. Methods: We used five US observational databases to develop patient-level prediction models. A target cohort of subjects with hypertensive drug exposures and outcome cohorts of subjects with acute (stroke and gastrointestinal bleeding) and chronic outcomes (diabetes and chronic kidney disease) were developed. Candidate predictors that exist on or prior to the target index date were derived within the following lookback periods: 14, 30, 90, 180, 365, 730, and all days prior to index were evaluated. We predicted the risk of outcomes occurring 1 day until 365 days after index. Ten lasso logistic models for each lookback period were generated to create a distribution of area under the curve (AUC) metrics to evaluate the discriminative performance of the models. Impact on external validation performance was investigated across five databases. Results: Our results show that a shorter lookback time for the acute outcomes, stroke and gastrointestinal bleeding results in equivalent performance as using longer lookback times. A lookback time of at least 365 days for the chronic outcomes, diabetes and renal impairment, results in equivalent performance as using lookback times greater than 365 days.Conclusions: Our results suggest the optimal model performance and choice of length of lookback is dependent on the outcome type (acute or chronic). Our study illustrates that use of at least 365 days results in equivalent performance as using longer lookback periods. Researchers should evaluate lookback in the context of the prediction question to determine optimal model performance.

Download Full-text

Classification Models using Circulating Neutrophil Transcripts Can Detect Unruptured Intracranial Aneurysm

10.21203/rs.3.rs-17161/v2 ◽

2020 ◽

Author(s):

Kerry E Poppenberg ◽

Vincent M Tutino ◽

Lu Li ◽

Muhammad Waqas ◽

Armond June ◽

...

Keyword(s):

Machine Learning ◽

Random Forest ◽

Prediction Models ◽

Model Performance ◽

Supervised Machine Learning ◽

Support Vector ◽

Learning Methods ◽

Training Cohort ◽

Network Analyses ◽

Machine Learning Methods

Abstract Background: Intracranial aneurysms (IAs) are dangerous because of their potential to rupture. We previously found significant RNA expression differences in circulating neutrophils between patients with and without unruptured IAs and trained machine learning models to predict presence of IA using 40 neutrophil transcriptomes. Here, we aim to develop a predictive model for unruptured IA using neutrophil transcriptomes from a larger population and more robust machine learning methods. Methods: Neutrophil RNA extracted from the blood of 134 patients (55 with IA, 79 IA-free controls) was subjected to next-generation RNA sequencing. In a randomly-selected training cohort (n=94), the Least Absolute Shrinkage and Selection Operator (LASSO) selected transcripts, from which we constructed prediction models via 4 well-established supervised machine-learning algorithms (K-Nearest Neighbors, Random Forest, and Support Vector Machines with Gaussian and cubic kernels). We tested the models in the remaining samples (n=40) and assessed model performance by receiver-operating-characteristic (ROC) curves. Real-time quantitative polymerase chain reaction (RT-qPCR) of 9 IA-associated genes was used to verify gene expression in a subset of 49 neutrophil RNA samples. We also examined the potential influence of demographics and comorbidities on model prediction. Results: Feature selection using LASSO in the training cohort identified 37 IA-associated transcripts. Models trained using these transcripts had a maximum accuracy of 90% in the testing cohort. The testing performance across all methods had an average area under ROC curve (AUC)=0.97, an improvement over our previous models. The Random Forest model performed best across both training and testing cohorts. RT-qPCR confirmed expression differences in 7 of 9 genes tested. Gene ontology and IPA network analyses performed on the 37 model genes reflected dysregulated inflammation, cell signaling, and apoptosis processes. In our data, demographics and comorbidities did not affect model performance. Conclusions: We improved upon our previous IA prediction models based on circulating neutrophil transcriptomes by increasing sample size and by implementing LASSO and more robust machine learning methods. Future studies are needed to validate these models in larger cohorts and further investigate effect of covariates.

Download Full-text

Classification Models using Circulating Neutrophil Transcripts Can Detect Unruptured Intracranial Aneurysm

10.21203/rs.3.rs-17161/v1 ◽

2020 ◽

Author(s):

Kerry E Poppenberg ◽

Vincent M Tutino ◽

Lu Li ◽

Muhammad Waqas ◽

Armond June ◽

...

Keyword(s):

Machine Learning ◽

Random Forest ◽

Prediction Models ◽

Model Performance ◽

Supervised Machine Learning ◽

Support Vector ◽

Learning Methods ◽

Training Cohort ◽

Network Analyses ◽

Machine Learning Methods

Abstract Background Intracranial aneurysms (IAs) are dangerous because of their potential to rupture. We previously found significant RNA expression differences in circulating neutrophils between patients with and without unruptured IAs and trained machine learning models to predict presence of IA using 40 neutrophil transcriptomes. Here, we aim to develop a predictive model for unruptured IA using neutrophil transcriptomes from a larger population and more robust machine learning methods. Methods Neutrophil RNA extracted from the blood of 134 patients (55 with IA, 79 IA-free controls) was subjected to next-generation RNA sequencing. In a randomly-selected training cohort (n=94), the Least Absolute Shrinkage and Selection Operator (LASSO) selected transcripts, from which we constructed prediction models via 4 well-established supervised machine-learning algorithms (K-Nearest Neighbors, Random Forest, and Support Vector Machines with Gaussian and cubic kernels). We tested the models in the remaining samples (n=40) and assessed model performance by receiver-operating-characteristic (ROC) curves. Real-time quantitative polymerase chain reaction (RT-qPCR) of 10 IA-associated genes was used to verify gene expression in a subset of 50 neutrophil RNA samples. We also examined the potential influence of demographics and comorbidities on model prediction. Results Feature selection using LASSO in the training cohort identified 37 IA-associated transcripts. Models trained using these transcripts had a maximum accuracy of 90% in the testing cohort. The testing performance across all methods had an average area under ROC curve (AUC)=0.97, an improvement over our previous models. The Random Forest model performed best across both training and testing cohorts. RT-qPCR confirmed expression differences in 8 of 10 genes tested. Gene ontology and IPA network analyses performed on the 37 model genes reflected dysregulated inflammation, cell signaling, and apoptosis processes. In our data, demographics and comorbidities did not affect model performance. Conclusions We improved upon our previous IA prediction models based on circulating neutrophil transcriptomes by increasing sample size and by implementing LASSO and more robust machine learning methods. Future studies are needed to validate these models in larger cohorts and further investigate effect of covariates.

Download Full-text

Predictors of Purchasing a Hearing Aid After an Evaluation Period: A Prospective Study in Dutch Older Hearing Aid Candidates

American Journal of Audiology ◽

10.1044/2019_aja-heal18-18-0163 ◽

2019 ◽

Vol 28 (3S) ◽

pp. 802-805 ◽

Cited By ~ 1

Author(s):

Marieke Pronk ◽

Janine F. J. Meijerink ◽

Sophia E. Kramer ◽

Martijn W. Heymans ◽

Jana Besser

Keyword(s):

Help Seeking ◽

Hearing Aid ◽

Model Performance ◽

Area Under The Curve ◽

Secondary Data ◽

Support Program ◽

Evaluation Period ◽

A Prospective Study ◽

Study Participants ◽

Baseline Covariates

Purpose The current study aimed to identify factors that distinguish between older (50+ years) hearing aid (HA) candidates who do and do not purchase HAs after having gone through an HA evaluation period (HAEP). Method Secondary data analysis of the SUpport PRogram trial was performed ( n = 267 older, 1st-time HA candidates). All SUpport PRogram participants started an HAEP shortly after study enrollment. Decision to purchase an HA by the end of the HAEP was the outcome of interest of the current study. Participants' baseline covariates (22 in total) were included as candidate predictors. Multivariable logistic regression modeling (backward selection and reclassification tables) was used. Results Of all candidate predictors, only pure-tone average (average of 1, 2, and 4 kHz) hearing loss emerged as a significant predictor (odds ratio = 1.03, 95% confidence interval [1.03, 1.17]). Model performance was weak (Nagelkerke R 2 = .04, area under the curve = 0.61). Conclusions These data suggest that, once HA candidates have decided to enter an HAEP, factors measured early in the help-seeking journey do not predict well who will and will not purchase an HA. Instead, factors that act during the HAEP may hold this predictive value. This should be examined.

Download Full-text

DESIGN AN “IN HOUSE” SOFTWARE FOR SCREENING PREECLAMPSIA BY MATERNAL FACTORS AND MEAN ARTERIAL PRESSURE AT 11 – 13 WEEKS IN COMMUNE HEALTH CENTERS.

Journal of Medicine and Pharmacy ◽

10.34071/jmp.2015.3.16 ◽

2015 ◽

pp. 115-126

Author(s):

Viet Nhan Nguyen ◽

Ngoc Thanh Cao ◽

Thi Minh Thi Ha ◽

Van Duc Vo ◽

Quang Vinh Truong ◽

...

Keyword(s):

Mean Arterial Pressure ◽

Arterial Pressure ◽

Gestational Hypertension ◽

Area Under The Curve ◽

Health Centers ◽

Maternal Factors ◽

Good Tool ◽

Uterine Artery Doppler ◽

Study Results ◽

In Pregnancy

Objective: Design an “in house” software for screening preeclampsia by maternal factors and mean arterial pressure at 11 – 13 gestational weeks in commune health centers. Methods: Based on the algorithms for calculating the risk of preeclampsia (PE) by maternal factors and mean artirial pressure at 11 - 13 gestational weeks in the study results of the authors, an “in house” software was deigned in Excel. The results of prediction preeclampsia by The Fetal Medicine Foundation (FMF)(version 2.3) were compared with the results by “in house” software in 1110 singleton pregnant women. Results: The “in house” software met the requirements for calculating the risks of PE and save data. FMF risk for gestational hypertension disorder in pregnancy by maternal factors, mean arterial pressure,uterine artery Doppler and PAPP-A has an area under the curve of 0.68 (95%CI: 0.59 – 0.78). The “in house” software risk for gestational hypertension in pregnancy by maternal factors, mean arterial pressure has an area under the curve of 0.643 (0.55 – 0.73) There was no statistically significant different between two programs (p:0.52). The risk cut-off 1:50 in the prediction of gestational hypertension of the “in house” software was used to identify the group of high risk with detetion rate (DR) 28.6% (95%CI: 14.9-42.2) comparing to 40.5% (95%CI:25.6-55.3) of FMF. Conclusion: The FMF version 2.3 is better but in the absence of Doppler ultrasound and PAPP-A test in the commune health cares, the “in house” software for screening PE is a good tool for councelling, following up and early intervention for PE.

Download Full-text