Prediction of Cardiac Arrest in the Emergency Department Based on Machine Learning and Sequential Characteristics: Model Development and Retrospective Clinical Validation Study (Preprint)

BACKGROUND The development and application of clinical prediction models using machine learning in clinical decision support systems is attracting increasing attention. OBJECTIVE The aims of this study were to develop a prediction model for cardiac arrest in the emergency department (ED) using machine learning and sequential characteristics and to validate its clinical usefulness. METHODS This retrospective study was conducted with ED patients at a tertiary academic hospital who suffered cardiac arrest. To resolve the class imbalance problem, sampling was performed using propensity score matching. The data set was chronologically allocated to a development cohort (years 2013 to 2016) and a validation cohort (year 2017). We trained three machine learning algorithms with repeated 10-fold cross-validation. RESULTS The main performance parameters were the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). The random forest algorithm (AUROC 0.97; AUPRC 0.86) outperformed the recurrent neural network (AUROC 0.95; AUPRC 0.82) and the logistic regression algorithm (AUROC 0.92; AUPRC=0.72). The performance of the model was maintained over time, with the AUROC remaining at least 80% across the monitored time points during the 24 hours before event occurrence. CONCLUSIONS We developed a prediction model of cardiac arrest in the ED using machine learning and sequential characteristics. The model was validated for clinical usefulness by chronological visualization focused on clinical usability.

Download Full-text

Prediction of Cardiac Arrest in the Emergency Department Based on Machine Learning and Sequential Characteristics: Model Development and Retrospective Clinical Validation Study

JMIR Medical Informatics ◽

10.2196/15932 ◽

2020 ◽

Vol 8 (8) ◽

pp. e15932

Author(s):

Sungjun Hong ◽

Sungjoo Lee ◽

Jeonghoon Lee ◽

Won Chul Cha ◽

Kyunga Kim

Keyword(s):

Machine Learning ◽

Emergency Department ◽

Cardiac Arrest ◽

Prediction Model ◽

Prediction Models ◽

Characteristic Curve ◽

Machine Learning Algorithms ◽

Clinical Usefulness ◽

Class Imbalance Problem ◽

Data Set

Background The development and application of clinical prediction models using machine learning in clinical decision support systems is attracting increasing attention. Objective The aims of this study were to develop a prediction model for cardiac arrest in the emergency department (ED) using machine learning and sequential characteristics and to validate its clinical usefulness. Methods This retrospective study was conducted with ED patients at a tertiary academic hospital who suffered cardiac arrest. To resolve the class imbalance problem, sampling was performed using propensity score matching. The data set was chronologically allocated to a development cohort (years 2013 to 2016) and a validation cohort (year 2017). We trained three machine learning algorithms with repeated 10-fold cross-validation. Results The main performance parameters were the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). The random forest algorithm (AUROC 0.97; AUPRC 0.86) outperformed the recurrent neural network (AUROC 0.95; AUPRC 0.82) and the logistic regression algorithm (AUROC 0.92; AUPRC=0.72). The performance of the model was maintained over time, with the AUROC remaining at least 80% across the monitored time points during the 24 hours before event occurrence. Conclusions We developed a prediction model of cardiac arrest in the ED using machine learning and sequential characteristics. The model was validated for clinical usefulness by chronological visualization focused on clinical usability.

Download Full-text

Predicting Loan Approval of Bank Direct Marketing Data Using Ensemble Machine Learning Algorithms

International Journal of Circuits, Systems and Signal Processing ◽

10.46300/9106.2020.14.117 ◽

2020 ◽

Vol 14 ◽

Keyword(s):

Machine Learning ◽

Prediction Model ◽

Prediction Models ◽

Machine Learning Algorithms ◽

Decision Makers ◽

Machine Learning Techniques ◽

Data Set ◽

Ensemble Machine Learning ◽

Marketing Data ◽

Loan Approval

The Bank Marketing data set at Kaggle is mostly used in predicting if bank clients will subscribe a long-term deposit. We believe that this data set could provide more useful information such as predicting whether a bank client could be approved for a loan. This is a critical choice that has to be made by decision makers at the bank. Building a prediction model for such high-stakes decision does not only require high model prediction accuracy, but also needs a reasonable prediction interpretation. In this research, different ensemble machine learning techniques have been deployed such as Bagging and Boosting. Our research results showed that the loan approval prediction model has an accuracy of 83.97%, which is approximately 25% better than most state-of-the-art other loan prediction models found in the literature. As well, the model interpretation efforts done in this research was able to explain a few critical cases that the bank decision makers may encounter; therefore, the high accuracy of the designed models was accompanied with a trust in prediction. We believe that the achieved model accuracy accompanied with the provided interpretation information are vitally needed for decision makers to understand how to maintain balance between security and reliability of their financial lending system, while providing fair credit opportunities to their clients.

Download Full-text

Determination of Significant Features for Building an Efficient Heart Disease Prediction System

International Journal of Recent Technology and Engineering - 2 ◽

10.35940/ijrte.b3393.078219 ◽

2019 ◽

Vol 8 (2) ◽

pp. 4499-4504

Keyword(s):

Machine Learning ◽

Heart Disease ◽

Prediction Model ◽

Prediction Models ◽

Heart Diseases ◽

Medical Diagnostics ◽

Medical Data ◽

Machine Learning Algorithms ◽

Prediction System ◽

Early Stages

Heart diseases are responsible for the greatest number of deaths all over the world. These diseases are usually not detected in early stages as the cost of medical diagnostics is not affordable by a majority of the people. Research has shown that machine learning methods have a great capability to extract valuable information from the medical data. This information is used to build the prediction models which provide cost effective technological aid for a medical practitioner to detect the heart disease in early stages. However, the presence of some irrelevant and redundant features in medical data deteriorates the competence of the prediction system. This research was aimed to improve the accuracy of the existing methods by removing such features. In this study, brute force-based algorithm of feature selection was used to determine relevant significant features. After experimenting rigorously with 7528 possible combinations of features and 5 machine learning algorithms, 8 important features were identified. A prediction model was developed using these significant features. Accuracy of this model is experimentally calculated to be 86.4%which is higher than the results of existing studies. The prediction model proposed in this study shall help in predicting heart disease efficiently.

Download Full-text

Software Maintainability: Systematic Literature Review and Current Trends

International Journal of Software Engineering and Knowledge Engineering ◽

10.1142/s0218194016500431 ◽

2016 ◽

Vol 26 (08) ◽

pp. 1221-1253 ◽

Cited By ~ 16

Author(s):

Ruchika Malhotra ◽

Anuradha Chug

Keyword(s):

Machine Learning ◽

Prediction Model ◽

Evaluation System ◽

Prediction Models ◽

Learning Algorithms ◽

Machine Learning Algorithms ◽

Design Metrics ◽

Current Trends ◽

Software Maintainability ◽

Early Phases

Software maintenance is an expensive activity that consumes a major portion of the cost of the total project. Various activities carried out during maintenance include the addition of new features, deletion of obsolete code, correction of errors, etc. Software maintainability means the ease with which these operations can be carried out. If the maintainability can be measured in early phases of the software development, it helps in better planning and optimum resource utilization. Measurement of design properties such as coupling, cohesion, etc. in early phases of development often leads us to derive the corresponding maintainability with the help of prediction models. In this paper, we performed a systematic review of the existing studies related to software maintainability from January 1991 to October 2015. In total, 96 primary studies were identified out of which 47 studies were from journals, 36 from conference proceedings and 13 from others. All studies were compiled in structured form and analyzed through numerous perspectives such as the use of design metrics, prediction model, tools, data sources, prediction accuracy, etc. According to the review results, we found that the use of machine learning algorithms in predicting maintainability has increased since 2005. The use of evolutionary algorithms has also begun in related sub-fields since 2010. We have observed that design metrics is still the most favored option to capture the characteristics of any given software before deploying it further in prediction model for determining the corresponding software maintainability. A significant increase in the use of public dataset for making the prediction models has also been observed and in this regard two public datasets User Interface Management System (UIMS) and Quality Evaluation System (QUES) proposed by Li and Henry is quite popular among researchers. Although machine learning algorithms are still the most popular methods, however, we suggest that researchers working on software maintainability area should experiment on the use of open source datasets with hybrid algorithms. In this regard, more empirical studies are also required to be conducted on a large number of datasets so that a generalized theory could be made. The current paper will be beneficial for practitioners, researchers and developers as they can use these models and metrics for creating benchmark and standards. Findings of this extensive review would also be useful for novices in the field of software maintainability as it not only provides explicit definitions, but also lays a foundation for further research by providing a quick link to all important studies in the said field. Finally, this study also compiles current trends, emerging sub-fields and identifies various opportunities of future research in the field of software maintainability.

Download Full-text

Sedimentary environment prediction of grain-size data based on machine learning approach

Interpretation ◽

10.1190/int-2019-0153.1 ◽

2020 ◽

Vol 8 (3) ◽

pp. SL71-SL78

Author(s):

Qiao Su ◽

Yanhui Zhu ◽

Fang Hu ◽

Xingyong Xu

Keyword(s):

Machine Learning ◽

Grain Size ◽

Prediction Model ◽

Sedimentary Environment ◽

Machine Learning Algorithms ◽

Grain Size Analysis ◽

Size Analysis ◽

Data Set ◽

Sedimentary Environments ◽

Size Data

Grain size is one of the most important records for sedimentary environment, and researchers have made remarkable progress in the interpretation of sedimentary environments by grain size analysis in the past few decades. However, these advances often depend on the personal experience of the scholars and combination with other methods used together. Here, we constructed a prediction model using the K-nearest neighbors algorithm, one of the machine learning methods, which can predict the sedimentary environments of one core through a known core. Compared to the results of other studies based on the comprehensive data set of grain size and four other indicators, this model achieved a high precision value only using the grain size data. We have also compared our prediction model with other mainstream machine learning algorithms, and the experimental results of six evaluation metrics shed light on that this prediction model can achieve the higher precision. The main errors of the model reflect the length of the conversation area of sedimentary environment, which is controlled by the sedimentary dynamics. This model can provide a quick comparison method of the cores in a similar environment; thus, it may point out the preliminary guidance for further study.

Download Full-text

In silico prediction of chemical neurotoxicity using machine learning

Toxicology Research ◽

10.1093/toxres/tfaa016 ◽

2020 ◽

Vol 9 (3) ◽

pp. 164-172

Author(s):

Changsheng Jiang ◽

Piaopiao Zhao ◽

Weihua Li ◽

Yun Tang ◽

Guixia Liu

Keyword(s):

Machine Learning ◽

Regression Models ◽

Cross Validation ◽

Prediction Models ◽

Drug Withdrawal ◽

Molecular Descriptors ◽

Computational Prediction ◽

Machine Learning Algorithms ◽

Training Set ◽

Data Set

Abstract Neurotoxicity is one of the main causes of drug withdrawal, and the biological experimental methods of detecting neurotoxic toxicity are time-consuming and laborious. In addition, the existing computational prediction models of neurotoxicity still have some shortcomings. In response to these shortcomings, we collected a large number of data set of neurotoxicity and used PyBioMed molecular descriptors and eight machine learning algorithms to construct regression prediction models of chemical neurotoxicity. Through the cross-validation and test set validation of the models, it was found that the extra-trees regressor model had the best predictive effect on neurotoxicity (${q}_{\mathrm{test}}^2$ = 0.784). In addition, we get the applicability domain of the models by calculating the standard deviation distance and the lever distance of the training set. We also found that some molecular descriptors are closely related to neurotoxicity by calculating the contribution of the molecular descriptors to the models. Considering the accuracy of the regression models, we recommend using the extra-trees regressor model to predict the chemical autonomic neurotoxicity.

Download Full-text

An Ensemble Prediction Model for Potential Student Recommendation Using Machine Learning

Symmetry ◽

10.3390/sym12050728 ◽

2020 ◽

Vol 12 (5) ◽

pp. 728 ◽

Cited By ~ 2

Author(s):

Lijuan Yan ◽

Yanshen Liu

Keyword(s):

Machine Learning ◽

Student Performance ◽

Prediction Models ◽

Characteristic Curve ◽

Machine Learning Algorithms ◽

Ensemble Prediction ◽

Support Vector ◽

Proposed Model ◽

Importance Analysis ◽

Better Than

Student performance prediction has become a hot research topic. Most of the existing prediction models are built by a machine learning method. They are interested in prediction accuracy but pay less attention to interpretability. We propose a stacking ensemble model to predict and analyze student performance in academic competition. In this model, student performance is classified into two symmetrical categorical classes. To improve accuracy, three machine learning algorithms, including support vector machine (SVM), random forest, and AdaBoost are established in the first level and then integrated by logistic regression via stacking. A feature importance analysis was applied to identify important variables. The experimental data were collected from four academic years in Hankou University. According to comparative studies on five evaluation metrics (precision, recall, F1, error, and area under the receiver operating characteristic curve ( AUC ) in this analysis, the proposed model generally performs better than compared models. The important variables identified from the analysis are interpretable, they can be used as guidance to select potential students.

Download Full-text

Comparing predictions made by a prediction model, clinical score, and physicians

Applied Clinical Informatics ◽

10.4338/aci-2013-04-ra-0029 ◽

2013 ◽

Vol 04 (03) ◽

pp. 376-391 ◽

Cited By ~ 19

Author(s):

K.J. Farion ◽

W. Michalowski ◽

D. O’Sullivan ◽

J. Sayyad-Shirabad ◽

S. Wilk

Keyword(s):

Prediction Model ◽

Prediction Models ◽

Phase 1 ◽

Characteristic Curve ◽

Clinical Score ◽

Machine Learning Techniques ◽

Phase 2 ◽

Two Phase ◽

Data Set ◽

Predictive Values

SummaryBackground: Asthma exacerbations are one of the most common medical reasons for children to be brought to the hospital emergency department (ED). Various prediction models have been proposed to support diagnosis of exacerbations and evaluation of their severity.Objectives: First, to evaluate prediction models constructed from data using machine learning techniques and to select the best performing model. Second, to compare predictions from the selected model with predictions from the Pediatric Respiratory Assessment Measure (PRAM) score, and predictions made by ED physicians.Design: A two-phase study conducted in the ED of an academic pediatric hospital. In phase 1 data collected prospectively using paper forms was used to construct and evaluate five prediction models, and the best performing model was selected. In phase 2 data collected prospectively using a mobile system was used to compare the predictions of the selected prediction model with those from PRAM and ED physicians.Measurements: Area under the receiver operating characteristic curve and accuracy in phase 1; accuracy, sensitivity, specificity, positive and negative predictive values in phase 2.Results: In phase 1 prediction models were derived from a data set of 240 patients and evaluated using 10-fold cross validation. A naive Bayes (NB) model demonstrated the best performance and it was selected for phase 2. Evaluation in phase 2 was conducted on data from 82 patients. Predictions made by the NB model were less accurate than the PRAM score and physicians (accuracy of 70.7%, 73.2% and 78.0% respectively), however, according to McNemar’s test it is not possible to conclude that the differences between predictions are statistically significant.Conclusion: Both the PRAM score and the NB model were less accurate than physicians. The NB model can handle incomplete patient data and as such may complement the PRAM score. However, it requires further research to improve its accuracy.

Download Full-text

Prediction of Type 2 Diabetes Based on Machine Learning Algorithm

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18063317 ◽

2021 ◽

Vol 18 (6) ◽

pp. 3317

Author(s):

Henock M. Deberneh ◽

Intaek Kim

Keyword(s):

Machine Learning ◽

Type 2 Diabetes ◽

Prediction Model ◽

Prediction Models ◽

Machine Learning Algorithms ◽

Superior Performance ◽

Recursive Feature Elimination ◽

Medical Institute ◽

Support Vector

Prediction of type 2 diabetes (T2D) occurrence allows a person at risk to take actions that can prevent onset or delay the progression of the disease. In this study, we developed a machine learning (ML) model to predict T2D occurrence in the following year (Y + 1) using variables in the current year (Y). The dataset for this study was collected at a private medical institute as electronic health records from 2013 to 2018. To construct the prediction model, key features were first selected using ANOVA tests, chi-squared tests, and recursive feature elimination methods. The resultant features were fasting plasma glucose (FPG), HbA1c, triglycerides, BMI, gamma-GTP, age, uric acid, sex, smoking, drinking, physical activity, and family history. We then employed logistic regression, random forest, support vector machine, XGBoost, and ensemble machine learning algorithms based on these variables to predict the outcome as normal (non-diabetic), prediabetes, or diabetes. Based on the experimental results, the performance of the prediction model proved to be reasonably good at forecasting the occurrence of T2D in the Korean population. The model can provide clinicians and patients with valuable predictive information on the likelihood of developing T2D. The cross-validation (CV) results showed that the ensemble models had a superior performance to that of the single models. The CV performance of the prediction models was improved by incorporating more medical history from the dataset.

Download Full-text

Machine Learning Algorithms to Predict In-Hospital Mortality in Patients with Traumatic Brain Injury

Journal of Personalized Medicine ◽

10.3390/jpm11111144 ◽

2021 ◽

Vol 11 (11) ◽

pp. 1144

Author(s):

Sheng-Der Hsu ◽

En Chao ◽

Sy-Jou Chen ◽

Dueng-Yuan Hueng ◽

Hsiang-Yun Lan ◽

...

Keyword(s):

Machine Learning ◽

Traumatic Brain Injury ◽

Emergency Department ◽

Brain Injury ◽

High Risk ◽

Hospital Mortality ◽

Prediction Models ◽

Care Delivery ◽

Machine Learning Algorithms ◽

Clinical Measures

Traumatic brain injury (TBI) can lead to severe adverse clinical outcomes, including death and disability. Early detection of in-hospital mortality in high-risk populations may enable early treatment and potentially reduce mortality using machine learning. However, there is limited information on in-hospital mortality prediction models for TBI patients admitted to emergency departments. The aim of this study was to create a model that successfully predicts, from clinical measures and demographics, in-hospital mortality in a sample of TBI patients admitted to the emergency department. Of the 4881 TBI patients who were screened at the emergency department at a high-level first aid duty hospital in northern Taiwan, 3331 were assigned in triage to Level I or Level II using the Taiwan Triage and Acuity Scale from January 2008 to June 2018. The most significant predictors of in-hospital mortality in TBI patients were the scores on the Glasgow coma scale, the injury severity scale, and systolic blood pressure in the emergency department admission. This study demonstrated the effective cutoff values for clinical measures when using machine learning to predict in-hospital mortality of patients with TBI. The prediction model has the potential to further accelerate the development of innovative care-delivery protocols for high-risk patients.

Download Full-text