A new risk stratification score for patients with suspected cardiac chest pain in emergency departments, based on machine learning

Abstract Background Clinical risk scores and machine learning models based on routine laboratory values could assist in automated early identification of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) patients at risk for severe clinical outcomes. They can guide patient triage, inform allocation of health care resources, and contribute to the improvement of clinical outcomes. Methods In- and out-patients tested positive for SARS-CoV-2 at the Insel Hospital Group Bern, Switzerland, between February 1st and August 31st (‘first wave’, n = 198) and September 1st through November 16th 2020 (‘second wave’, n = 459) were used as training and prospective validation cohort, respectively. A clinical risk stratification score and machine learning (ML) models were developed using demographic data, medical history, and laboratory values taken up to 3 days before, or 1 day after, positive testing to predict severe outcomes of hospitalization (a composite endpoint of admission to intensive care, or death from any cause). Test accuracy was assessed using the area under the receiver operating characteristic curve (AUROC). Results Sex, C-reactive protein, sodium, hemoglobin, glomerular filtration rate, glucose, and leucocytes around the time of first positive testing (− 3 to + 1 days) were the most predictive parameters. AUROC of the risk stratification score on training data (AUROC = 0.94, positive predictive value (PPV) = 0.97, negative predictive value (NPV) = 0.80) were comparable to the prospective validation cohort (AUROC = 0.85, PPV = 0.91, NPV = 0.81). The most successful ML algorithm with respect to AUROC was support vector machines (median = 0.96, interquartile range = 0.85–0.99, PPV = 0.90, NPV = 0.58). Conclusion With a small set of easily obtainable parameters, both the clinical risk stratification score and the ML models were predictive for severe outcomes at our tertiary hospital center, and performed well in prospective validation.

Download Full-text

Risk stratification for prediction of adverse coronary events in emergency department chest pain patients with a machine learning score compared with the TIMI score

International Journal of Cardiology ◽

10.1016/j.ijcard.2014.09.199 ◽

2014 ◽

Vol 177 (3) ◽

pp. 1095-1097 ◽

Cited By ~ 6

Author(s):

Nan Liu ◽

Marcus Aik Beng Lee ◽

Andrew Fu Wah Ho ◽

Benjamin Haaland ◽

Stephanie Fook-Chong ◽

...

Keyword(s):

Machine Learning ◽

Emergency Department ◽

Chest Pain ◽

Risk Stratification ◽

Timi Score ◽

Coronary Events ◽

Learning Score ◽

Pain Patients

Download Full-text

Machine Learning Dimensionality Reduction Showed Marginal Performance Benefit Over Stepwise Regression for Risk Stratification of Chest Pain Patients in the Emergency Department

10.21203/rs.3.rs-43703/v1 ◽

2020 ◽

Author(s):

Nan Liu ◽

Marcel Lucas Chee ◽

Zhi Xiong Koh ◽

Su Li Leow ◽

Andrew Fu Wah Ho ◽

...

Keyword(s):

Machine Learning ◽

Logistic Regression ◽

Chest Pain ◽

Risk Stratification ◽

Dimensionality Reduction ◽

Prediction Models ◽

Stepwise Logistic Regression ◽

Stepwise Variable Selection ◽

Reduction Methods ◽

Pain Patients

Abstract Background: Chest pain is among the most common presenting complaints in the emergency department (ED). Swift and accurate risk stratification of chest pain patients in the ED may improve patient outcomes and reduce unnecessary costs. Traditional logistic regression with stepwise variable selection has been used to build risk prediction models for ED chest pain patients. In this study, we aimed to investigate if machine learning dimensionality reduction methods can achieve superior performance than the stepwise approach in deriving risk stratification models. Methods: A retrospective analysis was conducted on the data of patients >20 years old who presented to the ED of Singapore General Hospital with chest pain between September 2010 and July 2015. Variables used included demographics, medical history, laboratory findings, heart rate variability (HRV), and HRnV parameters calculated from five to six-minute electrocardiograms (ECGs). The primary outcome was 30-day major adverse cardiac events (MACE), which included death, acute myocardial infarction, and revascularization. Candidate variables identified using univariable analysis were then used to generate the stepwise logistic regression model and eight machine learning dimensionality reduction prediction models. A separate set of models was derived by excluding troponin. Receiver operating characteristic (ROC) and calibration analysis was used to compare model performance.Results: 795 patients were included in the analysis, of which 247 (31%) met the primary outcome of 30-day MACE. Patients with MACE were older and more likely to be male. All eight dimensionality reduction methods marginally but non-significantly outperformed stepwise variable selection; The multidimensional scaling algorithm performed the best with an area under the curve (AUC) of 0.901. All HRnV-based models generated in this study outperformed several existing clinical scores in ROC analysis.Conclusions: HRnV-based models using stepwise logistic regression performed better than existing chest pain scores for predicting MACE, with only marginal improvements using machine learning dimensionality reduction. Moreover, traditional stepwise approach benefits from model transparency and interpretability; in comparison, machine learning dimensionality reduction models are black boxes, making them difficult to explain in clinical practice.

Download Full-text

Utilizing machine learning dimensionality reduction for risk stratification of chest pain patients in the emergency department

BMC Medical Research Methodology ◽

10.1186/s12874-021-01265-2 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Nan Liu ◽

Marcel Lucas Chee ◽

Zhi Xiong Koh ◽

Su Li Leow ◽

Andrew Fu Wah Ho ◽

...

Keyword(s):

Machine Learning ◽

Emergency Department ◽

Chest Pain ◽

Risk Stratification ◽

Dimensionality Reduction ◽

Primary Outcome ◽

Prediction Models ◽

Stepwise Variable Selection ◽

Reduction Methods ◽

Pain Patients

Abstract Background Chest pain is among the most common presenting complaints in the emergency department (ED). Swift and accurate risk stratification of chest pain patients in the ED may improve patient outcomes and reduce unnecessary costs. Traditional logistic regression with stepwise variable selection has been used to build risk prediction models for ED chest pain patients. In this study, we aimed to investigate if machine learning dimensionality reduction methods can improve performance in deriving risk stratification models. Methods A retrospective analysis was conducted on the data of patients > 20 years old who presented to the ED of Singapore General Hospital with chest pain between September 2010 and July 2015. Variables used included demographics, medical history, laboratory findings, heart rate variability (HRV), and heart rate n-variability (HRnV) parameters calculated from five to six-minute electrocardiograms (ECGs). The primary outcome was 30-day major adverse cardiac events (MACE), which included death, acute myocardial infarction, and revascularization within 30 days of ED presentation. We used eight machine learning dimensionality reduction methods and logistic regression to create different prediction models. We further excluded cardiac troponin from candidate variables and derived a separate set of models to evaluate the performance of models without using laboratory tests. Receiver operating characteristic (ROC) and calibration analysis was used to compare model performance. Results Seven hundred ninety-five patients were included in the analysis, of which 247 (31%) met the primary outcome of 30-day MACE. Patients with MACE were older and more likely to be male. All eight dimensionality reduction methods achieved comparable performance with the traditional stepwise variable selection; The multidimensional scaling algorithm performed the best with an area under the curve of 0.901. All prediction models generated in this study outperformed several existing clinical scores in ROC analysis. Conclusions Dimensionality reduction models showed marginal value in improving the prediction of 30-day MACE for ED chest pain patients. Moreover, they are black box models, making them difficult to explain and interpret in clinical practice.

Download Full-text

Applications of machine learning to undifferentiated chest pain in the emergency department: A systematic review

PLoS ONE ◽

10.1371/journal.pone.0252612 ◽

2021 ◽

Vol 16 (8) ◽

pp. e0252612

Author(s):

Jonathon Stewart ◽

Juan Lu ◽

Adrian Goudie ◽

Mohammed Bennamoun ◽

Peter Sprivulis ◽

...

Keyword(s):

Machine Learning ◽

Myocardial Infarction ◽

Systematic Review ◽

Acute Myocardial Infarction ◽

Emergency Department ◽

Chest Pain ◽

Risk Stratification ◽

Clinical Decision ◽

Risk Of Bias ◽

Current Risk

Background Chest pain is amongst the most common reason for presentation to the emergency department (ED). There are many causes of chest pain, and it is important for the emergency physician to quickly and accurately diagnose life threatening causes such as acute myocardial infarction (AMI). Multiple clinical decision tools have been developed to assist clinicians in risk stratifying patients with chest. There is growing recognition that machine learning (ML) will have a significant impact on the practice of medicine in the near future and may assist with diagnosis and risk stratification. This systematic review aims to evaluate how ML has been applied to adults presenting to the ED with undifferentiated chest pain and assess if ML models show improved performance when compared to physicians or current risk stratification techniques. Methods and findings We conducted a systematic review of journal articles that applied a ML technique to an adult patient presenting to an emergency department with undifferentiated chest pain. Multiple databases were searched from inception through to November 2020. In total, 3361 articles were screened, and 23 articles were included. We did not conduct a metanalysis due to a high level of heterogeneity between studies in both their methods, and reporting. The most common primary outcomes assessed were diagnosis of acute myocardial infarction (AMI) (12 studies), and prognosis of major adverse cardiovascular event (MACE) (6 studies). There were 14 retrospective studies and 5 prospective studies. Four studies reported the development of a machine learning model retrospectively then tested it prospectively. The most common machine learning methods used were artificial neural networks (14 studies), random forest (6 studies), support vector machine (5 studies), and gradient boosting (2 studies). Multiple studies achieved high accuracy in both the diagnosis of AMI in the ED setting, and in predicting mortality and composite outcomes over various timeframes. ML outperformed existing risk stratification scores in all cases, and physicians in three out of four cases. The majority of studies were single centre, retrospective, and without prospective or external validation. There were only 3 studies that were considered low risk of bias and had low applicability concerns. Two studies reported integrating the ML model into clinical practice. Conclusions Research on applications of ML for undifferentiated chest pain in the ED has been ongoing for decades. ML has been reported to outperform emergency physicians and current risk stratification tools to diagnose AMI and predict MACE but has rarely been integrated into practice. Many studies assessing the use of ML in undifferentiated chest pain in the ED have a high risk of bias. It is important that future studies make use of recently developed standardised ML reporting guidelines, register their protocols, and share their datasets and code. Future work is required to assess the impact of ML model implementation on clinical decision making, patient orientated outcomes, and patient and physician acceptability. Trial registration International Prospective Register of Systematic Reviews registration number: CRD42020184977.

Download Full-text

Machine learning dimensionality reduction for heart rate n-variability (HRnV) based risk stratification of chest pain patients in the emergency department

10.1101/2020.07.05.20146571 ◽

2020 ◽

Author(s):

Nan Liu ◽

Marcel Lucas Chee ◽

Zhi Xiong Koh ◽

Su Li Leow ◽

Andrew Fu Wah Ho ◽

...

Keyword(s):

Machine Learning ◽

Logistic Regression ◽

Chest Pain ◽

Risk Stratification ◽

Dimensionality Reduction ◽

Prediction Models ◽

Stepwise Logistic Regression ◽

Stepwise Variable Selection ◽

Reduction Methods ◽

Pain Patients

AbstractBackgroundChest pain is among the most common presenting complaints in the emergency department (ED). Swift and accurate risk stratification of chest pain patients in the ED may improve patient outcomes and reduce unnecessary costs. Traditional logistic regression with stepwise variable selection has been used to build risk prediction models for ED chest pain patients. In this study, we aimed to investigate if machine learning dimensionality reduction methods can achieve superior performance than the stepwise approach in deriving risk stratification models.MethodsA retrospective analysis was conducted on the data of patients >20 years old who presented to the ED of Singapore General Hospital with chest pain between September 2010 and July 2015. Variables used included demographics, medical history, laboratory findings, heart rate variability (HRV), and HRnV parameters calculated from five to six-minute electrocardiograms (ECGs). The primary outcome was 30-day major adverse cardiac events (MACE), which included death, acute myocardial infarction, and revascularization. Candidate variables identified using univariable analysis were then used to generate the stepwise logistic regression model and eight machine learning dimensionality reduction prediction models. A separate set of models was derived by excluding troponin. Receiver operating characteristic (ROC) and calibration analysis was used to compare model performance.Results795 patients were included in the analysis, of which 247 (31%) met the primary outcome of 30-day MACE. Patients with MACE were older and more likely to be male. All eight dimensionality reduction methods marginally but non-significantly outperformed stepwise variable selection; The multidimensional scaling algorithm performed the best with an area under the curve (AUC) of 0.901. All HRnV-based models generated in this study outperformed several existing clinical scores in ROC analysis.ConclusionsHRnV-based models using stepwise logistic regression performed better than existing chest pain scores for predicting MACE, with only marginal improvements using machine learning dimensionality reduction. Moreover, traditional stepwise approach benefits from model transparency and interpretability; in comparison, machine learning dimensionality reduction models are black boxes, making them difficult to explain in clinical practice.

Download Full-text

Development and validation of a prognostic COVID-19 severity assessment (COSA) score and machine learning models for patient triage at a tertiary hospital

10.21203/rs.3.rs-165301/v1 ◽

2021 ◽

Author(s):

Verena Schöning ◽

Evangelia Liakoni ◽

Christine Baumgartner ◽

Aristomenis K. Exadaktylos ◽

Wolf E. Hautz ◽

...

Keyword(s):

Machine Learning ◽

Risk Stratification ◽

Clinical Outcomes ◽

Predictive Value ◽

Validation Cohort ◽

Tertiary Hospital ◽

Learning Models ◽

Clinical Risk ◽

Machine Learning Models ◽

Risk Stratification Score

Abstract Background: Clinical risk scores and machine learning models based on routine laboratory values could assist in automated early identification of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) patients at risk for severe clinical outcomes. They can guide patient triage, inform allocation of health care resources, and contribute to the improvement of clinical outcomes. Methods: In- and out-patients tested positive for SARS-CoV-2 at the Insel Hospital Group Bern, Switzerland, between February 1st and August 31st (‘first wave’, n=198) and September 1st through November 16th 2020 (‘second wave’, n=459) were used as training and prospective validation cohort, respectively. A clinical risk stratification score and machine learning (ML) models were developed using demographic data, medical history, and laboratory values taken up to three days before, or one day after, positive testing to predict severe outcomes of hospitalization (a composite endpoint of admission to intensive care, or death from any cause). Test accuracy was assessed using the area under the receiver operating characteristic curve (AUROC).Results: Sex, C-reactive protein, sodium, hemoglobin, glomerular filtration rate, glucose, and leucocytes around the time of first positive testing (‑3 to +1 days) were the most predictive parameters. AUROC of the risk stratification score on training data (AUROC = 0.94, positive predictive value (PPV) = 0.97, negative predictive value (NPV) = 0.80) were comparable to the prospective validation cohort (AUROC = 0.85, PPV = 0.91, NPV = 0.81). The most successful ML algorithm with respect to AUROC was support vector machines (median = 0.96, interquartile range = 0.85-0.99, PPV = 0.90, NPV = 0.58).Conclusion: With a small set of easily obtainable parameters, both the clinical risk stratification score and the ML models were predictive for severe outcomes at our tertiary hospital center, and performed well in prospective validation.

Download Full-text

Stress Echocardiography for Risk Stratification in Women With Chest Pain

Journal of the American College of Cardiology ◽

10.1016/s0735-1097(97)84607-4 ◽

1998 ◽

Vol 31 (2) ◽

pp. 208A

Author(s):

L Cortigiani

Keyword(s):

Chest Pain ◽

Risk Stratification ◽

Stress Echocardiography

Download Full-text

Risk stratification scores for hospitalization duration and disease progression in moderate and severe patients with COVID-19

BMC Pulmonary Medicine ◽

10.1186/s12890-021-01487-6 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Jiaqi Huang ◽

Yu Xu ◽

Bin Wang ◽

Ying Xiang ◽

Na Wu ◽

...

Keyword(s):

Platelet Count ◽

Lactate Dehydrogenase ◽

Risk Stratification ◽

Disease Progression ◽

Risk Score ◽

Healthcare Providers ◽

Maximum Body ◽

Score Systems ◽

Risk Categories ◽

Risk Stratification Score

Abstract Background During outbreak of Coronavirus Disease 2019 (COVID-19), healthcare providers are facing critical clinical decisions based on the prognosis of patients. Decision support tools of risk stratification are needed to predict outcomes in patients with different clinical types of COVID-19. Methods This retrospective cohort study recruited 2425 patients with moderate or severe COVID-19. A logistic regression model was used to select and estimate the factors independently associated with outcomes. Simplified risk stratification score systems were constructed to predict outcomes in moderate and severe patients with COVID-19, and their performances were evaluated by discrimination and calibration. Results We constructed two risk stratification score systems, named as STPCAL (including significant factors in the prediction model: number of clinical symptoms, the maximum body temperature during hospitalization, platelet count, C-reactive protein, albumin and lactate dehydrogenase) and TRPNCLP (including maximum body temperature during hospitalization, history of respiratory diseases, platelet count, neutrophil-to-lymphocyte ratio, creatinine, lactate dehydrogenase, and prothrombin time), to predict hospitalization duration for moderate patients and disease progression for severe patients, respectively. According to STPCAL score, moderate patients were classified into three risk categories for a longer hospital duration: low (Score 0–1, median = 8 days, with less than 20.0% probabilities), intermediate (Score 2–6, median = 13 days, with 30.0–78.9% probabilities), high (Score 7–9, median = 19 days, with more than 86.5% probabilities). Severe patients were stratified into three risk categories for disease progression: low risk (Score 0–5, with less than 12.7% probabilities), intermediate risk (Score 6–11, with 18.6–69.1% probabilities), and high risk (Score 12–16, with more than 77.9% probabilities) by TRPNCLP score. The two risk scores performed well with good discrimination and calibration. Conclusions Two easy-to-use risk stratification score systems were built to predict the outcomes in COVID-19 patients with different clinical types. Identifying high risk patients with longer stay or poor prognosis could assist healthcare providers in triaging patients when allocating limited healthcare during COVID-19 outbreak.

Download Full-text