scholarly journals Generalizability of an Automatic Explanation Method for Machine Learning Prediction Results on Asthma-Related Hospital Visits in Patients With Asthma: Quantitative Analysis (Preprint)

2020 ◽  
Author(s):  
Gang Luo ◽  
Claudia L Nau ◽  
William W Crawford ◽  
Michael Schatz ◽  
Robert S Zeiger ◽  
...  
Keyword(s):  
Machine Learning ◽  
Health Care ◽  
Health Care Systems ◽  
Asthma Management ◽  
Secondary Analysis ◽  
Emergency Department Visits ◽  
Learning Approaches ◽  
Care Systems ◽  
Hospital Visits ◽  
Intermountain Healthcare

BACKGROUND Asthma exerts a substantial burden on patients and health care systems. To facilitate preventive care for asthma management and improve patient outcomes, we recently developed two machine learning models, one on Intermountain Healthcare data and the other on Kaiser Permanente Southern California (KPSC) data, to forecast asthma-related hospital visits, including emergency department visits and hospitalizations, in the succeeding 12 months among patients with asthma. As is typical for machine learning approaches, these two models do not explain their forecasting results. To address the interpretability issue of black-box models, we designed an automatic method to offer rule format explanations for the forecasting results of any machine learning model on imbalanced tabular data and to suggest customized interventions with no accuracy loss. Our method worked well for explaining the forecasting results of our Intermountain Healthcare model, but its generalizability to other health care systems remains unknown. OBJECTIVE The objective of this study is to evaluate the generalizability of our automatic explanation method to KPSC for forecasting asthma-related hospital visits. METHODS Through a secondary analysis of 987,506 data instances from 2012 to 2017 at KPSC, we used our method to explain the forecasting results of our KPSC model and to suggest customized interventions. The patient cohort covered a random sample of 70% of patients with asthma who had a KPSC health plan for any period between 2015 and 2018. RESULTS Our method explained the forecasting results for 97.57% (2204/2259) of the patients with asthma who were correctly forecasted to undergo asthma-related hospital visits in the succeeding 12 months. CONCLUSIONS For forecasting asthma-related hospital visits, our automatic explanation method exhibited an acceptable generalizability to KPSC. CLINICALTRIAL INTERNATIONAL REGISTERED REPORT RR2-10.2196/resprot.5039

scholarly journals Generalizability of an Automatic Explanation Method for Machine Learning Prediction Results on Asthma-Related Hospital Visits in Patients With Asthma: Quantitative Analysis

10.2196/24153 ◽  
2021 ◽  
Vol 23 (4) ◽  
pp. e24153
Author(s):  
Gang Luo ◽  
Claudia L Nau ◽  
William W Crawford ◽  
Michael Schatz ◽  
Robert S Zeiger ◽  
...  
Keyword(s):  
Machine Learning ◽  
Health Care ◽  
Health Care Systems ◽  
Asthma Management ◽  
Secondary Analysis ◽  
Emergency Department Visits ◽  
Learning Approaches ◽  
Care Systems ◽  
Hospital Visits ◽  
Intermountain Healthcare

Background Asthma exerts a substantial burden on patients and health care systems. To facilitate preventive care for asthma management and improve patient outcomes, we recently developed two machine learning models, one on Intermountain Healthcare data and the other on Kaiser Permanente Southern California (KPSC) data, to forecast asthma-related hospital visits, including emergency department visits and hospitalizations, in the succeeding 12 months among patients with asthma. As is typical for machine learning approaches, these two models do not explain their forecasting results. To address the interpretability issue of black-box models, we designed an automatic method to offer rule format explanations for the forecasting results of any machine learning model on imbalanced tabular data and to suggest customized interventions with no accuracy loss. Our method worked well for explaining the forecasting results of our Intermountain Healthcare model, but its generalizability to other health care systems remains unknown. Objective The objective of this study is to evaluate the generalizability of our automatic explanation method to KPSC for forecasting asthma-related hospital visits. Methods Through a secondary analysis of 987,506 data instances from 2012 to 2017 at KPSC, we used our method to explain the forecasting results of our KPSC model and to suggest customized interventions. The patient cohort covered a random sample of 70% of patients with asthma who had a KPSC health plan for any period between 2015 and 2018. Results Our method explained the forecasting results for 97.57% (2204/2259) of the patients with asthma who were correctly forecasted to undergo asthma-related hospital visits in the succeeding 12 months. Conclusions For forecasting asthma-related hospital visits, our automatic explanation method exhibited an acceptable generalizability to KPSC. International Registered Report Identifier (IRRID) RR2-10.2196/resprot.5039

scholarly journals A Comprehensive Study of Artificial Intelligence and Machine Learning Approaches in Confronting the Coronavirus (COVID-19) Pandemic

2021 ◽  
pp. 002073142110174
Author(s):  
Md Mijanur Rahman ◽  
Fatema Khatun ◽  
Ashik Uzzaman ◽  
Sadia Islam Sami ◽  
Md Al-Amin Bhuiyan ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Health Care ◽  
Language Processing ◽  
Probabilistic Models ◽  
Health Care Systems ◽  
Learning Approaches ◽  
Care Systems ◽  
Novel Coronavirus ◽  
Comprehensive Study

The novel coronavirus disease (COVID-19) has spread over 219 countries of the globe as a pandemic, creating alarming impacts on health care, socioeconomic environments, and international relationships. The principal objective of the study is to provide the current technological aspects of artificial intelligence (AI) and other relevant technologies and their implications for confronting COVID-19 and preventing the pandemic’s dreadful effects. This article presents AI approaches that have significant contributions in the fields of health care, then highlights and categorizes their applications in confronting COVID-19, such as detection and diagnosis, data analysis and treatment procedures, research and drug development, social control and services, and the prediction of outbreaks. The study addresses the link between the technologies and the epidemics as well as the potential impacts of technology in health care with the introduction of machine learning and natural language processing tools. It is expected that this comprehensive study will support researchers in modeling health care systems and drive further studies in advanced technologies. Finally, we propose future directions in research and conclude that persuasive AI strategies, probabilistic models, and supervised learning are required to tackle future pandemic challenges.

scholarly journals Developing a Predictive Model for Asthma-Related Hospital Encounters in Patients With Asthma in a Large, Integrated Health Care System: Secondary Analysis (Preprint)

2020 ◽  
Author(s):  
Gang Luo ◽  
Claudia L Nau ◽  
William W Crawford ◽  
Michael Schatz ◽  
Robert S Zeiger ◽  
...  
Keyword(s):  
Machine Learning ◽  
Health Care ◽  
Care Management ◽  
Binary Classification ◽  
Health Care Systems ◽  
Characteristic Curve ◽  
Secondary Analysis ◽  
Learning Model ◽  
Emergency Department Visits ◽  
Machine Learning Model

BACKGROUND Asthma causes numerous hospital encounters annually, including emergency department visits and hospitalizations. To improve patient outcomes and reduce the number of these encounters, predictive models are widely used to prospectively pinpoint high-risk patients with asthma for preventive care via care management. However, previous models do not have adequate accuracy to achieve this goal well. Adopting the modeling guideline for checking extensive candidate features, we recently constructed a machine learning model on Intermountain Healthcare data to predict asthma-related hospital encounters in patients with asthma. Although this model is more accurate than the previous models, whether our modeling guideline is generalizable to other health care systems remains unknown. OBJECTIVE This study aims to assess the generalizability of our modeling guideline to Kaiser Permanente Southern California (KPSC). METHODS The patient cohort included a random sample of 70.00% (397,858/568,369) of patients with asthma who were enrolled in a KPSC health plan for any duration between 2015 and 2018. We produced a machine learning model via a secondary analysis of 987,506 KPSC data instances from 2012 to 2017 and by checking 337 candidate features to project asthma-related hospital encounters in the following 12-month period in patients with asthma. RESULTS Our model reached an area under the receiver operating characteristic curve of 0.820. When the cutoff point for binary classification was placed at the top 10.00% (20,474/204,744) of patients with asthma having the largest predicted risk, our model achieved an accuracy of 90.08% (184,435/204,744), a sensitivity of 51.90% (2259/4353), and a specificity of 90.91% (182,176/200,391). CONCLUSIONS Our modeling guideline exhibited acceptable generalizability to KPSC and resulted in a model that is more accurate than those formerly built by others. After further enhancement, our model could be used to guide asthma care management. INTERNATIONAL REGISTERED REPORT RR2-10.2196/resprot.5039

scholarly journals Developing a Predictive Model for Asthma-Related Hospital Encounters in Patients With Asthma in a Large, Integrated Health Care System: Secondary Analysis

10.2196/22689 ◽  
2020 ◽  
Vol 8 (11) ◽  
pp. e22689
Author(s):  
Gang Luo ◽  
Claudia L Nau ◽  
William W Crawford ◽  
Michael Schatz ◽  
Robert S Zeiger ◽  
...  
Keyword(s):  
Machine Learning ◽  
Health Care ◽  
Care Management ◽  
Binary Classification ◽  
Health Care Systems ◽  
Characteristic Curve ◽  
Secondary Analysis ◽  
Learning Model ◽  
Emergency Department Visits ◽  
Machine Learning Model

Background Asthma causes numerous hospital encounters annually, including emergency department visits and hospitalizations. To improve patient outcomes and reduce the number of these encounters, predictive models are widely used to prospectively pinpoint high-risk patients with asthma for preventive care via care management. However, previous models do not have adequate accuracy to achieve this goal well. Adopting the modeling guideline for checking extensive candidate features, we recently constructed a machine learning model on Intermountain Healthcare data to predict asthma-related hospital encounters in patients with asthma. Although this model is more accurate than the previous models, whether our modeling guideline is generalizable to other health care systems remains unknown. Objective This study aims to assess the generalizability of our modeling guideline to Kaiser Permanente Southern California (KPSC). Methods The patient cohort included a random sample of 70.00% (397,858/568,369) of patients with asthma who were enrolled in a KPSC health plan for any duration between 2015 and 2018. We produced a machine learning model via a secondary analysis of 987,506 KPSC data instances from 2012 to 2017 and by checking 337 candidate features to project asthma-related hospital encounters in the following 12-month period in patients with asthma. Results Our model reached an area under the receiver operating characteristic curve of 0.820. When the cutoff point for binary classification was placed at the top 10.00% (20,474/204,744) of patients with asthma having the largest predicted risk, our model achieved an accuracy of 90.08% (184,435/204,744), a sensitivity of 51.90% (2259/4353), and a specificity of 90.91% (182,176/200,391). Conclusions Our modeling guideline exhibited acceptable generalizability to KPSC and resulted in a model that is more accurate than those formerly built by others. After further enhancement, our model could be used to guide asthma care management. International Registered Report Identifier (IRRID) RR2-10.2196/resprot.5039

scholarly journals Forecasting Future Asthma Hospital Encounters of Patients With Asthma in an Academic Health Care System: Predictive Model Development and Secondary Analysis Study (Preprint)

2020 ◽  
Author(s):  
Yao Tong ◽  
Amanda I Messinger ◽  
Adam B Wilcox ◽  
Sean D Mooney ◽  
Giana H Davidson ◽  
...  
Keyword(s):  
Health Care ◽  
Health Care System ◽  
Health Care Systems ◽  
Secondary Analysis ◽  
Academic Health ◽  
Machine Learning Model ◽  
Care Systems ◽  
Intermountain Healthcare ◽  
Modeling Strategy ◽  
Care System

BACKGROUND Asthma affects a large proportion of the population and leads to many hospital encounters involving both hospitalizations and emergency department visits every year. To lower the number of such encounters, many health care systems and health plans deploy predictive models to prospectively identify patients at high risk and offer them care management services for preventive care. However, the previous models do not have sufficient accuracy for serving this purpose well. Embracing the modeling strategy of examining many candidate features, we built a new machine learning model to forecast future asthma hospital encounters of patients with asthma at Intermountain Healthcare, a nonacademic health care system. This model is more accurate than the previously published models. However, it is unclear how well our modeling strategy generalizes to academic health care systems, whose patient composition differs from that of Intermountain Healthcare. OBJECTIVE This study aims to evaluate the generalizability of our modeling strategy to the University of Washington Medicine (UWM), an academic health care system. METHODS All adult patients with asthma who visited UWM facilities between 2011 and 2018 served as the patient cohort. We considered 234 candidate features. Through a secondary analysis of 82,888 UWM data instances from 2011 to 2018, we built a machine learning model to forecast asthma hospital encounters of patients with asthma in the subsequent 12 months. RESULTS Our UWM model yielded an area under the receiver operating characteristic curve (AUC) of 0.902. When placing the cutoff point for making binary classification at the top 10% (1464/14,644) of patients with asthma with the largest forecasted risk, our UWM model yielded an accuracy of 90.6% (13,268/14,644), a sensitivity of 70.2% (153/218), and a specificity of 90.91% (13,115/14,426). CONCLUSIONS Our modeling strategy showed excellent generalizability to the UWM, leading to a model with an AUC that is higher than all of the AUCs previously reported in the literature for forecasting asthma hospital encounters. After further optimization, our model could be used to facilitate the efficient and effective allocation of asthma care management resources to improve outcomes. INTERNATIONAL REGISTERED REPORT RR2-10.2196/resprot.5039

scholarly journals Forecasting Future Asthma Hospital Encounters of Patients With Asthma in an Academic Health Care System: Predictive Model Development and Secondary Analysis Study

10.2196/22796 ◽  
2021 ◽  
Vol 23 (4) ◽  
pp. e22796
Author(s):  
Yao Tong ◽  
Amanda I Messinger ◽  
Adam B Wilcox ◽  
Sean D Mooney ◽  
Giana H Davidson ◽  
...  
Keyword(s):  
Health Care ◽  
Health Care System ◽  
Health Care Systems ◽  
Secondary Analysis ◽  
Academic Health ◽  
Machine Learning Model ◽  
Care Systems ◽  
Intermountain Healthcare ◽  
Modeling Strategy ◽  
Care System

Background Asthma affects a large proportion of the population and leads to many hospital encounters involving both hospitalizations and emergency department visits every year. To lower the number of such encounters, many health care systems and health plans deploy predictive models to prospectively identify patients at high risk and offer them care management services for preventive care. However, the previous models do not have sufficient accuracy for serving this purpose well. Embracing the modeling strategy of examining many candidate features, we built a new machine learning model to forecast future asthma hospital encounters of patients with asthma at Intermountain Healthcare, a nonacademic health care system. This model is more accurate than the previously published models. However, it is unclear how well our modeling strategy generalizes to academic health care systems, whose patient composition differs from that of Intermountain Healthcare. Objective This study aims to evaluate the generalizability of our modeling strategy to the University of Washington Medicine (UWM), an academic health care system. Methods All adult patients with asthma who visited UWM facilities between 2011 and 2018 served as the patient cohort. We considered 234 candidate features. Through a secondary analysis of 82,888 UWM data instances from 2011 to 2018, we built a machine learning model to forecast asthma hospital encounters of patients with asthma in the subsequent 12 months. Results Our UWM model yielded an area under the receiver operating characteristic curve (AUC) of 0.902. When placing the cutoff point for making binary classification at the top 10% (1464/14,644) of patients with asthma with the largest forecasted risk, our UWM model yielded an accuracy of 90.6% (13,268/14,644), a sensitivity of 70.2% (153/218), and a specificity of 90.91% (13,115/14,426). Conclusions Our modeling strategy showed excellent generalizability to the UWM, leading to a model with an AUC that is higher than all of the AUCs previously reported in the literature for forecasting asthma hospital encounters. After further optimization, our model could be used to facilitate the efficient and effective allocation of asthma care management resources to improve outcomes. International Registered Report Identifier (IRRID) RR2-10.2196/resprot.5039

scholarly journals Trends in Emergency Department Visits and Hospital Admissions in Health Care Systems in 5 States in the First Months of the COVID-19 Pandemic in the US

2020 ◽  
Vol 180 (10) ◽  
pp. 1328 ◽  
Author(s):  
Molly M. Jeffery ◽  
Gail D’Onofrio ◽  
Hyung Paek ◽  
Timothy F. Platts-Mills ◽  
William E. Soares ◽  
...  
Keyword(s):  
Health Care ◽  
Emergency Department ◽  
Hospital Admissions ◽  
Health Care Systems ◽  
Emergency Department Visits ◽  
The Us ◽  
Care Systems

scholarly journals Automatically Explaining Machine Learning Prediction Results on Asthma Hospital Visits in Patients With Asthma: Secondary Analysis (Preprint)

2020 ◽  
Author(s):  
Gang Luo ◽  
Michael D Johnson ◽  
Flory L Nkoy ◽  
Shan He ◽  
Bryan L Stone
Keyword(s):  
Machine Learning ◽  
Performance Measures ◽  
Care Management ◽  
Secondary Analysis ◽  
Learning Model ◽  
Tabular Data ◽  
Tailored Interventions ◽  
Machine Learning Model ◽  
Hospital Visits ◽  
Intermountain Healthcare

BACKGROUND Asthma is a major chronic disease that poses a heavy burden on health care. To facilitate the allocation of care management resources aimed at improving outcomes for high-risk patients with asthma, we recently built a machine learning model to predict asthma hospital visits in the subsequent year in patients with asthma. Our model is more accurate than previous models. However, like most machine learning models, it offers no explanation of its prediction results. This creates a barrier for use in care management, where interpretability is desired. OBJECTIVE This study aims to develop a method to automatically explain the prediction results of the model and recommend tailored interventions without lowering the performance measures of the model. METHODS Our data were imbalanced, with only a small portion of data instances linking to future asthma hospital visits. To handle imbalanced data, we extended our previous method of automatically offering rule-formed explanations for the prediction results of any machine learning model on tabular data without lowering the model’s performance measures. In a secondary analysis of the 334,564 data instances from Intermountain Healthcare between 2005 and 2018 used to form our model, we employed the extended method to automatically explain the prediction results of our model and recommend tailored interventions. The patient cohort consisted of all patients with asthma who received care at Intermountain Healthcare between 2005 and 2018, and resided in Utah or Idaho as recorded at the visit. RESULTS Our method explained the prediction results for 89.7% (391/436) of the patients with asthma who, per our model’s correct prediction, were likely to incur asthma hospital visits in the subsequent year. CONCLUSIONS This study is the first to demonstrate the feasibility of automatically offering rule-formed explanations for the prediction results of any machine learning model on imbalanced tabular data without lowering the performance measures of the model. After further improvement, our asthma outcome prediction model coupled with the automatic explanation function could be used by clinicians to guide the allocation of limited asthma care management resources and the identification of appropriate interventions.

Sign in / Sign up

Export Citation Format

Share Document