Benchmarking in Congenital Heart Surgery Using Machine Learning-Derived Optimal Classification Trees

2021 ◽  
pp. 215013512110512
Author(s):  
Dimitris Bertsimas ◽  
Daisy Zhuo ◽  
Jordan Levine ◽  
Jack Dunn ◽  
Zdzislaw Tobota ◽  
...  
Keyword(s):  
Machine Learning ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Ventilatory Support ◽  
Classification Trees ◽  
Length Of Hospital Stay ◽  
Congenital Heart Surgery ◽  
Case Mix ◽  
Specific Patient ◽  
Optimal Classification

Background: We have previously shown that the machine learning methodology of optimal classification trees (OCTs) can accurately predict risk after congenital heart surgery (CHS). We have now applied this methodology to define benchmarking standards after CHS, permitting case-adjusted hospital-specific performance evaluation. Methods: The European Congenital Heart Surgeons Association Congenital Database data subset (31 792 patients) who had undergone any of the 10 “benchmark procedure group” primary procedures were analyzed. OCT models were built predicting hospital mortality (HM), and prolonged postoperative mechanical ventilatory support time (MVST) or length of hospital stay (LOS), thereby establishing case-adjusted benchmarking standards reflecting the overall performance of all participating hospitals, designated as the “virtual hospital.” These models were then used to predict individual hospitals’ expected outcomes (both aggregate and, importantly, for risk-matched patient cohorts) for their own specific cases and case-mix, based on OCT analysis of aggregate data from the “virtual hospital.” Results: The raw average rates were HM = 4.4%, MVST = 15.3%, and LOS = 15.5%. Of 64 participating centers, in comparison with each hospital's specific case-adjusted benchmark, 17.0% statistically (under 90% confidence intervals) overperformed and 26.4% underperformed with respect to the predicted outcomes for their own specific cases and case-mix. For MVST and LOS, overperformers were 34.0% and 26.4%, and underperformers were 28.3% and 43.4%, respectively. OCT analyses reveal hospital-specific patient cohorts of either overperformance or underperformance. Conclusions: OCT benchmarking analysis can assess hospital-specific case-adjusted performance after CHS, both overall and patient cohort-specific, serving as a tool for hospital self-assessment and quality improvement.

scholarly journals Adverse Outcomes Prediction for Congenital Heart Surgery: A Machine Learning Approach

2021 ◽  
pp. 215013512110071
Author(s):  
Dimitris Bertsimas ◽  
Daisy Zhuo ◽  
Jack Dunn ◽  
Jordan Levine ◽  
Eugenio Zuccarelli ◽  
...  
Keyword(s):  
Machine Learning ◽  
Risk Factors ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Classification Trees ◽  
Congenital Heart Surgery ◽  
Assessment Tools ◽  
Hospital Length Of Stay ◽  
Total Quality ◽  
Optimal Classification

Objective: Risk assessment tools typically used in congenital heart surgery (CHS) assume that various possible risk factors interact in a linear and additive fashion, an assumption that may not reflect reality. Using artificial intelligence techniques, we sought to develop nonlinear models for predicting outcomes in CHS. Methods: We built machine learning (ML) models to predict mortality, postoperative mechanical ventilatory support time (MVST), and hospital length of stay (LOS) for patients who underwent CHS, based on data of more than 235,000 patients and 295,000 operations provided by the European Congenital Heart Surgeons Association Congenital Database. We used optimal classification trees (OCTs) methodology for its interpretability and accuracy, and compared to logistic regression and state-of-the-art ML methods (Random Forests, Gradient Boosting), reporting their area under the curve (AUC or c-statistic) for both training and testing data sets. Results: Optimal classification trees achieve outstanding performance across all three models (mortality AUC = 0.86, prolonged MVST AUC = 0.85, prolonged LOS AUC = 0.82), while being intuitively interpretable. The most significant predictors of mortality are procedure, age, and weight, followed by days since previous admission and any general preoperative patient risk factors. Conclusions: The nonlinear ML-based models of OCTs are intuitively interpretable and provide superior predictive power. The associated risk calculator allows easy, accurate, and understandable estimation of individual patient risks, in the theoretical framework of the average performance of all centers represented in the database. This methodology has the potential to facilitate decision-making and resource optimization in CHS, enabling total quality management and precise benchmarking initiatives.

Reimbursement After Congenital Heart Surgery in Germany: Impact of Early Postoperative Extubation

2020 ◽  
Vol 11 (5) ◽  
pp. 557-562 ◽  
Author(s):  
Peter Murin ◽  
Viktoria Weixler ◽  
Mi-Young Cho ◽  
Valentin Vadiunec ◽  
Oliver Miera ◽  
...  
Keyword(s):  
Mechanical Ventilation ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Open Heart Surgery ◽  
Congenital Heart Surgery ◽  
Case Mix ◽  
Open Heart ◽  
Early Extubation ◽  
Cost Weight ◽  
Case Mix Index

Background: Duration of mechanical ventilation is an important variable used by German Diagnosis-Related Groups (G-DRG) system to establish cost weight values for reimbursement after congenital heart surgery. Infants are commonly ventilated after open heart surgery. As of year 2015, we strived to achieve early postoperative extubation. This work studies how this approach impacted reimbursement after infant open heart surgery. Methods: Data of infants who underwent surgery on cardiopulmonary bypass (CPB) from 2014 to 2018 were reviewed. Successful early extubation was defined as end of mechanical ventilation within 24 hours postoperatively, without reintubation at a later point. Mean cost weight values (case mix index [CMI]) of achieved DRGs were used for estimation of reimbursement. Evolutions over years of early extubation and of reimbursement were compared. Results: A total of 521 infants underwent operations on CPB. Of these, 161 (31%) procedures were of higher risk Society of Thoracic Surgery and the European Association for Cardio-Thoracic Surgery (STAT) categories 3 and 4. Early extubation was achieved in 205 (39%) patients. The rate increased from 14% (year 2014) to 57% (year 2018). Case mix index amounted to 8.87 ± 7.00 after early extubation, and 12.37 ± 7.85 after late extubation: P value <.0001. It was 8.77 ± 6.09 after early extubation in patients undergoing lower risk STAT categories 1 and 2 operations, and 8.09 ± 2.95 when categories 3 and 4 procedures were performed ( P = .18). An overall 14.4% decrease in hospital reimbursement per patient was observed. Conclusion: Early extubation could be progressively obtained in the majority of infants. This resulted in lower reimbursement. Surgical complexity was disregarded. The current G-DRG system appears to favor longer mechanical ventilation durations after infant open heart surgery.

Bilateral diaphragmatic palsy after congenital heart surgery: management options

2015 ◽  
Vol 26 (5) ◽  
pp. 927-930 ◽  
Author(s):  
Pradeep Bhaskar ◽  
Reyaz A. Lone ◽  
Ahmad Sallehuddin ◽  
Jiju John ◽  
Akhlaque N. Bhat ◽  
...  
Keyword(s):  
Congenital Heart ◽  
Heart Surgery ◽  
Recovery Of Function ◽  
Length Of Hospital Stay ◽  
Major Complication ◽  
Congenital Heart Surgery ◽  
Diaphragmatic Paralysis ◽  
Median Length ◽  
Management Options ◽  
Paradoxical Movement

AbstractDiaphragmatic paralysis following phrenic nerve injury is a major complication following congenital cardiac surgery. In contrast to unilateral paralysis, patients with bilateral diaphragmatic paralysis present a higher risk group, require different management methods, and have poorer prognosis. We retrospectively analysed seven patients who had bilateral diaphragmatic paralysis following congenital heart surgery during the period from July, 2006 to July, 2014. Considerations were given to the time to diagnosis of diaphragm paralysis, total ventilator days, interval after plication, and lengths of ICU and hospital stays. The incidence of bilateral diaphragmatic paralysis was 0.68% with a median age of 2 months (0.6–12 months). There was one neonate and six infants with a median weight of 4 kg (3–7 kg); five patients underwent unilateral plication of the paradoxical diaphragm following recovery of the other side, whereas the remaining two patients who did not demonstrate a paradoxical movement were successfully weaned from the ventilator following recovery of function in one of the diaphragms. The median ventilation time for the whole group was 48 days (20–90 days). The median length of ICU stay was 46 days (24–110 days), and the median length of hospital stay was 50 days (30–116 days). None of the patients required tracheostomy for respiratory support and there were no mortalities, although all the patients except one developed ventilator-associated pneumonia. The outcome of different management options for bilateral diaphragmatic paralysis following surgery for CHD is discussed.

Peritoneal dialysis during congenital heart surgery admissions: insights from a large database

2020 ◽  
Vol 30 (4) ◽  
pp. 451-455
Author(s):  
Rohit S. Loomba ◽  
Enrique G. Villarreal ◽  
Ronald A. Bronicki ◽  
Saul Flores
Keyword(s):  
Peritoneal Dialysis ◽  
Length Of Stay ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Septal Defect ◽  
Congenital Heart Surgery ◽  
Fluid Restriction ◽  
Specific Patient ◽  
Left Heart Syndrome ◽  
Univariate Analyses

AbstractBackground:The management of fluid overload after congenital heart surgery has been limited to diuretics, fluid restriction, and dialysis. This study was conducted to determine the association between peritoneal dialysis and important clinical outcomes in children undergoing congenital heart surgery.Methods:A retrospective review was conducted to identify patients under 18 years of age who underwent congenital heart surgery. The data were obtained over a 16-year period (1997–2012) from the Kids’ Inpatient Database. Data analysed consisted of demographics, diagnoses, type of congenital heart surgery, length of stay, cost of hospitalisation, and mortality. Logistic regression was performed to determine factors associated with peritoneal dialysis.Results:A total of 46,176 admissions after congenital heart surgery were included in the study. Of those, 181 (0.4%) utilised peritoneal dialysis. The mean age of the peritoneal dialysis group was 7.6 months compared to 39.6 months in those without peritoneal dialysis. The most common CHDs were atrial septal defect (37%), ventricular septal defect (32.6%), and hypoplastic left heart syndrome (18.8%). Univariate analyses demonstrated significantly greater length of stay, cost of admission, and mortality in those with peritoneal dialysis. Regression analyses demonstrated that peritoneal dialysis was independently associated with significant decrease in cost of admission (−$57,500) and significant increase in mortality (odds ratio 1.5).Conclusions:Peritoneal dialysis appears to be used in specific patient subsets and is independently associated with decreased cost of stay, although it is associated with increased mortality. Further studies are needed to describe risks and benefit of peritoneal dialysis in this population.

Stratification of complexity: The Risk Adjustment for Congenital Heart Surgery-1 Method and The Aristotle Complexity Score – past, present, and future

2008 ◽  
Vol 18 (S2) ◽  
pp. 163-168 ◽  
Author(s):  
Marshall Lewis Jacobs ◽  
Jeffrey Phillip Jacobs ◽  
Kathy J. Jenkins ◽  
Kimberlee Gauvreau ◽  
David R. Clarke ◽  
...  
Keyword(s):  
Cardiac Surgery ◽  
Quality Of Care ◽  
Risk Adjustment ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Congenital Heart Surgery ◽  
Case Mix ◽  
Objective Data ◽  
Complexity Score

AbstractMeaningful evaluation of quality of care must account for variations in the population of patients receiving treatment, or “case-mix”. In adult cardiac surgery, empirical clinical data, initially from tens of thousands, and more recently hundreds of thousands of operations, have been used to develop risk-models, to increase the accuracy with which the outcome of a given procedure on a given patient can be predicted, and to compare outcomes on non-identical patient groups between centres, surgeons and eras.In the adult cardiac database of The Society of Thoracic Surgeons, algorithms for risk-adjustment are based on over 1.5 million patients undergoing isolated coronary artery bypass grafting and over 100,000 patients undergoing isolated replacement of the aortic valve or mitral valve. In the pediatric and congenital cardiac database of The Society of Thoracic Surgeons, 61,014 operations are spread out over greater than 100 types of primary procedures. The problem of evaluating quality of care in the management of pediatric patients with cardiac diseases is very different, and in some ways a great deal more challenging, because of the smaller number of patients and the higher number of types of operations.In the field of pediatric cardiac surgery, the importance of the quantitation of the complexity of operations centers on the fact that outcomes analysis using raw measurements of mortality, without adjustment for complexity, is inadequate. Case-mix can vary greatly from program to program. Without stratification of complexity, the analysis of outcomes for congenital cardiac surgery will be flawed. Two major multi-institutional efforts have attempted to measure the complexity of pediatric cardiac operations: the Risk Adjustment in Congenital Heart Surgery-1 method and the Aristotle Complexity Score. Both systems were derived in large part from subjective probability, or expert opinion. Both systems are currently in wide use throughout the world and have been shown to correlate reasonably well with outcome.Efforts are underway to develop the next generation of these systems. The next generation will be based more on objective data, but will continue to utilize subjective probability where objective data is lacking. A goal, going forward, is to re-evaluate and further refine these tools so that, they can be, to a greater extent, derived from empirical data. During this process, ideally, the mortality elements of both the Aristotle Complexity Score and the Risk Adjustment in Congenital Heart Surgery-1 methodology will eventually unify and become one and the same. This review article examines these two systems of stratification of complexity and reviews the rationale for the development of each system, the current use of each system, the plans for future enhancement of each system, and the potential for unification of these two tools.

scholarly journals Explainable machine-learning predictions for complications after pediatric congenital heart surgery

2021 ◽  
Author(s):  
Xian Zeng ◽  
Yaoqin Hu ◽  
Liqi Shu ◽  
Jianhua Li ◽  
Huilong Duan ◽  
...  
Keyword(s):  
Machine Learning ◽  
Blood Pressure ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Time Series Data ◽  
Model Performance ◽  
Congenital Heart Surgery ◽  
Series Data ◽  
Complication Risk

Abstract The quality of treatment and prognosis after pediatric congenital heart surgery remains unsatisfactory. A reliable prediction model for postoperative complications of congenital heart surgery patients is essential to enable prompt initiation of therapy and improve the quality of prognosis. Here, we develop an interpretable machine-learning-based model that integrates patient demographics, surgery-specific features and intraoperative blood pressure data for accurately predicting complications after pediatric congenital heart surgery. We used blood pressure variability and the k-means algorithm combined with a smoothed formulation of dynamic time wrapping to extract features from time series data. In addition, SHAP framework was used to provide explanations of the prediction. Our model achieved the best performance both in binary and multi-label classification compared with other consensus-based risk models. In addition, this explainable model explains why a prediction was made to help improve the clinical understanding of complication risk and generate actionable knowledge in practice. The combination of model performance and interpretability is easy for clinicians to trust and provide insight into how they should respond before the condition worsens after pediatric congenital heart surgery.

scholarly journals Congenital Heart Surgery Case Mix Across North American Centers and Impact on Performance Assessment

2016 ◽  
Vol 102 (5) ◽  
pp. 1580-1587 ◽  
Author(s):  
Sara K. Pasquali ◽  
Amelia S. Wallace ◽  
J. William Gaynor ◽  
Marshall L. Jacobs ◽  
Sean M. O’Brien ◽  
...  
Keyword(s):  
Performance Assessment ◽  
North American ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Congenital Heart Surgery ◽  
Case Mix

scholarly journals Explainable machine-learning predictions for complications after pediatric congenital heart surgery

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Xian Zeng ◽  
Yaoqin Hu ◽  
Liqi Shu ◽  
Jianhua Li ◽  
Huilong Duan ◽  
...  
Keyword(s):  
Machine Learning ◽  
Blood Pressure ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Time Series Data ◽  
Model Performance ◽  
Congenital Heart Surgery ◽  
Series Data ◽  
Complication Risk

AbstractThe quality of treatment and prognosis after pediatric congenital heart surgery remains unsatisfactory. A reliable prediction model for postoperative complications of congenital heart surgery patients is essential to enable prompt initiation of therapy and improve the quality of prognosis. Here, we develop an interpretable machine-learning-based model that integrates patient demographics, surgery-specific features and intraoperative blood pressure data for accurately predicting complications after pediatric congenital heart surgery. We used blood pressure variability and the k-means algorithm combined with a smoothed formulation of dynamic time wrapping to extract features from time-series data. In addition, SHAP framework was used to provide explanations of the prediction. Our model achieved the best performance both in binary and multi-label classification compared with other consensus-based risk models. In addition, this explainable model explains why a prediction was made to help improve the clinical understanding of complication risk and generate actionable knowledge in practice. The combination of model performance and interpretability is easy for clinicians to trust and provide insight into how they should respond before the condition worsens after pediatric congenital heart surgery.

Updating an Empirically Based Tool for Analyzing Congenital Heart Surgery Mortality

2021 ◽  
Vol 12 (2) ◽  
pp. 246-281 ◽  
Author(s):  
Marshall L. Jacobs ◽  
Jeffrey P. Jacobs ◽  
Dylan Thibault ◽  
Kevin D. Hill ◽  
Brett R. Anderson ◽  
...  
Keyword(s):  
Mortality Risk ◽  
Congenital Heart ◽  
Heart Surgery ◽  
Congenital Heart Surgery ◽  
Case Mix ◽  
C Statistic ◽  
Multiple Component ◽  
Procedure Codes ◽  
Mortality Estimates ◽  
Component Procedure

Objectives: STAT Mortality Categories (developed 2009) stratify congenital heart surgery procedures into groups of increasing mortality risk to characterize case mix of congenital heart surgery providers. This update of the STAT Mortality Score and Categories is empirically based for all procedures and reflects contemporary outcomes. Methods: Cardiovascular surgical operations in the Society of Thoracic Surgeons Congenital Heart Surgery Database (January 1, 2010 – June 30, 2017) were analyzed. In this STAT 2020 Update of the STAT Mortality Score and Categories, the risk associated with a specific combination of procedures was estimated under the assumption that risk is determined by the highest risk individual component procedure. Operations composed of multiple component procedures were eligible for unique STAT Scores when the statistically estimated mortality risk differed from that of the highest risk component procedure. Bayesian modeling accounted for small denominators. Risk estimates were rescaled to STAT 2020 Scores between 0.1 and 5.0. STAT 2020 Category assignment was designed to minimize within-category variation and maximize between-category variation. Results: Among 161,351 operations at 110 centers (19,090 distinct procedure combinations), 235 types of single or multiple component operations received unique STAT 2020 Scores. Assignment to Categories resulted in the following distribution: STAT 2020 Category 1 includes 59 procedure codes with model-based estimated mortality 0.2% to 1.3%; Category 2 includes 73 procedure codes with mortality estimates 1.4% to 2.9%; Category 3 includes 46 procedure codes with mortality estimates 3.0% to 6.8%; Category 4 includes 37 procedure codes with mortality estimates 6.9% to 13.0%; and Category 5 includes 17 procedure codes with mortality estimates 13.5% to 38.7%. The number of procedure codes with empirically derived Scores has grown by 58% (235 in STAT 2020 vs 148 in STAT 2009). Of the 148 procedure codes with empirically derived Scores in 2009, approximately one-half have changed STAT Category relative to 2009 metrics. The New STAT 2020 Scores and Categories demonstrated good discrimination for predicting mortality in an independent validation sample (July 1, 2017-June 30, 2019; sample size 46,933 operations at 108 centers) with C-statistic = 0.791 for STAT 2020 Score and 0.779 for STAT 2020 Category. Conclusions: The updated STAT metrics reflect contemporary practice and outcomes. New empirically based STAT 2020 Scores and Category designations are assigned to a larger set of procedure codes, while accounting for risk associated with multiple component operations. Updating STAT metrics based on contemporary outcomes facilitates accurate assessment of case mix.

Sign in / Sign up

Export Citation Format

Share Document