Validation of the ‘paediatric extracorporeal membrane oxygenation prediction’ model in a UK extracorporeal membrane oxygenation centre

Perfusion ◽  
2020 ◽  
Vol 35 (8) ◽  
pp. 802-805
Author(s):  
Hari Krishnan Kanthimathinathan ◽  
Sarah Webb ◽  
David Ellis ◽  
Margaret Farley ◽  
Timothy J Jones
Keyword(s):  
Extracorporeal Membrane Oxygenation ◽  
Prediction Model ◽  
Risk Adjustment ◽  
Goodness Of Fit ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Standardized Mortality Ratio ◽  
Membrane Oxygenation ◽  
Mortality Ratio ◽  
Operating Characteristic Curve

Introduction: There is a need for a universal risk-adjustment model that may be used regardless of the indication and nature of neonatal or paediatric extracorporeal membrane oxygenation support. The ‘paediatric extracorporeal membrane oxygenation prediction’ model appeared to be a promising candidate but required external validation. Methods: We performed a validation study using institutional database of extracorporeal membrane oxygenation patients (2008-2019). We used the published paediatric extracorporeal membrane oxygenation prediction score calculator to derive estimated mortality based on the model in this cohort of patients in our institutional database. We used standardized mortality ratio, area under the receiver operating characteristic curve and Hosmer-Lemeshow goodness-of-fit test in 10 deciles to assess model performance. Results: We analysed 154 extracorporeal membrane oxygenation episodes in 150 patients. About 53% of the patients were full term (age ⩽30 days and gestation at birth ⩾37 weeks) neonates. The commonest category of extracorporeal membrane oxygenation support was cardiac (42%). The overall in-paediatric intensive care unit mortality was 37% (57/154) and the in-hospital mortality was 42% (64/154). Distribution of estimated mortality risk was similar to the derivation study. The calculated standardized mortality ratio was 0.81 based on the paediatric extracorporeal membrane oxygenation prediction model of risk-adjustment. The area under the receiver operating characteristic curve was 0.55 (0.45-0.64) and Hosmer-Lemeshow-test p value <0.001 was unable to support goodness-of-fit. Conclusion: This small single-centre study with a small number of events was unable to validate the paediatric extracorporeal membrane oxygenation prediction-model of risk-adjustment. Although this remains the most promising of all the available models, further validation in larger data sets and/or refinement may be required before widespread use.

External validation of the Pediatric Extracorporeal Membrane Oxygenation Prediction model for risk adjusting mortality

Perfusion ◽  
2020 ◽  
pp. 026765912095297
Author(s):  
David K Bailly ◽  
Jamie M Furlong-Dillard ◽  
Melissa Winder ◽  
Mark Lavering ◽  
Ryan P Barbaro ◽  
...  
Keyword(s):  
Extracorporeal Membrane Oxygenation ◽  
Prediction Model ◽  
Goodness Of Fit ◽  
Pediatric Patients ◽  
Life Support ◽  
Extracorporeal Life Support ◽  
Characteristic Curve ◽  
External Validation ◽  
Group Model ◽  
Membrane Oxygenation

Introduction: The Pediatric Extracorporeal Membrane Oxygenation Prediction (PEP) model was created to provide risk stratification for all pediatric patients requiring extracorporeal life support (ECLS). Our purpose was to externally validate the model using contemporaneous cases submitted to the Extracorporeal Life Support Organization (ELSO) registry. Methods: This multicenter, retrospective analysis included pediatric patients (<19 years) during their initial ECLS run for all indications between January 2012 and September 2014. Median values from the BATE dataset for activated partial thromboplastin time and internationalized normalized ratio were used as surrogates as these were missing in the ELSO group. Model discrimination was evaluated using area under the receiver operating characteristic curve (AUC), and goodness-of-fit was evaluated using the Hosmer-Lemeshow test. Results: A total of 4,342 patients in the ELSO registry were compared to 514 subjects from the bleeding and thrombosis on extracorporeal membrane oxygenation (BATE) dataset used to develop the PEP model. Overall mortality was similar (42% ELSO vs. 45% BATE). The c-statistic after external validation decreased from 0.75 to 0.64 and model calibration decreases most in the highest risk deciles. Conclusion: Discrimination of the PEP model remains modest after external validation using the largest pediatric ECLS cohort. While the model overestimates mortality for the highest risk patients, it remains the only prediction model applicable to both neonates and pediatric patients who require ECLS for any indication and thus maintains potential for application in research and quality benchmarking.

scholarly journals A Time-Updated, Parsimonious Model to Predict AKI in Hospitalized Children

2020 ◽  
Vol 31 (6) ◽  
pp. 1348-1357 ◽  
Author(s):  
Ibrahim Sandokji ◽  
Yu Yamamoto ◽  
Aditya Biswas ◽  
Tanima Arora ◽  
Ugochukwu Ugwuowo ◽  
...  
Keyword(s):  
Machine Learning ◽  
Prediction Model ◽  
Receiver Operating Characteristic Curve ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
External Validation ◽  
Health Record ◽  
Hospitalized Children ◽  
Operating Characteristic Curve ◽  
Electronic Health

BackgroundTimely prediction of AKI in children can allow for targeted interventions, but the wealth of data in the electronic health record poses unique modeling challenges.MethodsWe retrospectively reviewed the electronic medical records of all children younger than 18 years old who had at least two creatinine values measured during a hospital admission from January 2014 through January 2018. We divided the study population into derivation, and internal and external validation cohorts, and used five feature selection techniques to select 10 of 720 potentially predictive variables from the electronic health records. Model performance was assessed by the area under the receiver operating characteristic curve in the validation cohorts. The primary outcome was development of AKI (per the Kidney Disease Improving Global Outcomes creatinine definition) within a moving 48-hour window. Secondary outcomes included severe AKI (stage 2 or 3), inpatient mortality, and length of stay.ResultsAmong 8473 encounters studied, AKI occurred in 516 (10.2%), 207 (9%), and 27 (2.5%) encounters in the derivation, and internal and external validation cohorts, respectively. The highest-performing model used a machine learning-based genetic algorithm, with an overall receiver operating characteristic curve in the internal validation cohort of 0.76 [95% confidence interval (CI), 0.72 to 0.79] for AKI, 0.79 (95% CI, 0.74 to 0.83) for severe AKI, and 0.81 (95% CI, 0.77 to 0.86) for neonatal AKI. To translate this prediction model into a clinical risk-stratification tool, we identified high- and low-risk threshold points.ConclusionsUsing various machine learning algorithms, we identified and validated a time-updated prediction model of ten readily available electronic health record variables to accurately predict imminent AKI in hospitalized children.

scholarly journals Severity of Disease Estimation and Risk-Adjustment for Comparison of Outcomes in Mechanically Ventilated Patients Using Electronic Routine Care Data

2015 ◽  
Vol 36 (7) ◽  
pp. 807-815 ◽  
Author(s):  
Maaike S. M. van Mourik ◽  
Karel G. M. Moons ◽  
Michael V. Murphy ◽  
Marc J. M. Bonten ◽  
Michael Klompas ◽  
...  
Keyword(s):  
Disease Severity ◽  
Risk Adjustment ◽  
Laboratory Tests ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Routine Care ◽  
Mechanically Ventilated ◽  
Mechanically Ventilated Patients ◽  
Operating Characteristic Curve ◽  
Ventilated Patients

BACKGROUNDValid comparison between hospitals for benchmarking or pay-for-performance incentives requires accurate correction for underlying disease severity (case-mix). However, existing models are either very simplistic or require extensive manual data collection.OBJECTIVETo develop a disease severity prediction model based solely on data routinely available in electronic health records for risk-adjustment in mechanically ventilated patients.DESIGNRetrospective cohort study.PARTICIPANTSMechanically ventilated patients from a single tertiary medical center (2006–2012).METHODSPredictors were extracted from electronic data repositories (demographic characteristics, laboratory tests, medications, microbiology results, procedure codes, and comorbidities) and assessed for feasibility and generalizability of data collection. Models for in-hospital mortality of increasing complexity were built using logistic regression. Estimated disease severity from these models was linked to rates of ventilator-associated events.RESULTSA total of 20,028 patients were initiated on mechanical ventilation, of whom 3,027 deceased in hospital. For models of incremental complexity, area under the receiver operating characteristic curve ranged from 0.83 to 0.88. A simple model including demographic characteristics, type of intensive care unit, time to intubation, blood culture sampling, 8 common laboratory tests, and surgical status achieved an area under the receiver operating characteristic curve of 0.87 (95% CI, 0.86–0.88) with adequate calibration. The estimated disease severity was associated with occurrence of ventilator-associated events.CONCLUSIONSAccurate estimation of disease severity in ventilated patients using electronic, routine care data was feasible using simple models. These estimates may be useful for risk-adjustment in ventilated patients. Additional research is necessary to validate and refine these models.Infect. Control Hosp. Epidemiol. 2015;36(7):807–815

scholarly journals Validation of a Prediction Tool for Chemotherapy Toxicity in Older Adults With Cancer

2016 ◽  
Vol 34 (20) ◽  
pp. 2366-2371 ◽  
Author(s):  
Arti Hurria ◽  
Supriya Mohile ◽  
Ajeet Gajra ◽  
Heidi Klepin ◽  
Hyman Muss ◽  
...  
Keyword(s):  
Older Adults ◽  
High Risk ◽  
Prediction Model ◽  
Receiver Operating Characteristic Curve ◽  
Predictive Model ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Chemotherapy Toxicity ◽  
Operating Characteristic Curve ◽  
Grade 3

Purpose Older adults are at increased risk for chemotherapy toxicity, and standard oncology assessment measures cannot identify those at risk. A predictive model for chemotherapy toxicity was developed (N = 500) that consisted of geriatric assessment questions and other clinical variables. This study aims to externally validate this model in an independent cohort (N = 250). Patients and Methods Patients age ≥ 65 years with a solid tumor, fluent in English, and who were scheduled to receive a new chemotherapy regimen were recruited from eight institutions. Risk of chemotherapy toxicity was calculated (low, medium, or high risk) on the basis of the prediction model before the start of chemotherapy. Chemotherapy-related toxicity was captured (grade 3 [hospitalization indicated], grade 4 [life threatening], and grade 5 [treatment-related death]). Validation of the prediction model was performed by calculating the area under the receiver-operating characteristic curve. Results The study sample (N = 250) had a mean age of 73 years (range, 65 to 94 [standard deviation, 5.8]). More than one half of patients (58%) experienced grade ≥ 3 toxicity. Risk of toxicity increased with increasing risk score (36.7% low, 62.4% medium, 70.2% high risk; P < .001). The area under the curve of the receiver-operating characteristic curve was 0.65 (95% CI, 0.58 to 0.71), which was not statistically different from the development cohort (0.72; 95% CI, 0.68 to 0.77; P = .09). There was no association between Karnofsky Performance Status and chemotherapy toxicity (P = .25). Conclusion This study externally validated a chemotherapy toxicity predictive model for older adults with cancer. This predictive model should be considered when discussing the risks and benefits of chemotherapy with older adults.

scholarly journals Impact of Operator Expertise on Collection of the APACHE II Score and on the Derived Risk of Death and Standardized Mortality Ratio

2005 ◽  
Vol 33 (5) ◽  
pp. 585-590 ◽  
Author(s):  
D. Ledoux ◽  
S. Finfer ◽  
S. Mckinley
Keyword(s):  
Goodness Of Fit ◽  
Characteristic Curve ◽  
Receiver Operator Characteristic Curve ◽  
Clinical Staff ◽  
Apache Ii ◽  
Apache Ii Score ◽  
Standardized Mortality Ratio ◽  
Risk Of Death ◽  
Mortality Ratio ◽  
The Impact

We assessed the impact of operator expertise on collection of the APACHE II score, the derived risk of death and standardized mortality ratio in 465 consecutive patients admitted to a multi-disciplinary tertiary hospital ICU. Research coordinators and junior clinical staff independently collected the APACHE II variables; experts (senior clinical staff) rescored 20 % of the records. Agreement was moderate between junior clinical staff and research coordinators or senior clinical staff for most variables of the acute physiology score (weighted κ<0.6); agreement between research coordinators and senior clinical staff data collectors was good (weighted κ >0.75). The APACHE II score and its derived risk of death (ROD) were significantly lower using the junior clinical staff dataset compared to research coordinators and senior clinical staff (APACHE II score: 13.4±9.2 vs 16.8±8.5 vs 17.1±7.7, P<0.001; ROD: 14.7%±22.4% vs 21.6%±22.6% vs 20.8%±22.4%, P<0.01 respectively). The discriminative capacity was not altered by the lack of agreement (area under Receiver Operator Characteristic curve >0.8) but calibration of ROD from the junior clinical staff dataset was poor (Goodness-of-fit: P=0.001). The standardized mortality ratio (SMR) was higher with the junior clinical staff dataset (SMR: 1.22, 95% CI: 0.96-1.52 vs 0.87, 95% CI: 0.70-1.06 vs 0.76, 95% CI: 0.40-1.3 calculated from junior clinical staff, research coordinators and senior clinical staff data-sets respectively). We conclude that the expertise of data collectors significantly influences the APACHE II score, the derived risk of death and the standardized mortality ratio. Given the importance of such scores, ICUs should be provided with sufficient resources to train and employ dedicated data collectors.

scholarly journals A machine learning–based 1-year mortality prediction model after hospital discharge for clinical patients with acute coronary syndrome

2019 ◽  
Vol 26 (2) ◽  
pp. 1289-1304 ◽  
Author(s):  
Syed Waseem Abbas Sherazi ◽  
Yu Jun Jeong ◽  
Moon Hyun Jae ◽  
Jang-Whan Bae ◽  
Jong Yun Lee
Keyword(s):  
Machine Learning ◽  
Cardiovascular Disease ◽  
Acute Coronary Syndrome ◽  
Prediction Model ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Mortality Prediction ◽  
Coronary Syndrome ◽  
Mortality Prediction Model ◽  
Operating Characteristic Curve

Cardiovascular disease is the leading cause of death worldwide so, early prediction and diagnosis of cardiovascular disease is essential for patients affected by this fatal disease. The goal of this article is to propose a machine learning–based 1-year mortality prediction model after discharge in clinical patients with acute coronary syndrome. We used the Korea Acute Myocardial Infarction Registry data set, a cardiovascular disease database registered in 52 hospitals in Korea for 1 November 2005–30 January 2008 and selected 10,813 subjects with 1-year follow-up traceability. The ranges of hyperparameters to find the best prediction model were selected from four different machine learning models. Then, we generated each machine learning–based mortality prediction model with hyperparameters completed the range fitness via grid search using training data and was evaluated by fourfold stratified cross-validation. The best prediction model with the highest performance was found, and its hyperparameters were extracted. Finally, we compared the performance of machine learning–based mortality prediction models with GRACE in area under the receiver operating characteristic curve, precision, recall, accuracy, and F-score. The area under the receiver operating characteristic curve in applied machine learning algorithms was averagely improved up to 0.08 than in GRACE, and their major prognostic factors were different. This implementation would be beneficial for prediction and early detection of major adverse cardiovascular events in acute coronary syndrome patients.

Validation of the CRASH model in the prediction of 18-month mortality and unfavorable outcome in severe traumatic brain injury requiring decompressive craniectomy

2014 ◽  
Vol 120 (5) ◽  
pp. 1131-1137 ◽  
Author(s):  
Stephen Honeybul ◽  
Kwok M. Ho ◽  
Christopher R. P. Lind ◽  
Grant R. Gillett
Keyword(s):  
Traumatic Brain Injury ◽  
Brain Injury ◽  
Prediction Model ◽  
Decompressive Craniectomy ◽  
Severe Traumatic Brain Injury ◽  
Operating Characteristic ◽  
Injury Severity ◽  
Characteristic Curve ◽  
Operating Characteristic Curve ◽  
Crash Model

Object The goal in this study was to assess the validity of the corticosteroid randomization after significant head injury (CRASH) collaborators prediction model in predicting mortality and unfavorable outcome at 18 months in patients with severe traumatic brain injury (TBI) requiring decompressive craniectomy. In addition, the authors aimed to assess whether this model was well calibrated in predicting outcome across a wide spectrum of severity of TBI requiring decompressive craniectomy. Methods This prospective observational cohort study included all patients who underwent a decompressive craniectomy following severe TBI at the two major trauma hospitals in Western Australia between 2004 and 2012 and for whom 18-month follow-up data were available. Clinical and radiological data on initial presentation were entered into the Web-based model and the predicted outcome was compared with the observed outcome. In validating the CRASH model, the authors used area under the receiver operating characteristic curve to assess the ability of the CRASH model to differentiate between favorable and unfavorable outcomes. Results The ability of the CRASH 6-month unfavorable prediction model to differentiate between unfavorable and favorable outcomes at 18 months after decompressive craniectomy was good (area under the receiver operating characteristic curve 0.85, 95% CI 0.80–0.90). However, the model's calibration was not perfect. The slope and the intercept of the calibration curve were 1.66 (SE 0.21) and −1.11 (SE 0.14), respectively, suggesting that the predicted risks of unfavorable outcomes were not sufficiently extreme or different across different risk strata and were systematically too high (or overly pessimistic), respectively. Conclusions The CRASH collaborators prediction model can be used as a surrogate index of injury severity to stratify patients according to injury severity. However, clinical decisions should not be based solely on the predicted risks derived from the model, because the number of patients in each predicted risk stratum was still relatively small and hence the results were relatively imprecise. Notwithstanding these limitations, the model may add to a clinician's ability to have better-informed conversations with colleagues and patients' relatives about prognosis.

Sign in / Sign up

Export Citation Format

Share Document