scholarly journals Using the OHDSI network to develop and externally validate a patient-level prediction model for Heart Failure in Type II Diabetes Mellitus

2021 ◽  
Author(s):  
Ross D. Williams ◽  
Jenna M. Reps ◽  
Jan A Kors ◽  
Patrick B Ryan ◽  
Ewout Steyerberg ◽  
...  
Keyword(s):  
Diabetes Mellitus ◽  
Heart Failure ◽  
High Performance ◽  
Prediction Models ◽  
External Validation ◽  
The United States ◽  
The Other ◽  
Patient Treatment ◽  
Administrative Claims ◽  
Second Line

AbstractIntroductionHeart Failure (HF) and Type 2 Diabetes Mellitus (T2DM) frequently coexist and exacerbate symptoms of each other. Treatments are available for T2DM that also provide beneficial treatment effects for HF. Guidelines recommend that patients with HF should be given Sodium-glucose co-transporter-2 inhibitors in preference to other second-line treatments for T2DM. Increasing personalization of treatment means that patients who have or are at risk of HF receive a customised treatment. We aimed to develop and externally validate prediction models to predict the 1-year risk of incident HF in T2DM patients starting second-line treatment.MethodsWe analysed a federated network of electronic medical records and administrative claims data from five databases (CCAE, MDCD, MDCR, Optum Clinformatics and Optum EHR) in the United States. We used each database to develop a model to predict 1-year risk of incident HF in patients initialising a second pharmaceutical intervention, following initial treatment with metformin for T2DM. We then performed internal validation for each model as well as external validation using the other databases.ResultsA total of 403,187 patients were included in the study. We developed 5 models with discrimination ranging from 0.72 to 0.80 at external validation in the other databases. Consistent high performance was noted for models developed in CCAE, Optum Clinformatics and Optum EHR with AUCs ranging from 0.74 to 0.81. For these models, calibration was acceptable.ConclusionThree high-performing prediction models were developed for this problem. The CCAE developed model was selected for recommendation as it achieved the same discrimination and better calibration performance than the Optum Clinformatics and Optum EHR models, whilst selecting fewer covariates and as such was selected as the best developed model. The models could be useful in stratifying patient treatment, planning healthcare utilization and reducing cost by aiding in increasing preparedness of healthcare providers.

scholarly journals Models for predicting treatment efficacy of antiepileptic drugs and prognosis of treatment withdrawal in epilepsy patients

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Shijun Yang ◽  
Bin Wang ◽  
Xiong Han
Keyword(s):  
Antiepileptic Drugs ◽  
Statistical Models ◽  
Prediction Models ◽  
External Validation ◽  
Machine Learning Algorithms ◽  
Treatment Withdrawal ◽  
Patient Treatment ◽  
Recent Developments ◽  
Progression Of Disease ◽  
Patients With Epilepsy

AbstractAlthough antiepileptic drugs (AEDs) are the most effective treatment for epilepsy, 30–40% of patients with epilepsy would develop drug-refractory epilepsy. An accurate, preliminary prediction of the efficacy of AEDs has great clinical significance for patient treatment and prognosis. Some studies have developed statistical models and machine-learning algorithms (MLAs) to predict the efficacy of AEDs treatment and the progression of disease after treatment withdrawal, in order to provide assistance for making clinical decisions in the aim of precise, personalized treatment. The field of prediction models with statistical models and MLAs is attracting growing interest and is developing rapidly. What’s more, more and more studies focus on the external validation of the existing model. In this review, we will give a brief overview of recent developments in this discipline.

Abstract 196: Development and Validation of Machine Learning-based Race-specific Models to Predict 10-year Risk of Heart Failure: A Multi-cohort Analysis

Circulation ◽  
2020 ◽  
Vol 142 (Suppl_3) ◽  
Author(s):  
Matthew W Segar ◽  
Byron Jaeger ◽  
Kershaw V Patel ◽  
Vijay Nambi ◽  
Chiadi E Ndumele ◽  
...  
Keyword(s):  
Machine Learning ◽  
Risk Factors ◽  
Heart Failure ◽  
Risk Prediction ◽  
Prediction Models ◽  
External Validation ◽  
Superior Performance ◽  
Risk Models ◽  
Black And White ◽  
White Adults

Introduction: Heart failure (HF) risk and the underlying biological risk factors vary by race. Machine learning (ML) may improve race-specific HF risk prediction but this has not been fully evaluated. Methods: The study included participants from 4 cohorts (ARIC, DHS, JHS, and MESA) aged > 40 years, free of baseline HF, and with adjudicated HF event follow-up. Black adults from JHS and white adults from ARIC were used to derive race-specific ML models to predict 10-year HF risk. The ML models were externally validated in subgroups of black and white adults from ARIC (excluding JHS participants) and pooled MESA/DHS cohorts and compared to prior established HF risk scores developed in ARIC and MESA. Harrell’s C-index and Greenwood-Nam-D’Agostino chi-square were used to assess discrimination and calibration, respectively. Results: In the derivation cohorts, 288 of 4141 (7.0%) black and 391 of 8242 (4.7%) white adults developed HF over 10 years. The ML models had excellent discrimination in both black and white participants (C-indices = 0.88 and 0.89). In the external validation cohorts for black participants from ARIC (excluding JHS, N = 1072) and MESA/DHS pooled cohorts (N = 2821), 131 (12.2%) and 115 (4.1%) developed HF. The ML model had adequate calibration and demonstrated superior discrimination compared to established HF risk models (Fig A). A consistent pattern was also observed in the external validation cohorts of white participants from the MESA/DHS pooled cohorts (N=3236; 100 [3.1%] HF events) (Fig A). The most important predictors of HF in both races were NP levels. Cardiac biomarkers and glycemic parameters were most important among blacks while LV hypertrophy and prevalent CVD and traditional CV risk factors were the strongest predictors among whites (Fig B). Conclusions: Race-specific and ML-based HF risk models that integrate clinical, laboratory, and biomarker data demonstrated superior performance when compared to traditional risk prediction models.

scholarly journals Estimating the number of preventable cardiovascular disease events in the United States using the EMPA-REG OUTCOME trial results and National Health and Nutrition Examination Survey

2020 ◽  
Vol 17 (4) ◽  
pp. 147916412094567
Author(s):  
Nathan D Wong ◽  
Wenjun Fan ◽  
Jonathan Pak
Keyword(s):  
Diabetes Mellitus ◽  
Heart Failure ◽  
Cardiovascular Disease ◽  
Confidence Interval ◽  
National Health ◽  
The United States ◽  
Eligibility Criteria ◽  
Health And Nutrition ◽  
The Difference ◽  
Event Rates

Aim: We examined eligibility and preventable cardiovascular disease events in US adults with diabetes mellitus from the Empagliflozin Cardiovascular Outcome Event Trial in Type 2 Diabetes Mellitus Patients (EMPA-REG OUTCOME). Methods: We identified adults with diabetes mellitus eligible for EMPA-REG OUTCOME based on trial eligibility criteria available from the National Health and Nutrition Examination Surveys, 2007–2016. We estimated composite cardiovascular disease endpoints, as well as all-cause deaths, death from cardiovascular disease and hospitalizations for heart failure from trial treatment and placebo event rates, the difference indicating the preventable events. Results: Among 29,629 US adults aged ⩾18 years (representing 231.9 million), 4672 (27.3 million) had diabetes mellitus, with 342 (1.86 million) meeting eligibility criteria of EMPA-REG OUTCOME. We estimated from trial primary endpoint event rates of 10.5% and 12.1% in the empagliflozin and placebo groups, respectively, that based on the ‘treatment’ of our 1.86 million estimated EMPA-REG OUTCOME eligible subjects, 12,066 (95% confidence interval: 10,352–13,780) cardiovascular disease events could be prevented annually. Estimated annual preventable deaths from any cause, cardiovascular causes and hospitalizations from heart failure were 17,078 (95% confidence interval: 14,652–19,504), 14,479 (95% confidence interval: 12,422–16,536) and 9467 (95% confidence interval: 8122–10,812), respectively. Conclusion: Empagliflozin, if provided to EMPA-REG OUTCOME eligible US adults, may prevent many cardiovascular disease events, cardiovascular and total deaths, as well as heart failure hospitalizations.

An economic analysis of axitinib and sorafenib for second-line treatment of cytokine-refractory patients with advanced renal cell carcinoma in the United States (US).

2013 ◽  
Vol 31 (15_suppl) ◽  
pp. e15601-e15601
Author(s):  
Ipek Özer-Stillman ◽  
Apoorva Ambavane ◽  
Paul Cislo
Keyword(s):  
Economic Analysis ◽  
The United States ◽  
Phase Iii ◽  
Administrative Claims ◽  
Line Treatment ◽  
Second Line ◽  
Cost Difference ◽  
Life Years ◽  
The Us ◽  
The Cost

e15601 Background: Cytokines are a first-line treatment option for a subset of advanced RCC patients in the US. After progression on cytokines, NCCN guidelines recommend targeted agents, such as axitinib and sorafenib. Subgroup analysis of post-cytokine patients in the phase III AXIS trial found that axitinib increased median progression free survival (PFS) compared with sorafenib (12.0 vs. 6.6 months, p<0.0001), while overall survival (OS) showed no difference (29.4 vs. 27.8 months, p=0.144). An economic analysis for this subgroup was conducted from a US healthcare payer perspective. Methods: A cohort partition model with monthly cycles was constructed to estimate direct medical costs and health outcomes, discounted at 3.0% per annum, over cohort lifetime. Patients were apportioned into 3 health states (progression-free, progressed and dead) based on OS and PFS Kaplan-Meier curves for the post-cytokine subgroup in the AXIS trial. Active treatment was applied until progression, followed by best supportive care (BSC) alone thereafter. The wholesale acquisition costs were based from RedBook. Adverse event (AE) management costs were obtained from published studies. AE rates and utility values were informed by the AXIS trial. Administrative claims data from MarketScan Database were analyzed to estimate costs for BSC and routine care of second-line advanced RCC patients. Results: The total per-patient lifetime costs were estimated to be $242,750 for axitinib and $168,880 for sorafenib and most of the cost difference (84%) was due to the higher total medication cost of axitinib. The cost difference was sensitive to dose intensity and length of treatment. The difference in quality-adjusted life-years (QALY) for axitinib versus sorafenib was minor (1.3 versus 1.2) and the incremental cost-effectiveness ratio (ICER) for axitinib compared with sorafenib was $683,209/QALY. Conclusions: For cytokine-refractory advanced RCC patients, axitinib resulted in an ICER > $650,000/QALY versus sorafenib due to high drug costs and lack of OS benefit, indicating that axitinib may not present good value for money as 2nd line treatment when compared to sorafenib in the US.

scholarly journals Coronavirus Disease 2019 Hospitalizations Attributable to Cardiometabolic Conditions in the United States: A Comparative Risk Assessment Analysis

2021 ◽  
Author(s):  
Meghan O’Hearn ◽  
Junxiu Liu ◽  
Frederick Cudhea ◽  
Renata Micha ◽  
Dariush Mozaffarian
Keyword(s):  
Diabetes Mellitus ◽  
Heart Failure ◽  
Risk Assessment ◽  
The United States ◽  
Sensitivity Analyses ◽  
Cardiometabolic Health ◽  
Comparative Risk Assessment ◽  
Surveillance Network ◽  
Comparative Risk ◽  
Race Ethnicity

BACKGROUND Risk of coronavirus disease 2019 (COVID‐19) hospitalization is robustly linked to cardiometabolic health. We estimated the absolute and proportional COVID‐19 hospitalizations in US adults attributable to 4 major US cardiometabolic conditions, separately and jointly, and by race/ethnicity, age, and sex. METHODS AND RESULTS We used the best available estimates of independent associations of cardiometabolic conditions with a risk of COVID‐19 hospitalization; nationally representative data on cardiometabolic conditions from the National Health and Nutrition Examination Survey 2015 to 2018; and US COVID‐19 hospitalizations stratified by age, sex, and race/ethnicity from the Centers for Disease Control and Prevention’s Coronavirus Disease 2019–Associated Hospitalization Surveillance Network database and from the COVID Tracking Project to estimate the numbers and proportions of COVID‐19 hospitalizations attributable to diabetes mellitus, obesity, hypertension, and heart failure. Inputs were combined in a comparative risk assessment framework, with probabilistic sensitivity analyses and 1000 Monte Carlo simulations to jointly incorporate stratum‐specific uncertainties in data inputs. As of November 18, 2020, an estimated 906 849 COVID‐19 hospitalizations occurred in US adults. Of these, an estimated 20.5% (95% uncertainty interval [UIs], 18.9–22.1) of COVID‐19 hospitalizations were attributable to diabetes mellitus, 30.2% (UI, 28.2–32.3) to total obesity (body mass index ≥30 kg/m 2 ), 26.2% (UI, 24.3–28.3) to hypertension, and 11.7% (UI, 9.5–14.1) to heart failure. Considered jointly, 63.5% (UI, 61.6–65.4) or 575 419 (UI, 559 072–593 412) of COVID‐19 hospitalizations were attributable to these 4 conditions. Large differences were seen in proportions of cardiometabolic risk–attributable COVID‐19 hospitalizations by age and race/ethnicity, with smaller differences by sex. CONCLUSIONS A substantial proportion of US COVID‐19 hospitalizations appear attributable to major cardiometabolic conditions. These results can help inform public health prevention strategies to reduce COVID‐19 healthcare burdens.

scholarly journals External validation of a claims-based model to predict left ventricular ejection fraction class in patients with heart failure

PLoS ONE ◽  
2021 ◽  
Vol 16 (6) ◽  
pp. e0252903
Author(s):  
Mufaddal Mahesri ◽  
Kristyn Chin ◽  
Abheenava Kumar ◽  
Aditya Barve ◽  
Rachel Studer ◽  
...  
Keyword(s):  
Heart Failure ◽  
Ejection Fraction ◽  
External Validation ◽  
Model Performance ◽  
Left Ventricular ◽  
Left Ventricular Ejection ◽  
Administrative Claims ◽  
Validation Sample ◽  
Ventricular Ejection Fraction ◽  
Medicare Claims

Background Ejection fraction (EF) is an important prognostic factor in heart failure (HF), but administrative claims databases lack information on EF. We previously developed a model to predict EF class from Medicare claims. Here, we evaluated the performance of this model in an external validation sample of commercial insurance enrollees. Methods Truven MarketScan claims linked to electronic medical records (EMR) data (IBM Explorys) containing EF measurements were used to identify a cohort of US patients with HF between 01-01-2012 and 10-31-2019. By applying the previously developed model, patients were classified into HF with reduced EF (HFrEF) or preserved EF (HFpEF). EF values recorded in EMR data were used to define gold-standard HFpEF (LVEF ≥45%) and HFrEF (LVEF<45%). Model performance was reported in terms of overall accuracy, positive predicted values (PPV), and sensitivity for HFrEF and HFpEF. Results A total of 7,001 HF patients with an average age of 71 years were identified, 1,700 (24.3%) of whom had HFrEF. An overall accuracy of 0.81 (95% CI: 0.80–0.82) was seen in this external validation sample. For HFpEF, the model had sensitivity of 0.96 (95%CI, 0.95–0.97) and PPV of 0.81 (95% CI, 0.81–0.82); while for HFrEF, the sensitivity was 0.32 (95%CI, 0.30–0.34) and PPV was 0.73 (95%CI, 0.69–0.76). These results were consistent with what was previously published in US Medicare claims data. Conclusions The successful validation of the Medicare claims-based model provides evidence that this model may be used to identify patient subgroups with specific EF class in commercial claims databases as well.

Abstract 17185: Modest Performance of Heart Failure Clinical Prediction Models: A Systematic External Validation Study

Circulation ◽  
2018 ◽  
Vol 138 (Suppl_1) ◽  
Author(s):  
Jenica N Upshaw ◽  
Jason Nelson ◽  
Benjamin Wessler ◽  
Benjamin Koethe ◽  
Christine Lundquist ◽  
...  
Keyword(s):  
Heart Failure ◽  
Prediction Models ◽  
External Validation ◽  
Significant Heterogeneity ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Derivation Cohort ◽  
C Statistic ◽  
Improved Model ◽  
Relative Change

Introduction: Most heart failure (HF) clinical prediction models (CPMs] have not been independently externally validated. We sought to test the performance of HF models in a diverse population using a systematic approach. Methods: A systematic review identified CPMs predicting outcomes for patients with HF. Individual patient data from 5 large publicaly available clinical trials enrolling patients with chronic HF were matched to published CPMs based on similarity in populations and available outcome and predictor variables in the clinical trial databases. CPM performance was evaluated for discrimination (c-statistic, % relative change in c-statistic) and calibration (Harrell’s E and E 90 , the mean and the 90% quantile of the error distribution from the smoothed loess observed value) for the original and recalibrated models. Results: Out of 135 HF CPMs reviewed, we identified 45 CPM-trial pairs including 13 unique CPMs. The outcome was mortality for all of the models with a trial match. During external validations, median c-statistic was 0.595 (IQR 0.563 to 0.630) with a median relative decrease in the c-statistic of -57 % (IQR, -49% to -71%) compared to the c-statistic reported in the derivation cohort. Overall, the median Harrell’s E was 0.09 (IQR, 0.04 to 0.135) and E 90 was 0.11 (IQR, 0.07 to 0.21). Recalibration of the intercept and slope led to substantially improved calibration with median change in Harrell’s E of -35% [IQR 0 to -75%] for the intercept and -56% [IQR -17% to -75%] for the intercept and slope. Refitting model covariates improved the median c-statistic by 38% to 0.629 [IQR 0.613 to 0.649]. Conclusion: For HF CPMs, independent external validations demonstrate that CPMs perform significantly worse than originally presented; however with significant heterogeneity. Recalibration of the intercept and slope improved model calibration. These results underscore the need to carefully consider the derivation cohort characteristics when using published CPMs.

scholarly journals Bias Implications of Outcome Misclassification in Observational Studies Evaluating Association Between Treatments and All‐Cause or Cardiovascular Mortality Using Administrative Claims

2020 ◽  
Vol 9 (17) ◽  
Author(s):  
Rishi J. Desai ◽  
Raisa Levin ◽  
Kueiyu Joshua Lin ◽  
Elisabetta Patorno
Keyword(s):  
Diabetes Mellitus ◽  
Heart Failure ◽  
Type 2 Diabetes ◽  
Type 2 Diabetes Mellitus ◽  
Cardiovascular Mortality ◽  
Administrative Claims ◽  
National Death Index ◽  
All Cause Mortality ◽  
Predicted Values

Background The bias implications of outcome misclassification arising from imperfect capture of mortality in claims‐based studies are not well understood. Methods and Results We identified 2 cohorts of patients: (1) type 2 diabetes mellitus (n=8.6 million), and (2) heart failure (n=3.1 million), from Medicare claims (2012–2016). Within the 2 cohorts, mortality was identified from claims using the following approaches: (1) all‐place all‐cause mortality, (2) in‐hospital all‐cause mortality, (3) all‐place cardiovascular mortality (based on diagnosis codes for a major cardiovascular event within 30 days of death date), or (4) in‐hospital cardiovascular mortality, and compared against National Death Index identified mortality. Empirically identified sensitivity and specificity based on observed values in the 2 cohorts were used to conduct Monte Carlo simulations for treatment effect estimation under differential and nondifferential misclassification scenarios. From National Death Index, 1 544 805 deaths (549 996 [35.6%] cardiovascular deaths) in the type 2 diabetes mellitus cohort and 1 175 202 deaths (523 430 [44.5%] cardiovascular deaths) in the heart failure cohort were included. Sensitivity was 99.997% and 99.207% for the all‐place all‐cause mortality approach, whereas it was 27.71% and 33.71% for the in‐hospital all‐cause mortality approach in the type 2 diabetes mellitus and heart failure cohorts, respectively, with perfect positive predicted values. For all‐place cardiovascular mortality, sensitivity was 52.01% in the type 2 diabetes mellitus cohort and 53.83% in the heart failure cohort with positive predicted values of 49.98% and 54.45%, respectively. Simulations suggested a possibility for substantial bias in treatment effects. Conclusions Approaches to identify mortality from claims had variable performance compared with the National Death Index. Investigators should anticipate the potential for bias from outcome misclassification when using administrative claims to capture mortality.

scholarly journals External validation of a claims-based model to predict left ventricular ejection fraction class in patients with heart failure

2020 ◽  
Author(s):  
Mufaddal Mahesri ◽  
Kristyn Chin ◽  
Abheenava Kumar ◽  
Aditya Barve ◽  
Rachel Studer ◽  
...  
Keyword(s):  
Heart Failure ◽  
Ejection Fraction ◽  
External Validation ◽  
Model Performance ◽  
Left Ventricular ◽  
Left Ventricular Ejection ◽  
Administrative Claims ◽  
Validation Sample ◽  
Ventricular Ejection Fraction ◽  
Medicare Claims

ABSTRACTBACKGROUNDEjection fraction (EF) is an important prognostic factor in heart failure (HF), but administrative claims databases lack information on EF. We previously developed a model to predict EF class from Medicare claims. Here, we evaluated the performance of this model in an external validation sample of commercial insurance enrollees.METHODTruven MarketScan claims linked to electronic medical records (EMR) data (IBM Explorys) containing EF measurements were used to identify a cohort of US patients with HF between 01-01-2012 and 10-31-2019. By applying the previously developed model, patients were classified into HF with reduced EF (HFrEF) or preserved EF (HFpEF). EF values recorded in EMR data were used to define gold-standard HFpEF (LVEF ≥45%) and HFrEF (LVEF<45%). Model performance was reported in terms of overall accuracy, positive predicted values (PPV), and sensitivity for HFrEF and HFpEF.RESULTSA total of 7,001 HF patients with an average age of 71 years were identified, 1,700 (24.3%) of whom had HFrEF. An overall accuracy of 0.81 (95% CI: 0.80-0.82) was seen in this external validation sample. For HFpEF, the model had sensitivity of 0.96 (95%CI, 0.95-0.97) and PPV of 0.81 (95% CI, 0.81-0.82); while for HFrEF, the sensitivity was 0.32 (95%CI, 0.30-0.34) and PPV was 0.73 (95%CI, 0.69-0.76). These results were consistent with what was previously published in US Medicare claims data.CONCLUSIONSThe successful validation of the Medicare claims-based model provides evidence that this model may be used to identify patient subgroups with specific EF class in commercial claims databases as well.

Sign in / Sign up

Export Citation Format

Share Document