Evidence Synthesis Methods to Estimate Subgroup Effects for Distributional Cost-effectiveness Analysis of New Treatments

Distributional cost-effectiveness analysis (DCEA) is an extension of conventional cost-effectiveness analysis to quantify health equity impacts. Although health disparities are recognized as an important concern, the typical analyses conducted to inform health technology assessment of a new intervention do not include a DCEA. One of the reasons brought forward is the relative sparseness of the available evidence for a new intervention. The objective of this paper is to review advanced evidence synthesis methods to estimate subgroup specific treatment effects relevant for a DCEA of new interventions. The paper will outline the evidence needs and gaps, present alternative evidence synthesis methods followed by an illustrative example, and conclude with some practical recommendations. Evidence challenges for estimating relative treatment effects are due to lack of inclusion of relevant subgroups in the randomized controlled trials (RCTs), lack of access to individual patient data, small subgroups resulting in uncertain effects, and reporting gaps. Evidence synthesis methods can help overcome evidence gaps by considering all relevant direct, indirect, and external evidence simultaneously. Methods of potential relevance include (network) meta-analysis with shrinkage estimation, conventional (network) meta-regression analysis, multi-level (network) meta-regression analysis, and generalized evidence synthesis. For a new intervention for which only RCT evidence is available and no real-world data, estimates can be improved if the assumption of exchangeable subgroup effects or the shared or exchangeable effect-modifier assumption among competing interventions can be defended. Future research is needed to assess the pros and cons of different methods for different data gap scenarios.

Evidence Synthesis Methods to Estimate Subgroup Effects for Distributional Cost-effectiveness Analysis of New Treatments

Distributional cost-effectiveness analysis (DCEA) is an extension of conventional cost-effectiveness analysis to quantify health equity impacts. Although health disparities are recognized as an important concern, the typical analyses conducted to inform health technology assessment of a new intervention do not include a DCEA. One of the reasons brought forward is the relative sparseness of the available evidence for a new intervention. The objective of this paper is to review advanced evidence synthesis methods to estimate subgroup specific treatment effects relevant for a DCEA of new interventions. The paper will outline the evidence needs and gaps, present alternative evidence synthesis methods followed by an illustrative example, and conclude with some practical recommendations. Evidence challenges for estimating relative treatment effects are due to lack of inclusion of relevant subgroups in the randomized controlled trials (RCTs), lack of access to individual patient data, small subgroups resulting in uncertain effects, and reporting gaps. Evidence synthesis methods can help overcome evidence gaps by considering all relevant direct, indirect, and external evidence simultaneously. Methods of potential relevance include (network) meta-analysis with shrinkage estimation, conventional (network) meta-regression analysis, multi-level (network) meta-regression analysis, and generalized evidence synthesis. For a new intervention for which only RCT evidence is available and no real-world data, estimates can be improved if the assumption of exchangeable subgroup effects or the shared or exchangeable effect-modifier assumption among competing interventions can be defended. Future research is needed to assess the pros and cons of different methods for different data gap scenarios.

Health Equity Impact Evaluation of New Treatments – Evidence Synthesis Methods to Overcome Data Gaps

Background: Distributional cost-effectiveness analysis (DCEA) has been introduced as an extension of conventional cost-effectiveness analysis to quantify health equity impacts. Although health disparities are recognized as an important concern, the typical analyses conducted to inform health technology assessment of a new intervention do not include a formal health equity impact evaluation or DCEA. One of the reasons is that the clinical trials for new interventions frequently do not have the power or are not designed to estimate the required treatment effects for sub-populations across which you want to analyze equity. The objective of the paper is to discuss how gaps in evidence regarding equity-relevant subgroup effects for new and existing interventions can potentially be overcome with advanced Bayesian evidence synthesis methods to facilitate a credible model-based DCEA. Methods: First, the evidence needs and challenges for a model-based DCEA are outlined. Next, alternative evidence synthesis methods will be summarized, followed by an illustrative example of implementing these methods. The paper will conclude with some practical recommendations. Results: The key evidence challenges for a DCEA relate to estimating relative treatment effects due to lack of inclusion of relevant subgroups in the randomized controlled trials (RCTs), lack of access to individual patient data (IPD) for all trials, small subgroups resulting in uncertain effects, and reporting gaps. Advanced Bayesian evidence synthesis methods can help overcome evidence gaps by considering all relevant direct, indirect, and external evidence simultaneously. Methods discussed include (network) meta-analysis with shrinkage estimation, conventional (network) meta-regression analysis, multi-level (network) meta-regression analysis, and generalized evidence synthesis. For a new intervention for which only RCT evidence is available and no real-world data, estimates can be improved if the assumption of exchangeable subgroup effects or the shared or exchangeable effect-modifier assumption among competing interventions can be defended. Furthermore, formal expert elicitation is worthwhile to improve estimates. Conclusion: This paper provides an overview of advanced evidence synthesis methods that may help overcome typical gaps in the evidence base to perform model-based DCEA along with some practical recommendations. Future simulation studies are needed to assess the pros and cons of different methods for different data gap scenarios.

Subgroup Analysis in Pulmonary Hypertension-specific Therapy Trials: a Systematic Review

Background. Pulmonary hypertension (PH) treatment decisions are driven by randomized controlled trials (RCTs) results. Subgroup analyses are often performed to assess whether the intervention effect will change due to the patient’s characteristics. As subgroup claims may mislead clinician treatment decisions, there is a need for standards of such analyses.Objective. To evaluate the appropriateness and interpretation of subgroup analysis performed in pulmonary hypertension-specific therapy RCTs.Methods. A systematic review of the literature for pulmonary hypertension-specific therapy RCTs published between January 2000 and December 2020 was conducted. Claims of subgroup effects were evaluated with Sun X et al., 2012 criteria.Results. 30 RCTs were included. Evaluated subgroup analyses presented: a high number of subgroup analyses reported, lack of prespecification, and interaction test. The trial protocol was not available for most RCTs; significant differences were found in those articles which published the protocol. Authors reported 13 claims of subgroup effect, with 12 claims meeting 4 or fewer Sun criteria. Conclusion. Subgroup analyses in pulmonary hypertension-specific therapies are of poor quality. The lack of published protocols limited our capability to assess whether the published results correspond to the initially predefined analyses. Most claims of subgroup effect did not meet critical criteria.

Metformin in Patients With COVID-19: A Systematic Review and Meta-Analysis

Importance/Background: The coronavirus disease (COVID-19) pandemic is a critical public health issue. Evidence has shown that metformin favorably influences COVID-19 outcomes. This study aimed to assess the benefits and risks of metformin in COVID-19 patients.Methods: We searched the PubMed, Embase, Cochrane Library, and Chinese Biomedical Literature Database from inception to February 18, 2021. Observational studies assessing the association between metformin use and the outcomes of COVID-19 patients were included. The primary outcome was mortality, and the secondary outcomes included intubation, deterioration, and hospitalization. Random-effects weighted models were used to pool the specific effect sizes. Subgroup analyses were conducted by stratifying the meta-analysis by region, diabetic status, the adoption of multivariate model, age, risk of bias, and timing for adding metformin.Results: We identified 28 studies with 2,910,462 participants. Meta-analysis of 19 studies showed that metformin is associated with 34% lower COVID-19 mortality [odds ratio (OR), 0.66; 95% confidence interval (CI), 0.56–0.78; I2 = 67.9%] and 27% lower hospitalization rate (pooled OR, 0.73; 95% CI, 0.53–1.00; I2 = 16.8%). However, we did not identify any subgroup effects. The meta-analysis did not identify statistically significant association between metformin and intubation and deterioration of COVID-19 (OR, 0.94; 95% CI, 0.77–1.16; I2 = 0.0% for intubation and OR, 2.04; 95% CI, 0.65–6.34; I2 = 79.4% for deterioration of COVID-19), respectively.Conclusions: Metformin use among COVID-19 patients was associated with a reduced risk of mortality and hospitalization. Our findings suggest a relative benefit for metformin use in nursing home and hospitalized COVID-19 patients. However, randomized controlled trials are warranted to confirm the association between metformin use and COVID-19 outcomes.Study Registration: The study was registered on the PROSPERO on Feb 23, 2021 (CRD42021238722).

Improving the Estimation of Subgroup Effects for Clinical Trial Participants with Multimorbidity by Incorporating Drug Class-Level Information in Bayesian Hierarchical Models: A Simulation Study

Background There is limited guidance for using common drug therapies in the context of multimorbidity. In part, this is because their effectiveness for patients with specific comorbidities cannot easily be established using subgroup analyses in clinical trials. Here, we use simulations to explore the feasibility and implications of concurrently estimating effects of related drug treatments in patients with multimorbidity by partially pooling subgroup efficacy estimates across trials. Methods We performed simulations based on the characteristics of 161 real clinical trials of noninsulin glucose-lowering drugs for diabetes, estimating subgroup effects for patients with a hypothetical comorbidity across related trials in different scenarios using Bayesian hierarchical generalized linear models. We structured models according to an established ontology—the World Health Organization Anatomic Chemical Therapeutic Classifications—allowing us to nest all trials within drugs and all drugs within anatomic chemical therapeutic classes, with effects partially pooled at each level of the hierarchy. In a range of scenarios, we compared the performance of this model to random effects meta-analyses of all drugs individually. Results Hierarchical, ontology-based Bayesian models were unbiased and accurately recovered simulated comorbidity-drug interactions. Compared with single-drug meta-analyses, they offered a relative increase in precision of up to 250% in some scenarios because of information sharing across the hierarchy. Because of the relative precision of the approaches, a large proportion of small subgroup effects was detectable only using the hierarchical model. Conclusions By assuming that similar drugs may have similar subgroup effects, Bayesian hierarchical models based on structures defined by existing ontologies can be used to improve the precision of treatment efficacy estimates in patients with multimorbidity, with potential implications for clinical decision making.

Diabetes in Patients With Heart Failure With Reduced Ejection Fraction During Hospitalization: A Retrospective Observational Study

BackgroundDiabetes is prevalent worldwide including hospitalized patients with heart failure with reduced ejection fraction (HFrEF). This retrospective study investigated the association of diabetes with in-hospital adverse events in patients with HFrEF.MethodsWe analyzed data from electronic medical records of patients hospitalized with HFrEF in West China Hospital of Sichuan University from January 1, 2011, to September 30, 2018. Propensity score matching balances the baseline characteristics between patients with and without diabetes. Logistic and Poisson regressions investigated the association of diabetes with risks of intubation, cardiogenic shock, acute kidney injury (AKI), intensive care unit (ICU) admission and death during hospitalization, and length of ICU and hospital stay in the matched cases.ResultsAmong 6,022 eligible patients (including 1,998 with diabetes), 1,930 patient pairs with and without diabetes were included by propensity score matching. Patients with diabetes had a significantly increased risk of intubation (odds ratio [OR], 2.69; 95% confidence interval [CI], 2.25–3.22; P<0.001), cardiogenic shock (OR, 2.01; 95% CI, 1.72–2.35; P<0.001), AKI at any stage (OR, 1.67; 95% CI, 1.44–1.94; P<0.001), ICU admission (OR, 1.89; 95% CI, 1.65–2.15; P<0.001), and death (OR, 4.25; 95% CI, 3.06–6.02; P<0.001) during hospitalization. Patients with diabetes had longer ICU (median difference, 1.47 days; 95% CI, 0.96–2.08; P<0.001) and hospital stay (2.20 days; 95% CI, 1.43–2.86; P<0.001) than those without diabetes. There were potential subgroup effects by age and by hypertension, and CKD status on the association of diabetes with risk of AKI at any stage; and subgroup effects by sex and CKD status on the association of diabetes with risk of intubation. The increase in length of hospital stay was larger in patients without hypertension than those with hypertension.ConclusionsAmong patients with HFrEF, those with diabetes have a worse prognosis, including a higher risk of in-hospital intubation, cardiogenic shock, AKI, ICU admission and death during hospitalization, and longer ICU and hospital stay.

Frequentist rules for regulatory approval of subgroups in phase III trials: A fresh look at an old problem

Background The number of Phase III trials that include a biomarker in design and analysis has increased due to interest in personalised medicine. For genetic mutations and other predictive biomarkers, the trial sample comprises two subgroups, one of which, say [Formula: see text] is known or suspected to achieve a larger treatment effect than the other [Formula: see text]. Despite treatment effect heterogeneity, trials often draw patients from both subgroups, since the lower responding [Formula: see text] subgroup may also gain benefit from the intervention. In this case, regulators/commissioners must decide what constitutes sufficient evidence to approve the drug in the [Formula: see text] population. Methods and Results Assuming trial analysis can be completed using generalised linear models, we define and evaluate three frequentist decision rules for approval. For rule one, the significance of the average treatment effect in [Formula: see text] should exceed a pre-defined minimum value, say [Formula: see text]. For rule two, the data from the low-responding group [Formula: see text] should increase statistical significance. For rule three, the subgroup-treatment interaction should be non-significant, using type I error chosen to ensure that estimated difference between the two subgroup effects is acceptable. Rules are evaluated based on conditional power, given that there is an overall significant treatment effect. We show how different rules perform according to the distribution of patients across the two subgroups and when analyses include additional (stratification) covariates in the analysis, thereby conferring correlation between subgroup effects. Conclusions When additional conditions are required for approval of a new treatment in a lower response subgroup, easily applied rules based on minimum effect sizes and relaxed interaction tests are available. Choice of rule is influenced by the proportion of patients sampled from the two subgroups but less so by the correlation between subgroup effects.

Integrating expert opinions with clinical trial data to analyse low-powered subgroup analyses: a Bayesian analysis of the VeRDiCT trial

BackgroundTypically, subgroup analyses in clinical trials are conducted by comparing the intervention effect in each subgroup by means of an interaction test. However, trials are rarely, if ever, adequately powered for interaction tests, so clinically important interactions may go undetected. We discuss the application of Bayesian methods by using expert opinions alongside the trial data. We applied this methodology to the VeRDiCT trial investigating the effect of preoperative volume replacement therapy (VRT) versus no VRT (usual care) in diabetic patients undergoing cardiac surgery. Two subgroup effects were of clinical interest, a) preoperative renal failure and b) preoperative type of antidiabetic medication. MethodsClinical experts were identified within the VeRDiCT trial centre in the UK. A questionnaire was designed to elicit opinions on the impact of VRT on the primary outcome of time from surgery until medically fit for hospital discharge, in the different subgroups. Prior beliefs of the subgroup effect of VRT were elicited face-to-face using two unconditional and one conditional questions per subgroup analysis. The robustness of results to the ‘community of priors’ was assessed. The community of priors was built using the expert priors for the mean average treatment effect, the interaction effect or both in a Bayesian Cox proportional hazards model implemented in the STAN software in R. ResultsExpert opinions were obtained from 7 clinicians (6 cardiac surgeons and 1 cardiac anaesthetist). Participating experts believed VRT could reduce the length of recovery compared to usual care and the greatest benefit was expected in the subgroups with the more severe comorbidity. The Bayesian posterior estimates were more precise compared to the frequentist maximum likelihood estimate and were shifted toward the overall mean treatment effect. Conclusions In the VeRDiCT trial, the Bayesian analysis did not provide evidence of a difference in treatment effect across subgroups. However, this approach increased the precision of the estimated subgroup effects and produced more stable treatment effect point estimates than the frequentist approach. Trial methodologists are encouraged to prospectively consider Bayesian subgroup analyses when low-powered interaction tests are planned.Trial registration ISRCTN, ISRCTN02159606. Registered 29th October 2008.