Health Equity Impact Evaluation of New Treatments – Evidence Synthesis Methods to Overcome Data Gaps

Background: Distributional cost-effectiveness analysis (DCEA) has been introduced as an extension of conventional cost-effectiveness analysis to quantify health equity impacts. Although health disparities are recognized as an important concern, the typical analyses conducted to inform health technology assessment of a new intervention do not include a formal health equity impact evaluation or DCEA. One of the reasons is that the clinical trials for new interventions frequently do not have the power or are not designed to estimate the required treatment effects for sub-populations across which you want to analyze equity. The objective of the paper is to discuss how gaps in evidence regarding equity-relevant subgroup effects for new and existing interventions can potentially be overcome with advanced Bayesian evidence synthesis methods to facilitate a credible model-based DCEA. Methods: First, the evidence needs and challenges for a model-based DCEA are outlined. Next, alternative evidence synthesis methods will be summarized, followed by an illustrative example of implementing these methods. The paper will conclude with some practical recommendations. Results: The key evidence challenges for a DCEA relate to estimating relative treatment effects due to lack of inclusion of relevant subgroups in the randomized controlled trials (RCTs), lack of access to individual patient data (IPD) for all trials, small subgroups resulting in uncertain effects, and reporting gaps. Advanced Bayesian evidence synthesis methods can help overcome evidence gaps by considering all relevant direct, indirect, and external evidence simultaneously. Methods discussed include (network) meta-analysis with shrinkage estimation, conventional (network) meta-regression analysis, multi-level (network) meta-regression analysis, and generalized evidence synthesis. For a new intervention for which only RCT evidence is available and no real-world data, estimates can be improved if the assumption of exchangeable subgroup effects or the shared or exchangeable effect-modifier assumption among competing interventions can be defended. Furthermore, formal expert elicitation is worthwhile to improve estimates. Conclusion: This paper provides an overview of advanced evidence synthesis methods that may help overcome typical gaps in the evidence base to perform model-based DCEA along with some practical recommendations. Future simulation studies are needed to assess the pros and cons of different methods for different data gap scenarios.

An evidence synthesis approach for combining different data sources illustrated using entomological efficacy of insecticides for indoor residual spraying

Background: Prospective malaria public health interventions are initially tested for entomological impact using standardised experimental hut trials. In some cases, data are collated as aggregated counts of potential outcomes from mosquito feeding attempts given the presence of an insecticidal intervention. Comprehensive data i.e. full breakdowns of probable outcomes of mosquito feeding attempts, are more rarely available. Bayesian evidence synthesis is a framework that explicitly combines data sources to enable the joint estimation of parameters and their uncertainties. The aggregated and comprehensive data can be combined using an evidence synthesis approach to enhance our inference about the potential impact of vector control products across different settings over time. Methods: Aggregated and comprehensive data from a meta-analysis of the impact of Actellic 300CS (Syngenta), an indoor residual spray (IRS) product, used on wall surfaces to kill mosquitoes and reduce malaria transmission, were analysed using a series of statistical models to understand the benefits and limitations of each. Results: Many more data are available in aggregated format (N = 23 datasets, 5 studies) relative to comprehensive format (N = 3 datasets, 2 studies). The evidence synthesis model was most robust at predicting the probability of mosquitoes dying or surviving and blood-feeding. Generating odds ratios from the correlated Bernoulli random sample indicates that when mortality and blood-feeding are positively correlated, as exhibited in our data, the number of successfully fed mosquitoes will be under-estimated. Analysis of either dataset alone is problematic because aggregated data require an assumption of independence and there are few and variable data in the comprehensive format. Conclusions: We developed an approach to combine sources from trials to maximise the inference that can be made from such data. Bayesian evidence synthesis enables inference from multiple datasets simultaneously to give a more informative result and highlight conflicts between sources. Advantages and limitations of these models are discussed.

Inferring the COVID-19 IFR with a simple Bayesian evidence synthesis of seroprevalence study data and imprecise mortality data

Estimating the COVID-19 infection fatality rate (IFR) has proven to be particularly challenging --and rather controversial-- due to the fact that both the data on deaths and the data on the number of individuals infected are subject to many different biases. We consider a Bayesian evidence synthesis approach which, while simple enough for researchers to understand and use, accounts for many important sources of uncertainty inherent in both the seroprevalence and mortality data. We estimate the COVID-19 IFR to be 0.38% (95% prediction interval of (0.03%, 1.19%)) for a typical population where the proportion of those aged over 65 years old is 9% (the approximate worldwide value). Our results suggest that, despite immense efforts made to better understand the COVID-19 IFR, there remains a large amount of uncertainty and unexplained heterogeneity surrounding this important statistic.

Exploring Risks of Human Challenge Trials for COVID-19

Human Challenge Trials (HCTs) are a potential method to accelerate development of vaccines and therapeutics. However, HCTs for COVID-19 pose ethical and practical challenges, in part due to the unclear and developing risks. In this paper, we introduce an interactive model for exploring some risks of a SARS-COV-2 dosing study, a prerequisite for any COVID-19 challenge trials. The risk estimates we use are based on a Bayesian evidence synthesis model which can incorporate new data on infection fatality rates (IFRs) to patients, and infer rates of hospitalization. We have also created a web tool to explore risk under different study design parameters and participant scenarios. Finally, we use our model to estimate individual risk, as well as the overall mortality and hospitalization risk in a dosing study.Based on the Bayesian model we expect IFR for someone between 20 and 30 years of age to be 17.5 in 100,000, with 95% uncertainty interval from 12.8 to 23.6. Using this estimate, we find that a simple 50-person dosing trial using younger individuals has a 99.1% (95% CI: 98.8% to 99.4%) probability of no fatalities, and a 92.8% (95% CI: 90.3% to 94.6%) probability of no cases requiring hospitalization. However, this IFR will be reduced in an HCT via screening for comorbidities, as well as providing medical care and aggressive treatment for any cases which occur, so that with stronger assumptions, we project the risk to be as low as 3.1 per 100,000, with a 99.85% (95% CI: 99.7% to 99.9%) chance of no fatalities, and a 98.7% (95% CI: 97.4% to 99.3%) probability of no cases requiring hospitalization.

A Review of Applications of the Bayes Factor in Psychological Research

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian evidence synthesis, Bayesian variable selection and model averaging, and Bayesian evaluation of cognitive models. We elaborate what each application entails, give illustrative examples, and provide an overview of key references and software with links to other applications. The paper is concluded with a discussion of the opportunities and pitfalls of Bayes factor applications and a sketch of corresponding future research lines.