Using Threshold Analysis to Assess the Robustness of Public Health Intervention Recommendations from Network Meta-Analyses: Application to Poison Prevention in Households with Children Under Five

BackgroundIn the appraisal of clinical interventions, complex evidence synthesis methods, such as network meta-analysis (NMA), are commonly used to investigate the effectiveness of multiple interventions in a single analysis. The results from a NMA can inform clinical guidelines directly or be used as inputs into a decision-analytic model assessing the cost-effectiveness of the interventions. However, there is hesitancy in using complex evidence synthesis methods when evaluating public health interventions. This is due to significant heterogeneity across studies investigating such interventions and concerns about their quality. Threshold analysis has been developed to help assess and quantify the robustness of recommendations made based on results obtained from NMAs to potential limitations of the data. Developed in the context of clinical guidelines, the method may prove useful also in the context of public health interventions. In this paper, we illustrate the use of the method in the study investigating the effectiveness of interventions aiming to increase the uptake of poison prevention behaviours in homes with children aged 0-5.MethodsRandom effects NMA was carried out to assess the effectiveness of several interventions for increasing the uptake of poison prevention behaviours, focusing on the safe storage of other household products outcome. Threshold analysis was then applied to the NMA to assess the robustness of the intervention recommendations made based on the NMA.Results15 studies assessing seven interventions were included in the NMA. The results of the NMA indicated that complex intervention, including Education, Free/low-cost equipment, Fitting equipment and Home safety inspection, was the most effective intervention at promoting poison prevention behaviours. However, the threshold analyses highlighted that this intervention recommendation was not robust.Conclusions In our case study, threshold analysis allowed us to demonstrate that the intervention recommendation for promoting poison prevention behaviours was not robust to changes in the evidence due to potential bias. Therefore, caution should be taken when considering such interventions in practice. We have illustrated the potential benefit of threshold analysis and, therefore, encourage the use of the method in practice as a sensitivity analysis for NMA of public health interventions.

Economic threshold analysis of delivering a task-sharing treatment for common mental disorders at scale: the Friendship Bench, Zimbabwe

BackgroundTask-sharing treatment approaches offer a pragmatic approach to treating common mental disorders in low-income and middle-income countries (LMICs). The Friendship Bench (FB), developed in Zimbabwe with increasing adoption in other LMICs, is one example of this type of treatment model using lay health workers (LHWs) to deliver treatment.ObjectiveTo consider the level of treatment coverage required for a recent scale-up of the FB in Zimbabwe to be considered cost-effective.MethodsA modelling-based deterministic threshold analysis conducted within a ‘cost-utility’ framework using a recommended cost-effectiveness threshold.FindingsThe FB would need to treat an additional 3413 service users (10 per active LHW per year) for its scale-up to be considered cost-effective. This assumes a level of treatment effect observed under clinical trial conditions. The associated incremental cost-effectiveness ratio was $191 per year lived with disability avoided, assuming treatment coverage levels reported during 2020. The required treatment coverage for a cost-effective outcome is within the level of treatment coverage observed during 2020 and remained so even when assuming significantly compromised levels of treatment effect.ConclusionsThe economic case for a scaled-up delivery of the FB appears convincing in principle and its adoption at scale in LMIC settings should be given serious consideration.Clinical implicationsFurther evidence on the types of scale-up strategies that are likely to offer an effective and cost-effective means of sustaining required levels of treatment coverage will help focus efforts on approaches to scale-up that optimise resources invested in task-sharing programmes.

Quantifying previous SARS-CoV-2 infection through mixture modelling of antibody levels

As countries decide on vaccination strategies and how to ease movement restrictions, estimating the proportion of the population previously infected with SARS-CoV-2 is important for predicting the future burden of COVID-19. This proportion is usually estimated from serosurvey data in two steps: first the proportion above a threshold antibody level is calculated, then the crude estimate is adjusted using external estimates of sensitivity and specificity. A drawback of this approach is that the PCR-confirmed cases used to estimate the sensitivity of the threshold may not be representative of cases in the wider population—e.g., they may be more recently infected and more severely symptomatic. Mixture modelling offers an alternative approach that does not require external data from PCR-confirmed cases. Here we illustrate the bias in the standard threshold-based approach by comparing both approaches using data from several Kenyan serosurveys. We show that the mixture model analysis produces estimates of previous infection that are often substantially higher than the standard threshold analysis.

Probabilistic threshold analysis by pairwise stochastic approximation for decision-making under uncertainty

The concept of probabilistic parameter threshold analysis has recently been introduced as a way of probabilistic sensitivity analysis for decision-making under uncertainty, in particular, for health economic evaluations which compare two or more alternative treatments with consideration of uncertainty on outcomes and costs. In this paper we formulate the probabilistic threshold analysis as a root-finding problem involving the conditional expectations, and propose a pairwise stochastic approximation algorithm to search for the threshold value below and above which the choice of conditionally optimal decision options changes. Numerical experiments for both a simple synthetic testcase and a chemotherapy Markov model illustrate the effectiveness of our proposed algorithm, without any need for accurate estimation or approximation of conditional expectations which the existing approaches rely upon. Moreover we introduce a new measure called decision switching probability for probabilistic sensitivity analysis in this paper.