Contrasting approaches for addressing non-adherence in randomized controlled trials: An illustration from the REFLUX trial

Background: In many randomized controlled trials, a substantial proportion of patients do not comply with the treatment protocol to which they have been randomly assigned. Randomized controlled trials are required to report results according to the intention-to-treat estimand, but recent methodological guidance recognizes the importance of estimating other causal quantities. Methods: This article outlines an analytical framework for randomized controlled trials with non-compliance. We apply the ICH E9 (R1) addendum and combine it with the potential outcomes framework to define key estimands, outline the major assumptions for identification of each estimand, and highlight the assumptions that cannot be verified from the randomized controlled trial data. We contrast the assumptions and estimates in a re-analysis of the REFLUX trial. We report alternative estimates for the effectiveness of receipt of laparoscopic surgery versus medical management for patients with gastro-intestinal reflux disease. Results: The article finds that adjusted as-treated and per-protocol estimates were similar in magnitude to those based intention-to-treat methods. Instrumental variable estimates of the complier average causal effect were larger, with wider confidence intervals. Conclusion: We recommend that in randomized controlled trials with non-compliance, studies should outline which estimand is most relevant to the study context, evaluate key assumptions, and present estimates from a range of methods as a sensitivity analysis.

Azithromycin distribution and childhood mortality in compliance-related subgroups in Niger: complier average causal effect and spillovers in a cluster-randomized, placebo-controlled trial

Background Biannual azithromycin distribution to children 1–59 months old reduced all-cause mortality by 18% [incidence rate ratio (IRR) 0.82, 95% confidence interval (CI): 0.74, 0.90] in an intention-to-treat analysis of a randomized controlled trial in Niger. Estimation of the effect in compliance-related subgroups can support decision making around implementation of this intervention in programmatic settings. Methods The cluster-randomized, placebo-controlled design of the original trial enabled unbiased estimation of the effect of azithromycin on mortality rates in two subgroups: (i) treated children (complier average causal effect analysis); and (ii) untreated children (spillover effect analysis), using negative binomial regression. Results In Niger, 594 eligible communities were randomized to biannual azithromycin or placebo distribution and were followed from December 2014 to August 2017, with a mean treatment coverage of 90% [standard deviation (SD) 10%] in both arms. Subgroup analyses included 2581 deaths among treated children and 245 deaths among untreated children. Among treated children, the incidence rate ratio comparing mortality in azithromycin communities to placebo communities was 0.80 (95% CI: 0.72, 0.88), with mortality rates (deaths per 1000 person-years at risk) of 16.6 in azithromycin communities and 20.9 in placebo communities. Among untreated children, the incidence rate ratio was 0.91 (95% CI: 0.69, 1.21), with rates of 33.6 in azithromycin communities and 34.4 in placebo communities. Conclusions As expected, this analysis suggested similar efficacy among treated children compared with the intention-to-treat analysis. Though the results were consistent with a small spillover benefit to untreated children, this trial was underpowered to detect spillovers.

A Note on Posttreatment Selection in Studying Racial Discrimination in Policing

We discuss some causal estimands that are used to study racial discrimination in policing. A central challenge is that not all police–civilian encounters are recorded in administrative datasets and available to researchers. One possible solution is to consider the average causal effect of race conditional on the civilian already being detained by the police. We find that such an estimand can be quite different from the more familiar ones in causal inference and needs to be interpreted with caution. We propose using an estimand that is new for this context—the causal risk ratio, which has more transparent interpretation and requires weaker identification assumptions. We demonstrate this through a reanalysis of the NYPD Stop-and-Frisk dataset. Our reanalysis shows that the naive estimator that ignores the posttreatment selection in administrative records may severely underestimate the disparity in police violence between minorities and whites in these and similar data.

Causal judgment in the wild: evidence from the 2020 US presidential election

A given event has many causes, but people intuitively view some causes as more important than others. Models of causal judgment have been evaluated in controlled laboratory experiments, but they have yet to be tested in complex real-world settings. Here, we provide such a test, in the context of the 2020 U.S. presidential election. Across tens of thousands of simulations of possible election outcomes, we computed, for each state, an adjusted measure of the correlation between a Biden victory in that state and a Biden election victory. These effect size measures accurately predicted the extent to which U.S. participants (N=207, pre-registered) viewed victory in a given state as having caused Biden to win the presidency. This supports the theory that people intuitively select as causes of an outcome the factors with the largest average causal effect on that outcome across possible counterfactual worlds.

Causal Inference in Studying the Long-term Health Effects of Disasters: Challenges and Potential Solutions

Two frequently encountered but underrecognized challenges for causal inference in studying the long-term health effects of disasters among survivors include: (a) time-varying effects of disasters on a time-to-event outcome and (b) selection bias due to selective attrition. We review approaches to overcome these challenges and show application of the approaches to a real-world longitudinal data of older adults who were directly impacted by the 2011 earthquake and tsunami (n=4,857). To illustrate the problem of time-varying effects of disasters, we examined the association between degree of damage due to the tsunami and all-cause mortality. We compared results from Cox regression assuming proportional hazards versus adjusted parametric survival curves allowing for time-varying hazard ratios. To illustrate the problem of selection bias, we examined the association between proximity to the coast (a proxy for housing damage from the tsunami) and depressive symptoms. We corrected for selection bias due to attrition in the two post-disaster follow-up surveys (conducted in 2013 and 2016) using multivariable adjustment, inverse probability censoring weighting, and survivor average causal effect estimation. Our results demonstrate that the analytic approaches ignoring time-varying effects on mortality and selection bias due to selective attrition may underestimate the long-term health effects of disasters.

Should We or Should We Not Include Confidence Intervals in COVID-19 Death Forecasting? Evidence from a Survey Experiment

Forecasting during the COVID-19 pandemic entails a great deal of uncertainty. The same way that we would like electoral forecasters to systematically include their confidence intervals to account for such uncertainty, we assume that COVID-19-related forecasts should follow that norm. Based on literature on negative bias, we may expect the presence of uncertainty to affect citizens’ attitudes and behaviours, which would in turn have major implications on how we should present these sensitive forecasts. In this research we present the main findings of a survey experiment where citizens were exposed to a projection of the total number of deaths. We manipulated the exclusion (and inclusion) of graphically depicted confidence intervals in order to isolate the average causal effect of uncertainty. Our results show that accounting for uncertainty does not change (1) citizens’ perceptions of projections’ reliability, nor does it affect (2) their support for preventive public health measures. We conclude by discussing the implications of our findings.