A Unified Survey of Treatment Effect Heterogeneity Modelling and Uplift Modelling

A central question in many fields of scientific research is to determine how an outcome is affected by an action, i.e., to estimate the causal effect or treatment effect of an action. In recent years, in areas such as personalised healthcare, sociology, and online marketing, a need has emerged to estimate heterogeneous treatment effects with respect to individuals of different characteristics. To meet this need, two major approaches have been taken: treatment effect heterogeneity modelling and uplifting modelling. Researchers and practitioners in different communities have developed algorithms based on these approaches to estimate the heterogeneous treatment effects. In this article, we present a unified view of these two seemingly disconnected yet closely related approaches under the potential outcome framework. We provide a structured survey of existing methods following either of the two approaches, emphasising their inherent connections and using unified notation to facilitate comparisons. We also review the main applications of the surveyed methods in personalised marketing, personalised medicine, and sociology. Finally, we summarise and discuss the available software packages and source codes in terms of their coverage of different methods and applicability to different datasets, and we provide general guidelines for method selection.

Assessing effect heterogeneity of a randomized treatment using conditional inference trees

Treatment effect heterogeneity occurs when individual characteristics influence the effect of a treatment. We propose a novel approach that combines prognostic score matching and conditional inference trees to characterize effect heterogeneity of a randomized binary treatment. One key feature that distinguishes our method from alternative approaches is that it controls the Type I error rate, that is, the probability of identifying effect heterogeneity if none exists and retains the underlying subgroups. This feature makes our technique particularly appealing in the context of clinical trials, where there may be significant costs associated with erroneously declaring that effects differ across population subgroups. Treatment effect heterogeneity trees are able to identify heterogeneous subgroups, characterize the relevant subgroups and estimate the associated treatment effects. We demonstrate the efficacy of the proposed method using a comprehensive simulation study and illustrate our method using a nutrition trial dataset to evaluate effect heterogeneity within a patient population.

Does Stigma Moderate the Efficacy of Mental- and Behavioral-Health Interventions? Examining Individual and Contextual Sources of Treatment-Effect Heterogeneity

In this article, we argue that stigma may be an important, but heretofore underrecognized, source of heterogeneity in treatment effects of mental- and behavioral-health interventions. To support this hypothesis, we review recent evidence from randomized controlled trials and spatial meta-analyses suggesting that stigma may predict not only who responds more favorably to these health interventions (i.e., individuals with more stigma experiences), but also the social contexts that are more likely to undermine intervention effects (i.e., communities with greater structural stigma). By highlighting the potential role of personal and contextual stigma in shaping response to interventions, our review paves the way for additional research.

Reappraising Treatment Effect Heterogeneity in Schizophrenia: A Meta-analysis

Objective Determining whether individual patients differ in response to treatment ('treatment effect heterogeneity') is important as it is a prerequisite to developing personalised treatment approaches. Previous variability meta-analyses of response to antipsychotics in schizophrenia found no evidence for treatment effect heterogeneity. Conversely, individual patient data meta-analyses suggest treatment effect heterogeneity does exist. In the current paper we combine individual patient data with study level data to resolve this apparent contradiction and quantitively characterise antipsychotic treatment effect heterogeneity in schizophrenia. Method Individual patient data (IPD) was obtained from the Yale University Open Data Access (YODA) project. Clinical trials were identified in EMBASE, PsycInfo, and PubMed. Treatment effect heterogeneity was estimated from variability ratios derived from study-level data from 66 RCTs of antipsychotics in schizophrenia (N=17,202). This estimation required a correlation coefficient between placebo response and treatment effects to be estimated. We estimated this from both study level estimates of the 66 trials, and individual patient data (N=560). Results Both individual patient (r=-0.32, p=0.002) and study level (r=-0.38, p<0.001) analyses yielded a negative correlation between placebo response and treatment effect. Using these estimates we found evidence of clinically significant treatment effect heterogeneity for total symptoms (our most conservative estimate was SD = 13.5 Positive and Negative Syndrome Scale (PANSS) points). Mean treatment effects were 8.6 points which, given the estimated SD, suggests the top quartile of patients experienced beneficial treatment effects of at least 17.7 PANSS points, while the bottom quartile received no benefit as compared to placebo. Conclusions We found evidence of clinically meaningful treatment effect heterogeneity for antipsychotic treatment of schizophrenia. This suggests efforts to personalise treatment have potential for success, and demonstrates that variability meta-analyses of RCTs need to account for relationships between placebo response and treatment effects.

Inference for the treatment effect in longitudinal cluster randomized trials when treatment effect heterogeneity is ignored

In cluster-randomized trials, sometimes the effect of the intervention being studied differs between clusters, commonly referred to as treatment effect heterogeneity. In the analysis of stepped wedge and cluster-randomized crossover trials, it is possible to include terms in outcome regression models to allow for such treatment effect heterogeneity yet this is not frequently considered. Outside of some simulation studies of specific cases where the outcome is binary, the impact of failing to include terms for treatment effect heterogeneity on the variance of the treatment effect estimator is unknown. We analytically examine the impact of failing to include terms for treatment effect heterogeneity on the variance of the treatment effect estimator, when outcomes are continuous. Using analysis of variance and feasible generalized least squares we provide expressions for this variance. For both the cluster-randomized crossover design and the stepped wedge design, our analytic derivations indicate that failing to include treatment effect heterogeneity results in the estimates for variance of the treatment effect that are too small, leading to inflation of type I error rates. We therefore recommend assessing the sensitivity of sample size calculations and conclusions drawn from the analysis of cluster randomized trials to the inclusion of treatment effect heterogeneity.

The Triangulation WIthin a STudy (TWIST) framework for causal inference within pharmacogenetic research

In this paper we review the methodological underpinnings of the general pharmacogenetic approach for uncovering genetically-driven treatment effect heterogeneity. This typically utilises only individuals who are treated and relies on fairly strong baseline assumptions to estimate what we term the ‘genetically moderated treatment effect’ (GMTE). When these assumptions are seriously violated, we show that a robust but less efficient estimate of the GMTE that incorporates information on the population of untreated individuals can instead be used. In cases of partial violation, we clarify when Mendelian randomization and a modified confounder adjustment method can also yield consistent estimates for the GMTE. A decision framework is then described to decide when a particular estimation strategy is most appropriate and how specific estimators can be combined to further improve efficiency. Triangulation of evidence from different data sources, each with their inherent biases and limitations, is becoming a well established principle for strengthening causal analysis. We call our framework ‘Triangulation WIthin a STudy’ (TWIST)’ in order to emphasise that an analysis in this spirit is also possible within a single data set, using causal estimates that are approximately uncorrelated, but reliant on different sets of assumptions. We illustrate these approaches by re-analysing primary-care-linked UK Biobank data relating to CYP2C19 genetic variants, Clopidogrel use and stroke risk, and data relating to APOE genetic variants, statin use and Coronary Artery Disease.

Reducing Model Misspecification and Bias in the Estimation of Interactions

Analyzing variation in treatment effects across subsets of the population is an important way for social scientists to evaluate theoretical arguments. A common strategy in assessing such treatment effect heterogeneity is to include a multiplicative interaction term between the treatment and a hypothesized effect modifier in a regression model. Unfortunately, this approach can result in biased inferences due to unmodeled interactions between the effect modifier and other covariates, and including these interactions can lead to unstable estimates due to overfitting. In this paper, we explore the usefulness of machine learning algorithms for stabilizing these estimates and show how many off-the-shelf adaptive methods lead to two forms of bias: direct and indirect regularization bias. To overcome these issues, we use a post-double selection approach that utilizes several lasso estimators to select the interactions to include in the final model. We extend this approach to estimate uncertainty for both interaction and marginal effects. Simulation evidence shows that this approach has better performance than competing methods, even when the number of covariates is large. We show in two empirical examples that the choice of method leads to dramatically different conclusions about effect heterogeneity.