A Unified Survey of Treatment Effect Heterogeneity Modelling and Uplift Modelling

A central question in many fields of scientific research is to determine how an outcome is affected by an action, i.e., to estimate the causal effect or treatment effect of an action. In recent years, in areas such as personalised healthcare, sociology, and online marketing, a need has emerged to estimate heterogeneous treatment effects with respect to individuals of different characteristics. To meet this need, two major approaches have been taken: treatment effect heterogeneity modelling and uplifting modelling. Researchers and practitioners in different communities have developed algorithms based on these approaches to estimate the heterogeneous treatment effects. In this article, we present a unified view of these two seemingly disconnected yet closely related approaches under the potential outcome framework. We provide a structured survey of existing methods following either of the two approaches, emphasising their inherent connections and using unified notation to facilitate comparisons. We also review the main applications of the surveyed methods in personalised marketing, personalised medicine, and sociology. Finally, we summarise and discuss the available software packages and source codes in terms of their coverage of different methods and applicability to different datasets, and we provide general guidelines for method selection.

Are the Average Effects of Foster Care Placement Really Close to Zero?

In their provocative article, Barth and colleagues interrogate existing research on a series of claims about the child welfare system. In this reply, we focus on just one of their conclusions: that foster care placement does little, on average, to cause the poor outcomes of children who are ever placed in care. Our argument proceeds in three stages. In the first, we dispute the claim that the average effects of foster care placement on children are “settled” in any scientific sense. In the second, we note that the lack of agreement about what constitutes the appropriate counterfactual makes the idea of average effects of foster care placement in this area problematic. In the third, we problematize the idea that near-zero average effects equate to unimportant effects by showing how different types of effect heterogeneity may lead us to think differently about how the system is working.

The International Monetary Fund’s intervention in education systems and its impact on children’s chances of completing school

Enabling children to acquire an education is one of the most effective means to reduce inequality, poverty, and ill-health globally. While in normal times a government controls its educational policies, during times of macroeconomic instability, that control may shift to supporting international organizations, such as the International Monetary Fund (IMF). While much research has focused on which sectors has been affected by IMF policies, scholars have devoted little attention to the policy content of IMF interventions affecting the education sector and children’s education outcomes: denoted IMF-education policies. This article evaluates the extent which IMF-education policies exist in all programs and how these policies and IMF programs affect children’s likelihood of completing schools. While IMF-education policies have a small adverse effect yet statistically insignificant on children’s probability of completing school, these policies moderate effect heterogeneity for IMF programs. The effect of IMF programs (joint set of policies) adversely effect children’s chances of completing school by six percentage points. By analyzing how IMF-education policies but also how IMF programs affect the education sector in low- and middle-income countries, scholars will gain a deeper understanding of how such policies will likely affect downstream outcomes.

Neighborhood Effects, the Life Course, and Educational Outcomes: Four Theoretical Models of Effect Heterogeneity

In this chapter, I review research analyzing heterogeneity in neighborhood effects on educational attainment. Using a life-course perspective on neighborhood effects, I describe four potential models of effect heterogeneity: cumulative advantage, cumulative disadvantage, advantage leveling, and compensatory advantage. Extant research most thoroughly explores effect heterogeneity by family socioeconomic background with evidence in support of multiple models. Research on secondary outcomes like achievement and dropout finds evidence of a cumulative disadvantage model, whereas research on bachelor’s degree completion finds evidence of an advantage leveling model. Still, scholarship on heterogeneity in neighborhood effects is in its nascency, and I conclude this chapter with several recommendations for future directions in research.

Who benefits most from studying abroad? A conceptual and empirical overview

This editorial to the special issue on heterogeneous effects of studying abroad starts with a review of studies on the determinants and individual-level effects of studying abroad. On that basis, it illustrates the necessity to place more emphasis on effect heterogeneity in research on international student mobility. It then develops a typology of heterogeneous effects of studying abroad, which shall function as an agenda for future research in the field. Thereafter, the editorial introduces the contributions to the special issue. It concludes by summarising major findings and directions for future research.

Assessing effect heterogeneity of a randomized treatment using conditional inference trees

Treatment effect heterogeneity occurs when individual characteristics influence the effect of a treatment. We propose a novel approach that combines prognostic score matching and conditional inference trees to characterize effect heterogeneity of a randomized binary treatment. One key feature that distinguishes our method from alternative approaches is that it controls the Type I error rate, that is, the probability of identifying effect heterogeneity if none exists and retains the underlying subgroups. This feature makes our technique particularly appealing in the context of clinical trials, where there may be significant costs associated with erroneously declaring that effects differ across population subgroups. Treatment effect heterogeneity trees are able to identify heterogeneous subgroups, characterize the relevant subgroups and estimate the associated treatment effects. We demonstrate the efficacy of the proposed method using a comprehensive simulation study and illustrate our method using a nutrition trial dataset to evaluate effect heterogeneity within a patient population.

Does Stigma Moderate the Efficacy of Mental- and Behavioral-Health Interventions? Examining Individual and Contextual Sources of Treatment-Effect Heterogeneity

In this article, we argue that stigma may be an important, but heretofore underrecognized, source of heterogeneity in treatment effects of mental- and behavioral-health interventions. To support this hypothesis, we review recent evidence from randomized controlled trials and spatial meta-analyses suggesting that stigma may predict not only who responds more favorably to these health interventions (i.e., individuals with more stigma experiences), but also the social contexts that are more likely to undermine intervention effects (i.e., communities with greater structural stigma). By highlighting the potential role of personal and contextual stigma in shaping response to interventions, our review paves the way for additional research.

Reappraising Treatment Effect Heterogeneity in Schizophrenia: A Meta-analysis

Objective Determining whether individual patients differ in response to treatment ('treatment effect heterogeneity') is important as it is a prerequisite to developing personalised treatment approaches. Previous variability meta-analyses of response to antipsychotics in schizophrenia found no evidence for treatment effect heterogeneity. Conversely, individual patient data meta-analyses suggest treatment effect heterogeneity does exist. In the current paper we combine individual patient data with study level data to resolve this apparent contradiction and quantitively characterise antipsychotic treatment effect heterogeneity in schizophrenia. Method Individual patient data (IPD) was obtained from the Yale University Open Data Access (YODA) project. Clinical trials were identified in EMBASE, PsycInfo, and PubMed. Treatment effect heterogeneity was estimated from variability ratios derived from study-level data from 66 RCTs of antipsychotics in schizophrenia (N=17,202). This estimation required a correlation coefficient between placebo response and treatment effects to be estimated. We estimated this from both study level estimates of the 66 trials, and individual patient data (N=560). Results Both individual patient (r=-0.32, p=0.002) and study level (r=-0.38, p<0.001) analyses yielded a negative correlation between placebo response and treatment effect. Using these estimates we found evidence of clinically significant treatment effect heterogeneity for total symptoms (our most conservative estimate was SD = 13.5 Positive and Negative Syndrome Scale (PANSS) points). Mean treatment effects were 8.6 points which, given the estimated SD, suggests the top quartile of patients experienced beneficial treatment effects of at least 17.7 PANSS points, while the bottom quartile received no benefit as compared to placebo. Conclusions We found evidence of clinically meaningful treatment effect heterogeneity for antipsychotic treatment of schizophrenia. This suggests efforts to personalise treatment have potential for success, and demonstrates that variability meta-analyses of RCTs need to account for relationships between placebo response and treatment effects.