scholarly journals Efficient Discovery of Heterogeneous Treatment Effects in Randomized Experiments via Anomalous Pattern Detection

Author(s):  
Edward McFowland ◽  
Sriram Somanchi ◽  
Daniel Neill
2017 ◽  
Vol 25 (4) ◽  
pp. 413-434 ◽  
Author(s):  
Justin Grimmer ◽  
Solomon Messing ◽  
Sean J. Westwood

Randomized experiments are increasingly used to study political phenomena because they can credibly estimate the average effect of a treatment on a population of interest. But political scientists are often interested in how effects vary across subpopulations—heterogeneous treatment effects—and how differences in the content of the treatment affects responses—the response to heterogeneous treatments. Several new methods have been introduced to estimate heterogeneous effects, but it is difficult to know if a method will perform well for a particular data set. Rather than using only one method, we show how an ensemble of methods—weighted averages of estimates from individual models increasingly used in machine learning—accurately measure heterogeneous effects. Building on a large literature on ensemble methods, we show how the weighting of methods can contribute to accurate estimation of heterogeneous treatment effects and demonstrate how pooling models lead to superior performance to individual methods across diverse problems. We apply the ensemble method to two experiments, illuminating how the ensemble method for heterogeneous treatment effects facilitates exploratory analysis of treatment effects.


2011 ◽  
Vol 19 (1) ◽  
pp. 1-19 ◽  
Author(s):  
Kosuke Imai ◽  
Aaron Strauss

Although a growing number of political scientists are conducting randomized experiments, many of them only report the average treatment effects and do not systematically explore the variation in treatment effects across subpopulations. This is unfortunate from a scientific point of view because heterogeneous treatment effects can provide additional substantive insights. This current state of affairs is also problematic from a policy makers' perspective since such studies do not identify subgroups for which treatments are effective. In this paper, we propose a formal two-step framework that first identifies heterogeneous treatment effects from a randomized experiment and then uses this information to derive an optimal policy about which treatment should be given to whom. Our proposed method avoids the risk of false discoveries that are likely in post hoc subgroup analysis routinely conducted in the discipline. We discuss our methodology in the context of get-out-the-vote randomized field experiments and show how the proposed two-step framework can be applied in real-world settings.


2019 ◽  
Vol 116 (10) ◽  
pp. 4156-4165 ◽  
Author(s):  
Sören R. Künzel ◽  
Jasjeet S. Sekhon ◽  
Peter J. Bickel ◽  
Bin Yu

There is growing interest in estimating and analyzing heterogeneous treatment effects in experimental and observational studies. We describe a number of metaalgorithms that can take advantage of any supervised learning or regression method in machine learning and statistics to estimate the conditional average treatment effect (CATE) function. Metaalgorithms build on base algorithms—such as random forests (RFs), Bayesian additive regression trees (BARTs), or neural networks—to estimate the CATE, a function that the base algorithms are not designed to estimate directly. We introduce a metaalgorithm, the X-learner, that is provably efficient when the number of units in one treatment group is much larger than in the other and can exploit structural properties of the CATE function. For example, if the CATE function is linear and the response functions in treatment and control are Lipschitz-continuous, the X-learner can still achieve the parametric rate under regularity conditions. We then introduce versions of the X-learner that use RF and BART as base learners. In extensive simulation studies, the X-learner performs favorably, although none of the metalearners is uniformly the best. In two persuasion field experiments from political science, we demonstrate how our X-learner can be used to target treatment regimes and to shed light on underlying mechanisms. A software package is provided that implements our methods.


Sign in / Sign up

Export Citation Format

Share Document