scholarly journals Graphing and reporting heterogeneous treatment effects through reference classes

Trials ◽  
2020 ◽  
Vol 21 (1) ◽  
Author(s):  
James A. Watson ◽  
Chris C. Holmes
Keyword(s):  
Predictive Model ◽  
Treatment Effect ◽  
Multiple Testing ◽  
Treatment Effects ◽  
Population Heterogeneity ◽  
Baseline Risk ◽  
Two Stage ◽  
Heterogeneous Treatment Effects ◽  
Reference Classes ◽  
Baseline Covariates

Abstract Background Exploration and modelling of heterogeneous treatment effects as a function of baseline covariates is an important aspect of precision medicine in randomised controlled trials (RCTs). Randomisation generally guarantees the internal validity of an RCT, but heterogeneity in treatment effect can reduce external validity. Estimation of heterogeneous treatment effects is usually done via a predictive model for individual outcomes, where one searches for interactions between treatment allocation and important patient baseline covariates. However, such models are prone to overfitting and multiple testing and typically demand a transformation of the outcome measurement, for example, from the absolute risk in the original RCT to log-odds of risk in the predictive model. Methods We show how reference classes derived from baseline covariates can be used to explore heterogeneous treatment effects via a two-stage approach. We first estimate a risk score which captures on a single dimension some of the heterogeneity in outcomes of the trial population. Heterogeneity in the treatment effect can then be explored via reweighting schemes along this axis of variation. This two-stage approach bypasses the search for interactions with multiple covariates, thus protecting against multiple testing. It also allows for exploration of heterogeneous treatment effects on the original outcome scale of the RCT. This approach would typically be applied to multivariable models of baseline risk to assess the stability of average treatment effects with respect to the distribution of risk in the population studied. Case study We illustrate this approach using the single largest randomised treatment trial in severe falciparum malaria and demonstrate how the estimated treatment effect in terms of absolute mortality risk reduction increases considerably in higher risk strata. Conclusions ‘Local’ and ‘tilting’ reweighting schemes based on ranking patients by baseline risk can be used as a general approach for exploring, graphing and reporting heterogeneity of treatment effect in RCTs. Trial registration ISRCTN clinical trials registry: ISRCTN50258054. Prospectively registered on 22 July 2005.

scholarly journals Graphing and reporting heterogeneous treatment effects through reference classes

2019 ◽  
Author(s):  
James A. Watson ◽  
Chris C. Holmes
Keyword(s):  
Predictive Model ◽  
Treatment Effect ◽  
Multiple Testing ◽  
Treatment Effects ◽  
Outcome Measurement ◽  
Background Information ◽  
Baseline Risk ◽  
Individual Treatment ◽  
Heterogeneous Treatment Effects ◽  
Reference Classes

AbstractBackgroundExploration and modelling of individual treatment effects and treatment heterogeneity is an important aspect of precision medicine in randomized controlled trials (RCTs). The usual approach is to develop a predictive model for individual outcomes and then look for an interaction effect between treatment allocation and important patient covariates. However, such models are prone to overfitting and multiple testing, and typically demand a transformation of the outcome measurement, for example, from the absolute risk in the original RCT to log-odds of risk in the predictive model.MethodsWe show how reference classes derived from background information can be used to alleviate this problem through a two-stage approach where we first estimate a key aspect of heterogeneity in the trial population and then explore for an interaction with the treatment effect along this axis of variation. This bypasses the search for interactions, protecting against multiple testing, and allows for exploration of heterogeneous treatment effects on the original outcome scale of the RCT. This would typically be applied to multivariate models of baseline risk to assess the stability of average treatment effects with respect to the distribution of risk in the population studied. We show how ‘local’ and ‘tilting’ schemes based on ranking patients by baseline risk can be used as a general approach for exploring heterogeneity of treatment effect.ResultsWe illustrate this approach using the single largest randomised treatment trial in severe falciparum malaria and show how the estimated treatment effect in terms of absolute mortality risk reduction increases considerably for higher risk strata.

scholarly journals Flexible Estimation of Treatment Effect Parameters

2011 ◽  
Vol 101 (3) ◽  
pp. 544-551 ◽  
Author(s):  
Thomas MaCurdy ◽  
Xiaohong Chen ◽  
Han Hong
Keyword(s):  
Treatment Effect ◽  
Treatment Effects ◽  
Cell Structure ◽  
Structure Identification ◽  
Heterogeneous Treatment Effects ◽  
Common Cell

A variety of identification strategies have a common cell structure, in which the observed heterogeneity of the regression defines a partition of the sample into cells. Typically in the presence of exogenous covariates that define the cell structure, identification assumptions are imposed conditional on each value of the covariate, or cell by cell. Treatment effects across cells are typically heterogeneous. Researchers might be interested in unconditional parameters which are the averaged treatment effects across the cells. Alternatively, treatment effects can be estimated more efficiently if researchers are willing to impose additional parametric and semiparametric structures on the heterogeneous treatment effects across cells.

scholarly journals A New Framework for Online Testing of Heterogeneous Treatment Effect

2020 ◽  
Vol 34 (06) ◽  
pp. 10310-10317
Author(s):  
Miao Yu ◽  
Wenbin Lu ◽  
Rui Song
Keyword(s):  
Multiple Testing ◽  
Continuous Monitoring ◽  
Treatment Effects ◽  
Type I Error ◽  
Score Test ◽  
P Value ◽  
Type I ◽  
Online Testing ◽  
Heterogeneous Treatment Effects ◽  
New Framework

We propose a new framework for online testing of heterogeneous treatment effects. The proposed test, named sequential score test (SST), is able to control type I error under continuous monitoring and detect multi-dimensional heterogeneous treatment effects. We provide an online p-value calculation for SST, making it convenient for continuous monitoring, and extend our tests to online multiple testing settings by controlling the false discovery rate. We examine the empirical performance of the proposed tests and compare them with a state-of-art online test, named mSPRT using simulations and a real data. The results show that our proposed test controls type I error at any time, has higher detection power and allows quick inference on online A/B testing.

scholarly journals Heterogeneous Treatment Effects in the Presence of Self-Selection: A Propensity Score Perspective

2019 ◽  
Vol 50 (1) ◽  
pp. 350-385 ◽  
Author(s):  
Xiang Zhou ◽  
Yu Xie
Keyword(s):  
Propensity Score ◽  
Treatment Effect ◽  
Latent Variable ◽  
Treatment Effects ◽  
Social Research ◽  
Essential Feature ◽  
Economic Returns ◽  
Heterogeneous Treatment Effects ◽  
Self Selection ◽  
Wide Range

An essential feature common to all empirical social research is variability across units of analysis. Individuals differ not only in background characteristics but also in how they respond to a particular treatment, intervention, or stimulation. Moreover, individuals may self-select into treatment on the basis of anticipated treatment effects. To study heterogeneous treatment effects in the presence of self-selection, Heckman and Vytlacil developed a structural approach that builds on the marginal treatment effect (MTE). In this article, we extend the MTE-based approach through a redefinition of MTE. Specifically, we redefine MTE as the expected treatment effect conditional on the propensity score (rather than all observed covariates) as well as a latent variable representing unobserved resistance to treatment. As with the original MTE, the new MTE also can be used as a building block for evaluating standard causal estimands. However, the weights associated with the new MTE are simpler, more intuitive, and easier to compute. Moreover, the new MTE is a bivariate function and thus is easier to visualize than the original MTE. Finally, the redefined MTE immediately reveals treatment-effect heterogeneity among individuals who are at the margin of treatment. As a result, it can be used to evaluate a wide range of policy changes with little analytical twist and design policy interventions that optimize the marginal benefits of treatment. We illustrate the proposed method by estimating heterogeneous economic returns to college with National Longitudinal Study of Youth 1979 data.

scholarly journals A Unified Survey of Treatment Effect Heterogeneity Modelling and Uplift Modelling

2022 ◽  
Vol 54 (8) ◽  
pp. 1-36
Author(s):  
Weijia Zhang ◽  
Jiuyong Li ◽  
Lin Liu
Keyword(s):  
Treatment Effect ◽  
Treatment Effects ◽  
Causal Effect ◽  
Personalised Medicine ◽  
Online Marketing ◽  
Heterogeneous Treatment Effects ◽  
Source Codes ◽  
Outcome Framework ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

A central question in many fields of scientific research is to determine how an outcome is affected by an action, i.e., to estimate the causal effect or treatment effect of an action. In recent years, in areas such as personalised healthcare, sociology, and online marketing, a need has emerged to estimate heterogeneous treatment effects with respect to individuals of different characteristics. To meet this need, two major approaches have been taken: treatment effect heterogeneity modelling and uplifting modelling. Researchers and practitioners in different communities have developed algorithms based on these approaches to estimate the heterogeneous treatment effects. In this article, we present a unified view of these two seemingly disconnected yet closely related approaches under the potential outcome framework. We provide a structured survey of existing methods following either of the two approaches, emphasising their inherent connections and using unified notation to facilitate comparisons. We also review the main applications of the surveyed methods in personalised marketing, personalised medicine, and sociology. Finally, we summarise and discuss the available software packages and source codes in terms of their coverage of different methods and applicability to different datasets, and we provide general guidelines for method selection.

scholarly journals Correction to: Graphing and reporting heterogeneous treatment effects through reference classes

Trials ◽  
2020 ◽  
Vol 21 (1) ◽  
Author(s):  
James A. Watson ◽  
Chris C. Holmes
Keyword(s):  
Treatment Effects ◽  
Heterogeneous Treatment Effects ◽  
Reference Classes

An amendment to this paper has been published and can be accessed via the original article.

Covariance Regression Models for Studying Treatment Effect Heterogeneity Across One or More Outcomes: Understanding How Treatments Shape Inequality

2019 ◽  
pp. 004912411988244
Author(s):  
Deirdre Bloome ◽  
Daniel Schrage
Keyword(s):  
Treatment Effect ◽  
Regression Models ◽  
Treatment Effects ◽  
Well Being ◽  
Heterogeneous Treatment Effects ◽  
Multiple Dimensions ◽  
Treatment Effect Heterogeneity ◽  
Distributional Consequences ◽  
Average Treatment Effects ◽  
Effect Heterogeneity

Causal analyses typically focus on average treatment effects. Yet for substantive research on topics like inequality, interest extends to treatments’ distributional consequences. When individuals differ in their responses to treatment, three types of inequality may result. Treatment may shape inequalities between subgroups defined by pretreatment covariates, it may induce more inequality in one subgroup than another, or it may polarize people across multiple dimensions of well-being. We introduce a model, called a covariance regression, that captures all three types of inequality via the means, variances, and correlations between multiple outcomes. The model can test for heterogeneous treatment effects, quantify the heterogeneity, and explain its structure using covariates. Finding that a treatment creates inequalities could drive theoretical refinement and inform policy decisions (targeting groups where payoffs will be most predictable). We illustrate the utility of covariance regressions by analyzing the effects of sharing information about income inequality on redistributive preferences.

Broken or Fixed Effects?

2018 ◽  
Vol 8 (1) ◽  
Author(s):  
Charles E. Gibbons ◽  
Juan Carlos Suárez Serrato ◽  
Michael B. Urbancic
Keyword(s):  
Empirical Evidence ◽  
Treatment Effect ◽  
Fixed Effects ◽  
Treatment Effects ◽  
Average Treatment Effect ◽  
Significant Treatment Effect ◽  
Significant Treatment ◽  
Heterogeneous Treatment Effects ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

Abstract We replicate eight influential papers to provide empirical evidence that, in the presence of heterogeneous treatment effects, OLS with fixed effects (FE) is generally not a consistent estimator of the average treatment effect (ATE). We propose two alternative estimators that recover the ATE in the presence of group-specific heterogeneity. We document that heterogeneous treatment effects are common and the ATE is often statistically and economically different from the FE estimate. In all but one of our replications, there is statistically significant treatment effect heterogeneity and, in six, the ATEs are either economically or statistically different from the FE estimates.

Sign in / Sign up

Export Citation Format

Share Document