Broken or Fixed Effects?

2018 ◽  
Vol 8 (1) ◽  
Author(s):  
Charles E. Gibbons ◽  
Juan Carlos Suárez Serrato ◽  
Michael B. Urbancic
Keyword(s):  
Empirical Evidence ◽  
Treatment Effect ◽  
Fixed Effects ◽  
Treatment Effects ◽  
Average Treatment Effect ◽  
Significant Treatment Effect ◽  
Significant Treatment ◽  
Heterogeneous Treatment Effects ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

Abstract We replicate eight influential papers to provide empirical evidence that, in the presence of heterogeneous treatment effects, OLS with fixed effects (FE) is generally not a consistent estimator of the average treatment effect (ATE). We propose two alternative estimators that recover the ATE in the presence of group-specific heterogeneity. We document that heterogeneous treatment effects are common and the ATE is often statistically and economically different from the FE estimate. In all but one of our replications, there is statistically significant treatment effect heterogeneity and, in six, the ATEs are either economically or statistically different from the FE estimates.

scholarly journals A Unified Survey of Treatment Effect Heterogeneity Modelling and Uplift Modelling

2022 ◽  
Vol 54 (8) ◽  
pp. 1-36
Author(s):  
Weijia Zhang ◽  
Jiuyong Li ◽  
Lin Liu
Keyword(s):  
Treatment Effect ◽  
Treatment Effects ◽  
Causal Effect ◽  
Personalised Medicine ◽  
Online Marketing ◽  
Heterogeneous Treatment Effects ◽  
Source Codes ◽  
Outcome Framework ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

A central question in many fields of scientific research is to determine how an outcome is affected by an action, i.e., to estimate the causal effect or treatment effect of an action. In recent years, in areas such as personalised healthcare, sociology, and online marketing, a need has emerged to estimate heterogeneous treatment effects with respect to individuals of different characteristics. To meet this need, two major approaches have been taken: treatment effect heterogeneity modelling and uplifting modelling. Researchers and practitioners in different communities have developed algorithms based on these approaches to estimate the heterogeneous treatment effects. In this article, we present a unified view of these two seemingly disconnected yet closely related approaches under the potential outcome framework. We provide a structured survey of existing methods following either of the two approaches, emphasising their inherent connections and using unified notation to facilitate comparisons. We also review the main applications of the surveyed methods in personalised marketing, personalised medicine, and sociology. Finally, we summarise and discuss the available software packages and source codes in terms of their coverage of different methods and applicability to different datasets, and we provide general guidelines for method selection.

Covariance Regression Models for Studying Treatment Effect Heterogeneity Across One or More Outcomes: Understanding How Treatments Shape Inequality

2019 ◽  
pp. 004912411988244
Author(s):  
Deirdre Bloome ◽  
Daniel Schrage
Keyword(s):  
Treatment Effect ◽  
Regression Models ◽  
Treatment Effects ◽  
Well Being ◽  
Heterogeneous Treatment Effects ◽  
Multiple Dimensions ◽  
Treatment Effect Heterogeneity ◽  
Distributional Consequences ◽  
Average Treatment Effects ◽  
Effect Heterogeneity

Causal analyses typically focus on average treatment effects. Yet for substantive research on topics like inequality, interest extends to treatments’ distributional consequences. When individuals differ in their responses to treatment, three types of inequality may result. Treatment may shape inequalities between subgroups defined by pretreatment covariates, it may induce more inequality in one subgroup than another, or it may polarize people across multiple dimensions of well-being. We introduce a model, called a covariance regression, that captures all three types of inequality via the means, variances, and correlations between multiple outcomes. The model can test for heterogeneous treatment effects, quantify the heterogeneity, and explain its structure using covariates. Finding that a treatment creates inequalities could drive theoretical refinement and inform policy decisions (targeting groups where payoffs will be most predictable). We illustrate the utility of covariance regressions by analyzing the effects of sharing information about income inequality on redistributive preferences.

scholarly journals Looking Inside the Black Box: The Importance of Causal Mechanism and Treatment Effect Heterogeneity in Experimentally Evaluated Criminal Justice Interventions

2016 ◽  
Vol 31 (1) ◽  
pp. 89-112
Author(s):  
Na Chong Min
Keyword(s):  
Treatment Effect ◽  
Empirical Studies ◽  
Evaluation Research ◽  
Black Box ◽  
Average Treatment Effect ◽  
Causal Mechanism ◽  
Population Average ◽  
Treatment Effect Heterogeneity ◽  
Policies And Programs ◽  
Effect Heterogeneity

This paper discusses limitations of the ???black-box??? experimental archetype by highlighting the narrowness of outcome-focused approaches. For a more complete understanding of the nuanced implications of policies and programs, this study calls for an investigation of causal mechanism and treatment effect heterogeneity in experimentally evaluated interventions. This study draws on two distinct but closely related empirical studies, one undertaken by Na and Paternoster (2012) and the other by Na, Loughran, and Paternoster (2015), that go beyond the estimation of a population average treatment effect by adopting more recent methodological advancements that are still underappreciated and underutilized in evaluation research.

Estimating Treatment Effects in the Presence of Noncompliance and Nonresponse: The Generalized Endogenous Treatment Model

2011 ◽  
Vol 19 (2) ◽  
pp. 205-226 ◽  
Author(s):  
Kevin M. Esterling ◽  
Michael A. Neblo ◽  
David M. J. Lazer
Keyword(s):  
Treatment Effect ◽  
Instrumental Variable ◽  
Treatment Effects ◽  
Mean Squared Error ◽  
Principal Stratification ◽  
Squared Error ◽  
Treatment Effect Heterogeneity ◽  
Causal Treatment ◽  
Effect Heterogeneity ◽  
And Control

If ignored, noncompliance with a treatment or nonresponse on outcome measures can bias estimates of treatment effects in a randomized experiment. To identify and estimate causal treatment effects in the case where compliance and response depend on unobservables, we propose the parametric generalized endogenous treatment (GET) model. GET incorporates behavioral responses within an experiment to measure each subject's latent compliance type and identifies causal effects via principal stratification. Using simulation methods and an application to field experimental data, we show GET has a dramatically lower mean squared error for treatment effect estimates than existing approaches to principal stratification that impute, rather than measure, compliance type. In addition, we show that GET allows one to relax and test the instrumental variable exclusion restriction assumption, to test for the presence of treatment effect heterogeneity across a range of compliance types, and to test for treatment ignorability when treatment and control samples are balanced on observable covariates.

The Relationship between Job Quality and Crime: Examining Heterogeneity in Treatment and Treatment Effect

2020 ◽  
Vol 57 (6) ◽  
pp. 693-740
Author(s):  
Chae M. Jaynes
Keyword(s):  
Treatment Effect ◽  
Fixed Effects ◽  
Job Quality ◽  
Specific Effects ◽  
Fixed Effects Models ◽  
Treatment Effect Heterogeneity ◽  
Race Ethnicity ◽  
Effect Heterogeneity ◽  
The Relationship ◽  
Perceptual Measure

Objectives: This study evaluates the relationship between employment and crime through a holistic evaluation of both treatment and treatment effect heterogeneity. Methods: This study implements a perceptual measure of job quality (job satisfaction) and hybrid fixed effects models among a sample of high-risk adults. Analyses also consider the robustness of findings across alternative operationalizations of job quality and various sample subgroups. Results: Transitioning from not working to working in the lowest quality job can be criminogenic. Among those who are working, an improvement in job quality is not generally associated with offending. However, model and crime-specific effects are observed. Evidence of treatment effect heterogeneity is also found, suggesting the effect of job quality is moderated by race/ethnicity and location. Conclusions: These findings caution criminologists against making an assumption that employment is inversely related to offending and call into question our understanding of job quality as a general disincentive for crime. Rather, evidence suggests that improvements in job quality may result in modest reductions in offending, but only for certain types of crime and certain individuals within specific labor market contexts.

scholarly journals Extending the I-squared statistic to describe treatment effect heterogeneity in cluster, multi-centre randomized trials and individual patient data meta-analysis

2020 ◽  
pp. 096228022094855
Author(s):  
Karla Hemming ◽  
James P Hughes ◽  
Joanne E McKenzie ◽  
Andrew B Forbes
Keyword(s):  
Randomized Trial ◽  
Treatment Effect ◽  
Treatment Effects ◽  
Randomized Trials ◽  
Individual Patient Data ◽  
Meta Analysis ◽  
Cluster Randomized Trial ◽  
Patient Data ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

Treatment effect heterogeneity is commonly investigated in meta-analyses to identify if treatment effects vary across studies. When conducting an aggregate level data meta-analysis it is common to describe the magnitude of any treatment effect heterogeneity using the I-squared statistic, which is an intuitive and easily understood concept. The effect of a treatment might also vary across clusters in a cluster randomized trial, or across centres in multi-centre randomized trial, and it can be of interest to explore this at the analysis stage. In cross-over trials and other randomized designs, in which clusters or centres are exposed to both treatment and control conditions, this treatment effect heterogeneity can be identified. Here we derive and evaluate a comparable I-squared measure to describe the magnitude of heterogeneity in treatment effects across clusters or centres in randomized trials. We further show how this methodology can be used to estimate treatment effect heterogeneity in an individual patient data meta-analysis.

scholarly journals On the treatment effect heterogeneity of antidepressants in major depression. A Bayesian meta-analysis

2020 ◽  
Author(s):  
Constantin Volkmann ◽  
Alexander Volkmann ◽  
Christian A. Müller
Keyword(s):  
Major Depression ◽  
Random Effects ◽  
Treatment Effect ◽  
Clinical Relevance ◽  
Meta Analysis ◽  
Placebo Response ◽  
Average Treatment Effect ◽  
Average Treatment ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

AbstractBackgroundThe average treatment effect of antidepressants in major depression was found to be about 2 points on the 17-item Hamilton Depression Rating Scale, which lies below clinical relevance. Here, we searched for evidence of a relevant treatment effect heterogeneity that could justify the usage of antidepressants despite their low average treatment effect.MethodsBayesian meta-analysis of 169 randomized, controlled trials including 58,687 patients. We considered the effect sizes log variability ratio (lnVR) and log coefficient of variation ratio (lnCVR) to analyze the difference in variability of active and placebo response. We used Bayesian random-effects meta-analyses (REMA) for lnVR and lnCVR and fitted a random-effects meta-regression (REMR) model to estimate the treatment effect variability between antidepressants and placebo.ResultsThe variability ratio was found to be very close to 1 in the best fitting models (REMR: 95% HPD [0.98, 1.02], REMA: 95% HPD [1.00, 1.02]). The between-study variance τ2 under the REMA was found to be low (95% HPD [0.00, 0.00]). Simulations showed that a large treatment effect heterogeneity is only compatible with the data if a strong correlation between placebo response and individual treatment effect is assumed.ConclusionsThe published data from RCTs on antidepressants for the treatment of major depression is compatible with a near-constant treatment effect. Although it is impossible to rule out a substantial treatment effect heterogeneity, its existence seems rather unlikely. Since the average treatment effect of antidepressants falls short of clinical relevance, the current prescribing practice should be re-evaluated.

Abstract P106: Population-Based Drug Repurposing to Treat Thrombosis in COVID-19 Patients

Stroke ◽  
2021 ◽  
Vol 52 (Suppl_1) ◽  
Author(s):  
Yejin Kim ◽  
Luyao Chen ◽  
Xiaoqian Jiang ◽  
Sean I Savitz
Keyword(s):  
Propensity Score ◽  
Treatment Effect ◽  
Treatment Effects ◽  
Average Treatment Effect ◽  
Significant Treatment ◽  
Treatment Conditions ◽  
Increased Risk ◽  
Comparable Control ◽  
Admission Status ◽  
Pre Treatment

Introduction: Patients with coronavirus disease 2019 (COVID-19) have an increased risk of thrombosis. Our objective is to obtain population-level treatment effects of drugs on treating thrombosis in COVID-19. Methods: We conducted a retrospective analysis of Optum electronic health records (EHRs) with 34,043 hospitalized COVID-19 patients. We identified case-patient with thrombosis (stroke, deep vein thrombosis, pulmonary embolism, and myocardial infarction) using PheWas codes. The propensity score matching was used to select comparable control patients who survived without any thrombosis based on demographics and admission status (temperature and SpO 2 level). We computed the average treatment effect (ATT) for medication using advanced inverse propensity score weighting based on pre-treatment conditions (i.e., comorbidities in the last 6 months and medications in the last 2 months before hospitalization). Results: We identified 2,446 case-patients with thrombosis and 5,020 comparable control patients. There were a total of 540 drugs that were administered in at least 80 patients. We calculated the 540 drugs’ ATT coefficient. As a result, 23 drugs had a positive ATT coefficient with a p -value of less than 0.05. After filtering out commonly prescribed symptomatic drugs (e.g., Acetaminophen, Guaifenesin, and Ondansetron), we highlight the following drugs with statistically significant treatment effects: Atorvastatin (ATT=0.34), Ceftriaxone (ATT=0.26), Levothyroxine (ATT=0.26), Albuterol (ATT=0.25), Azithromycin (ATT=0.23), Enoxaparin (ATT=0.20), and Metformin (ATT=0.20). Conclusions: In this preliminary work, we identified anti-thrombotic drugs (Enoxaparin) but also anti-inflammatory drugs (Atorvastatin, Metformin) and possibly antibiotics that have a significant treatment effect in COVID-19 patients that could reduce risk of thrombosis. We also observed that several anti-thrombotic drugs (Apixaban and Ticagrelor) had negative treatment effects, which was partly due to an imbalance in pre-treatment conditions. Our future work is to incorporate more extensive data (such as lab tests and vital signs) into the propensity scores to better capture the severity of admission status.

Sign in / Sign up

Export Citation Format

Share Document