Estimation of Average Treatment Effects Using Panel Data when Treatment Effect Heterogeneity Depends on Unobserved Fixed Effects

The extent to which survey experiments conducted with nonrepresentative convenience samples are generalizable to target populations depends critically on the degree of treatment effect heterogeneity. Recent inquiries have found a strong correspondence between sample average treatment effects estimated in nationally representative experiments and in replication studies conducted with convenience samples. We consider here two possible explanations: low levels of effect heterogeneity or high levels of effect heterogeneity that are unrelated to selection into the convenience sample. We analyze subgroup conditional average treatment effects using 27 original–replication study pairs (encompassing 101,745 individual survey responses) to assess the extent to which subgroup effect estimates generalize. While there are exceptions, the overwhelming pattern that emerges is one of treatment effect homogeneity, providing a partial explanation for strong correspondence across both unconditional and conditional average treatment effect estimates.

Download Full-text

Covariance Regression Models for Studying Treatment Effect Heterogeneity Across One or More Outcomes: Understanding How Treatments Shape Inequality

Sociological Methods & Research ◽

10.1177/0049124119882449 ◽

2019 ◽

pp. 004912411988244

Author(s):

Deirdre Bloome ◽

Daniel Schrage

Keyword(s):

Treatment Effect ◽

Regression Models ◽

Treatment Effects ◽

Well Being ◽

Heterogeneous Treatment Effects ◽

Multiple Dimensions ◽

Treatment Effect Heterogeneity ◽

Distributional Consequences ◽

Average Treatment Effects ◽

Effect Heterogeneity

Causal analyses typically focus on average treatment effects. Yet for substantive research on topics like inequality, interest extends to treatments’ distributional consequences. When individuals differ in their responses to treatment, three types of inequality may result. Treatment may shape inequalities between subgroups defined by pretreatment covariates, it may induce more inequality in one subgroup than another, or it may polarize people across multiple dimensions of well-being. We introduce a model, called a covariance regression, that captures all three types of inequality via the means, variances, and correlations between multiple outcomes. The model can test for heterogeneous treatment effects, quantify the heterogeneity, and explain its structure using covariates. Finding that a treatment creates inequalities could drive theoretical refinement and inform policy decisions (targeting groups where payoffs will be most predictable). We illustrate the utility of covariance regressions by analyzing the effects of sharing information about income inequality on redistributive preferences.

Download Full-text

Estimation of time-varying average treatment effects using panel data when unobserved fixed effects affect potential outcomes differently

Economics Letters ◽

10.1016/j.econlet.2016.07.021 ◽

2016 ◽

Vol 146 ◽

pp. 82-84 ◽

Cited By ~ 3

Author(s):

Shosei Sakaguchi

Keyword(s):

Panel Data ◽

Fixed Effects ◽

Treatment Effects ◽

Potential Outcomes ◽

Time Varying ◽

Average Treatment ◽

Average Treatment Effects ◽

Estimation Of Time

Download Full-text

Broken or Fixed Effects?

Journal of Econometric Methods ◽

10.1515/jem-2017-0002 ◽

2018 ◽

Vol 8 (1) ◽

Cited By ~ 13

Author(s):

Charles E. Gibbons ◽

Juan Carlos Suárez Serrato ◽

Michael B. Urbancic

Keyword(s):

Empirical Evidence ◽

Treatment Effect ◽

Fixed Effects ◽

Treatment Effects ◽

Average Treatment Effect ◽

Significant Treatment Effect ◽

Significant Treatment ◽

Heterogeneous Treatment Effects ◽

Treatment Effect Heterogeneity ◽

Effect Heterogeneity

Abstract We replicate eight influential papers to provide empirical evidence that, in the presence of heterogeneous treatment effects, OLS with fixed effects (FE) is generally not a consistent estimator of the average treatment effect (ATE). We propose two alternative estimators that recover the ATE in the presence of group-specific heterogeneity. We document that heterogeneous treatment effects are common and the ATE is often statistically and economically different from the FE estimate. In all but one of our replications, there is statistically significant treatment effect heterogeneity and, in six, the ATEs are either economically or statistically different from the FE estimates.

Download Full-text

The Effect of Extension Services and Credit on Agricultural Production in Bolivia, Peru, and Colombia

10.18235/0003404 ◽

2021 ◽

Author(s):

Mateus C. R. Neves ◽

Felipe De Figueiredo Silva ◽

Carlos Otávio Freitas

Keyword(s):

Travel Time ◽

Treatment Effect ◽

Agricultural Production ◽

Treatment Effects ◽

Geographic Information ◽

Average Treatment Effect ◽

Extension Services ◽

Average Treatment ◽

Andean Countries ◽

Average Treatment Effects

In this paper we estimate the average treatment effect from access to extension services and credit on agricultural production in selected Andean countries (Bolivia, Peru, and Colombia). More specifically, we want to identify the effect of accessibility, here represented as travel time to the nearest area with 1,500 or more inhabitants per square kilometer or at least 50,000 inhabitants, on the likelihood of accessing extension and credit. To estimate the treatment effect and identify the effect of accessibility on these variables, we use data from the Colombian and Bolivian Agricultural Censuses of 2013 and 2014, respectively; a national agricultural survey from 2017 for Peru; and geographic information on travel time. We find that the average treatment effect for extension is higher compared to that of credit for farms in Bolivia and Peru, and lower for Colombia. The average treatment effects of extension and credit for Peruvian farms are $2,387.45 and $3,583.42 respectively. The average treatment effect for extension and credit are $941.92 and $668.69, respectively, while in Colombia are $1,365.98 and $1,192.51, respectively. We also find that accessibility and the likelihood of accessing these services are nonlinearly related. Results indicate that higher likelihood is associated with lower travel time, especially in the analysis of credit.

Download Full-text

Estimating Treatment Effects in the Presence of Noncompliance and Nonresponse: The Generalized Endogenous Treatment Model

Political Analysis ◽

10.1093/pan/mpr005 ◽

2011 ◽

Vol 19 (2) ◽

pp. 205-226 ◽

Cited By ~ 8

Author(s):

Kevin M. Esterling ◽

Michael A. Neblo ◽

David M. J. Lazer

Keyword(s):

Treatment Effect ◽

Instrumental Variable ◽

Treatment Effects ◽

Mean Squared Error ◽

Principal Stratification ◽

Squared Error ◽

Treatment Effect Heterogeneity ◽

Causal Treatment ◽

Effect Heterogeneity ◽

And Control

If ignored, noncompliance with a treatment or nonresponse on outcome measures can bias estimates of treatment effects in a randomized experiment. To identify and estimate causal treatment effects in the case where compliance and response depend on unobservables, we propose the parametric generalized endogenous treatment (GET) model. GET incorporates behavioral responses within an experiment to measure each subject's latent compliance type and identifies causal effects via principal stratification. Using simulation methods and an application to field experimental data, we show GET has a dramatically lower mean squared error for treatment effect estimates than existing approaches to principal stratification that impute, rather than measure, compliance type. In addition, we show that GET allows one to relax and test the instrumental variable exclusion restriction assumption, to test for the presence of treatment effect heterogeneity across a range of compliance types, and to test for treatment ignorability when treatment and control samples are balanced on observable covariates.

Download Full-text

The Relationship between Job Quality and Crime: Examining Heterogeneity in Treatment and Treatment Effect

Journal of Research in Crime and Delinquency ◽

10.1177/0022427820918899 ◽

2020 ◽

Vol 57 (6) ◽

pp. 693-740

Author(s):

Chae M. Jaynes

Keyword(s):

Treatment Effect ◽

Fixed Effects ◽

Job Quality ◽

Specific Effects ◽

Fixed Effects Models ◽

Treatment Effect Heterogeneity ◽

Race Ethnicity ◽

Effect Heterogeneity ◽

The Relationship ◽

Perceptual Measure

Objectives: This study evaluates the relationship between employment and crime through a holistic evaluation of both treatment and treatment effect heterogeneity. Methods: This study implements a perceptual measure of job quality (job satisfaction) and hybrid fixed effects models among a sample of high-risk adults. Analyses also consider the robustness of findings across alternative operationalizations of job quality and various sample subgroups. Results: Transitioning from not working to working in the lowest quality job can be criminogenic. Among those who are working, an improvement in job quality is not generally associated with offending. However, model and crime-specific effects are observed. Evidence of treatment effect heterogeneity is also found, suggesting the effect of job quality is moderated by race/ethnicity and location. Conclusions: These findings caution criminologists against making an assumption that employment is inversely related to offending and call into question our understanding of job quality as a general disincentive for crime. Rather, evidence suggests that improvements in job quality may result in modest reductions in offending, but only for certain types of crime and certain individuals within specific labor market contexts.

Download Full-text