scholarly journals Reconciling Seemingly Contradictory Results from the Oregon Health Insurance Experiment and the Massachusetts Health Reform

2021 ◽  
pp. 1-45
Author(s):  
Amanda E. Kowalski
Keyword(s):  
Health Insurance ◽  
Emergency Room ◽  
Treatment Effect ◽  
Health Reform ◽  
Contradictory Result ◽  
Treatment Effect Heterogeneity ◽  
Oregon Health ◽  
Effect Heterogeneity

Abstract A headline result from the Oregon Health Insurance Experiment is that emergency room (ER) utilization increased. A seemingly contradictory result from the Massachusetts health reform is that ER utilization decreased. I reconcile both results by identifying treatment effect heterogeneity within the Oregon experiment and extrapolating it to Massachusetts. Even though Oregon compliers increased their ER utilization, they were adversely selected relative to Oregon never takers, who would have decreased their ER utilization. Massachusetts expanded coverage from a higher level to healthier compliers. Therefore, Massachusetts compliers are comparable to a subset of Oregon never takers, which can reconcile the results.

scholarly journals Looking Inside the Black Box: The Importance of Causal Mechanism and Treatment Effect Heterogeneity in Experimentally Evaluated Criminal Justice Interventions

2016 ◽  
Vol 31 (1) ◽  
pp. 89-112
Author(s):  
Na Chong Min
Keyword(s):  
Treatment Effect ◽  
Empirical Studies ◽  
Evaluation Research ◽  
Black Box ◽  
Average Treatment Effect ◽  
Causal Mechanism ◽  
Population Average ◽  
Treatment Effect Heterogeneity ◽  
Policies And Programs ◽  
Effect Heterogeneity

This paper discusses limitations of the ???black-box??? experimental archetype by highlighting the narrowness of outcome-focused approaches. For a more complete understanding of the nuanced implications of policies and programs, this study calls for an investigation of causal mechanism and treatment effect heterogeneity in experimentally evaluated interventions. This study draws on two distinct but closely related empirical studies, one undertaken by Na and Paternoster (2012) and the other by Na, Loughran, and Paternoster (2015), that go beyond the estimation of a population average treatment effect by adopting more recent methodological advancements that are still underappreciated and underutilized in evaluation research.

Estimating Treatment Effects in the Presence of Noncompliance and Nonresponse: The Generalized Endogenous Treatment Model

2011 ◽  
Vol 19 (2) ◽  
pp. 205-226 ◽  
Author(s):  
Kevin M. Esterling ◽  
Michael A. Neblo ◽  
David M. J. Lazer
Keyword(s):  
Treatment Effect ◽  
Instrumental Variable ◽  
Treatment Effects ◽  
Mean Squared Error ◽  
Principal Stratification ◽  
Squared Error ◽  
Treatment Effect Heterogeneity ◽  
Causal Treatment ◽  
Effect Heterogeneity ◽  
And Control

If ignored, noncompliance with a treatment or nonresponse on outcome measures can bias estimates of treatment effects in a randomized experiment. To identify and estimate causal treatment effects in the case where compliance and response depend on unobservables, we propose the parametric generalized endogenous treatment (GET) model. GET incorporates behavioral responses within an experiment to measure each subject's latent compliance type and identifies causal effects via principal stratification. Using simulation methods and an application to field experimental data, we show GET has a dramatically lower mean squared error for treatment effect estimates than existing approaches to principal stratification that impute, rather than measure, compliance type. In addition, we show that GET allows one to relax and test the instrumental variable exclusion restriction assumption, to test for the presence of treatment effect heterogeneity across a range of compliance types, and to test for treatment ignorability when treatment and control samples are balanced on observable covariates.

Lessons Learned From Multi-regional Trials With Signals of Treatment Effect Heterogeneity

2020 ◽  
Vol 54 (1) ◽  
pp. 21-31
Author(s):  
Gang Li ◽  
Hui Quan ◽  
Gordon Lan ◽  
Soo Peter Ouyang ◽  
Fei Chen ◽  
...  
Keyword(s):  
Treatment Effect ◽  
Lessons Learned ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

scholarly journals Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach

2018 ◽  
Vol 7 (3) ◽  
pp. 613-628 ◽  
Author(s):  
Alexander Coppock
Keyword(s):  
Treatment Effect ◽  
Experimental Treatment ◽  
Mechanical Turk ◽  
Background Characteristics ◽  
The Public ◽  
Treatment Effect Heterogeneity ◽  
Survey Experiments ◽  
Amazon's Mechanical Turk ◽  
Effect Heterogeneity

To what extent do survey experimental treatment effect estimates generalize to other populations and contexts? Survey experiments conducted on convenience samples have often been criticized on the grounds that subjects are sufficiently different from the public at large to render the results of such experiments uninformative more broadly. In the presence of moderate treatment effect heterogeneity, however, such concerns may be allayed. I provide evidence from a series of 15 replication experiments that results derived from convenience samples like Amazon’s Mechanical Turk are similar to those obtained from national samples. Either the treatments deployed in these experiments cause similar responses for many subject types or convenience and national samples do not differ much with respect to treatment effect moderators. Using evidence of limited within-experiment heterogeneity, I show that the former is likely to be the case. Despite a wide diversity of background characteristics across samples, the effects uncovered in these experiments appear to be relatively homogeneous.

scholarly journals Evaluation of Treatment-Effect Heterogeneity Using Biomarkers Measured on a Continuous Scale: Subpopulation Treatment Effect Pattern Plot

2010 ◽  
Vol 28 (29) ◽  
pp. 4539-4544 ◽  
Author(s):  
Ann A. Lazar ◽  
Bernard F. Cole ◽  
Marco Bonetti ◽  
Richard D. Gelber
Keyword(s):  
Treatment Effect ◽  
Treatment Effectiveness ◽  
Ki 67 ◽  
Breast International Group ◽  
Hormone Receptor Positive ◽  
Continuous Scale ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity ◽  
Continuous Predictors ◽  
To Receive

The discovery of biomarkers that predict treatment effectiveness has great potential for improving medical care, particularly in oncology. These biomarkers are increasingly reported on a continuous scale, allowing investigators to explore how treatment efficacy varies as the biomarker values continuously increase, as opposed to using arbitrary categories of expression levels resulting in a loss of information. In the age of biomarkers as continuous predictors (eg, expression level percentage rather than positive v negative), alternatives to such dichotomized analyses are needed. The purpose of this article is to provide an overview of an intuitive statistical approach—the subpopulation treatment effect pattern plot (STEPP)—for evaluating treatment-effect heterogeneity when a biomarker is measured on a continuous scale. STEPP graphically explores the patterns of treatment effect across overlapping intervals of the biomarker values. As an example, STEPP methodology is used to explore patterns of treatment effect for varying levels of the biomarker Ki-67 in the BIG (Breast International Group) 1-98 randomized clinical trial comparing letrozole with tamoxifen as adjuvant therapy for postmenopausal women with hormone receptor–positive breast cancer. STEPP analyses showed patients with higher Ki-67 values who were assigned to receive tamoxifen had the poorest prognosis and may benefit most from letrozole.

Sign in / Sign up

Export Citation Format

Share Document