Information content of stepped‐wedge designs when treatment effect heterogeneity and/or implementation periods are present

2019 ◽  
Vol 38 (23) ◽  
pp. 4686-4701 ◽  
Author(s):  
Jessica Kasza ◽  
Monica Taljaard ◽  
Andrew B. Forbes
Keyword(s):  
Information Content ◽  
Treatment Effect ◽  
Stepped Wedge ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

Inference for the treatment effect in longitudinal cluster randomized trials when treatment effect heterogeneity is ignored

2021 ◽  
pp. 096228022110417
Author(s):  
Rhys Bowden ◽  
Andrew B Forbes ◽  
Jessica Kasza
Keyword(s):  
Treatment Effect ◽  
Randomized Trials ◽  
Error Rates ◽  
Type I ◽  
Cluster Randomized Trials ◽  
Stepped Wedge ◽  
Treatment Effect Heterogeneity ◽  
Cluster Randomized ◽  
Effect Heterogeneity ◽  
The Impact

In cluster-randomized trials, sometimes the effect of the intervention being studied differs between clusters, commonly referred to as treatment effect heterogeneity. In the analysis of stepped wedge and cluster-randomized crossover trials, it is possible to include terms in outcome regression models to allow for such treatment effect heterogeneity yet this is not frequently considered. Outside of some simulation studies of specific cases where the outcome is binary, the impact of failing to include terms for treatment effect heterogeneity on the variance of the treatment effect estimator is unknown. We analytically examine the impact of failing to include terms for treatment effect heterogeneity on the variance of the treatment effect estimator, when outcomes are continuous. Using analysis of variance and feasible generalized least squares we provide expressions for this variance. For both the cluster-randomized crossover design and the stepped wedge design, our analytic derivations indicate that failing to include treatment effect heterogeneity results in the estimates for variance of the treatment effect that are too small, leading to inflation of type I error rates. We therefore recommend assessing the sensitivity of sample size calculations and conclusions drawn from the analysis of cluster randomized trials to the inclusion of treatment effect heterogeneity.

scholarly journals Modeling clustering and treatment effect heterogeneity in parallel and stepped-wedge cluster randomized trials

2018 ◽  
Vol 37 (6) ◽  
pp. 883-898 ◽  
Author(s):  
Karla Hemming ◽  
Monica Taljaard ◽  
Andrew Forbes
Keyword(s):  
Treatment Effect ◽  
Randomized Trials ◽  
Cluster Randomized Trials ◽  
Stepped Wedge ◽  
Treatment Effect Heterogeneity ◽  
Cluster Randomized ◽  
Effect Heterogeneity

scholarly journals Looking Inside the Black Box: The Importance of Causal Mechanism and Treatment Effect Heterogeneity in Experimentally Evaluated Criminal Justice Interventions

2016 ◽  
Vol 31 (1) ◽  
pp. 89-112
Author(s):  
Na Chong Min
Keyword(s):  
Treatment Effect ◽  
Empirical Studies ◽  
Evaluation Research ◽  
Black Box ◽  
Average Treatment Effect ◽  
Causal Mechanism ◽  
Population Average ◽  
Treatment Effect Heterogeneity ◽  
Policies And Programs ◽  
Effect Heterogeneity

This paper discusses limitations of the ???black-box??? experimental archetype by highlighting the narrowness of outcome-focused approaches. For a more complete understanding of the nuanced implications of policies and programs, this study calls for an investigation of causal mechanism and treatment effect heterogeneity in experimentally evaluated interventions. This study draws on two distinct but closely related empirical studies, one undertaken by Na and Paternoster (2012) and the other by Na, Loughran, and Paternoster (2015), that go beyond the estimation of a population average treatment effect by adopting more recent methodological advancements that are still underappreciated and underutilized in evaluation research.

Estimating Treatment Effects in the Presence of Noncompliance and Nonresponse: The Generalized Endogenous Treatment Model

2011 ◽  
Vol 19 (2) ◽  
pp. 205-226 ◽  
Author(s):  
Kevin M. Esterling ◽  
Michael A. Neblo ◽  
David M. J. Lazer
Keyword(s):  
Treatment Effect ◽  
Instrumental Variable ◽  
Treatment Effects ◽  
Mean Squared Error ◽  
Principal Stratification ◽  
Squared Error ◽  
Treatment Effect Heterogeneity ◽  
Causal Treatment ◽  
Effect Heterogeneity ◽  
And Control

If ignored, noncompliance with a treatment or nonresponse on outcome measures can bias estimates of treatment effects in a randomized experiment. To identify and estimate causal treatment effects in the case where compliance and response depend on unobservables, we propose the parametric generalized endogenous treatment (GET) model. GET incorporates behavioral responses within an experiment to measure each subject's latent compliance type and identifies causal effects via principal stratification. Using simulation methods and an application to field experimental data, we show GET has a dramatically lower mean squared error for treatment effect estimates than existing approaches to principal stratification that impute, rather than measure, compliance type. In addition, we show that GET allows one to relax and test the instrumental variable exclusion restriction assumption, to test for the presence of treatment effect heterogeneity across a range of compliance types, and to test for treatment ignorability when treatment and control samples are balanced on observable covariates.

Lessons Learned From Multi-regional Trials With Signals of Treatment Effect Heterogeneity

2020 ◽  
Vol 54 (1) ◽  
pp. 21-31
Author(s):  
Gang Li ◽  
Hui Quan ◽  
Gordon Lan ◽  
Soo Peter Ouyang ◽  
Fei Chen ◽  
...  
Keyword(s):  
Treatment Effect ◽  
Lessons Learned ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity

scholarly journals Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach

2018 ◽  
Vol 7 (3) ◽  
pp. 613-628 ◽  
Author(s):  
Alexander Coppock
Keyword(s):  
Treatment Effect ◽  
Experimental Treatment ◽  
Mechanical Turk ◽  
Background Characteristics ◽  
The Public ◽  
Treatment Effect Heterogeneity ◽  
Survey Experiments ◽  
Amazon's Mechanical Turk ◽  
Effect Heterogeneity

To what extent do survey experimental treatment effect estimates generalize to other populations and contexts? Survey experiments conducted on convenience samples have often been criticized on the grounds that subjects are sufficiently different from the public at large to render the results of such experiments uninformative more broadly. In the presence of moderate treatment effect heterogeneity, however, such concerns may be allayed. I provide evidence from a series of 15 replication experiments that results derived from convenience samples like Amazon’s Mechanical Turk are similar to those obtained from national samples. Either the treatments deployed in these experiments cause similar responses for many subject types or convenience and national samples do not differ much with respect to treatment effect moderators. Using evidence of limited within-experiment heterogeneity, I show that the former is likely to be the case. Despite a wide diversity of background characteristics across samples, the effects uncovered in these experiments appear to be relatively homogeneous.

On the Importance of Treatment Effect Heterogeneity in Experimentally-Evaluated Criminal Justice Interventions

2015 ◽  
Vol 31 (2) ◽  
pp. 289-310 ◽  
Author(s):  
Chongmin Na ◽  
Thomas A. Loughran ◽  
Raymond Paternoster
Keyword(s):  
Criminal Justice ◽  
Treatment Effect ◽  
Treatment Effect Heterogeneity ◽  
Effect Heterogeneity
Sign in / Sign up

Export Citation Format

Share Document