Design-Based Covariate Adjustments in Paired Experiments

2020 ◽  
pp. 107699862094146
Author(s):  
Edward Wu ◽  
Johann A. Gagnon-Bartsch
Keyword(s):  
Random Forests ◽  
Potential Outcomes ◽  
Prediction Algorithm ◽  
Randomized Experiments ◽  
Control Groups ◽  
Trade Off ◽  
And Control

In paired experiments, participants are grouped into pairs with similar characteristics, and one observation from each pair is randomly assigned to treatment. The resulting treatment and control groups should be well-balanced; however, there may still be small chance imbalances. Building on work for completely randomized experiments, we propose a design-based method to adjust for covariate imbalances in paired experiments. We leave out each pair and impute its potential outcomes using any prediction algorithm such as lasso or random forests. This method addresses a unique trade-off that exists for paired experiments. By addressing this trade-off, the method has the potential to improve precision over existing methods.

scholarly journals The LOOP Estimator: Adjusting for Covariates in Randomized Experiments

2018 ◽  
Vol 42 (4) ◽  
pp. 458-488 ◽  
Author(s):  
Edward Wu ◽  
Johann A. Gagnon-Bartsch
Keyword(s):  
Variable Selection ◽  
Controlled Trial ◽  
Potential Outcomes ◽  
Prediction Algorithm ◽  
Randomized Experiments ◽  
Randomized Controlled ◽  
Statistical Assumptions ◽  
Leave One Out ◽  
And Control ◽  
Baseline Covariates

Background: When conducting a randomized controlled trial, it is common to specify in advance the statistical analyses that will be used to analyze the data. Typically, these analyses will involve adjusting for small imbalances in baseline covariates. However, this poses a dilemma, as adjusting for too many covariates can hurt precision more than it helps, and it is often unclear which covariates are predictive of outcome prior to conducting the experiment. Objectives: This article aims to produce a covariate adjustment method that allows for automatic variable selection, so that practitioners need not commit to any specific set of covariates prior to seeing the data. Results: In this article, we propose the “leave-one-out potential outcomes” estimator. We leave out each observation and then impute that observation’s treatment and control potential outcomes using a prediction algorithm such as a random forest. In addition to allowing for automatic variable selection, this estimator is unbiased under the Neyman–Rubin model, generally performs at least as well as the unadjusted estimator, and the experimental randomization largely justifies the statistical assumptions made.

scholarly journals A Theory of Experimenters: Robustness, Randomization, and Balance

2020 ◽  
Vol 110 (4) ◽  
pp. 1206-1230 ◽  
Author(s):  
Abhijit V. Banerjee ◽  
Sylvain Chassang ◽  
Sergio Montero ◽  
Erik Snowberg
Keyword(s):  
Decision Maker ◽  
Real World ◽  
Robust Performance ◽  
Control Groups ◽  
Trade Off ◽  
Performance Guarantees ◽  
Important Concern ◽  
Expected Performance ◽  
And Control ◽  
Shed Light

This paper studies the problem of experiment design by an ambiguity-averse decision-maker who trades off subjective expected performance against robust performance guarantees. This framework accounts for real-world experimenters’ preference for randomization. It also clarifies the circumstances in which randomization is optimal: when the available sample size is large and robustness is an important concern. We apply our model to shed light on the practice of rerandomization, used to improve balance across treatment and control groups. We show that rerandomization creates a trade-off between subjective performance and robust performance guarantees. However, robust performance guarantees diminish very slowly with the number of rerandomizations. This suggests that moderate levels of rerandomization usefully expand the set of acceptable compromises between subjective performance and robustness. Targeting a fixed quantile of balance is safer than targeting an absolute balance objective. (JEL C90, D81)

scholarly journals Lasso adjustments of treatment effect estimates in randomized experiments

2016 ◽  
Vol 113 (27) ◽  
pp. 7383-7390 ◽  
Author(s):  
Adam Bloniarz ◽  
Hanzhong Liu ◽  
Cun-Hui Zhang ◽  
Jasjeet S. Sekhon ◽  
Bin Yu
Keyword(s):  
Treatment Effect ◽  
Multivariate Regression ◽  
Asymptotic Variance ◽  
Ordinary Least Squares ◽  
Randomized Experiments ◽  
Control Groups ◽  
The Difference ◽  
Selection Operator ◽  
Simple Difference ◽  
And Control

We provide a principled way for investigators to analyze randomized experiments when the number of covariates is large. Investigators often use linear multivariate regression to analyze randomized experiments instead of simply reporting the difference of means between treatment and control groups. Their aim is to reduce the variance of the estimated treatment effect by adjusting for covariates. If there are a large number of covariates relative to the number of observations, regression may perform poorly because of overfitting. In such cases, the least absolute shrinkage and selection operator (Lasso) may be helpful. We study the resulting Lasso-based treatment effect estimator under the Neyman–Rubin model of randomized experiments. We present theoretical conditions that guarantee that the estimator is more efficient than the simple difference-of-means estimator, and we provide a conservative estimator of the asymptotic variance, which can yield tighter confidence intervals than the difference-of-means estimator. Simulation and data examples show that Lasso-based adjustment can be advantageous even when the number of covariates is less than the number of observations. Specifically, a variant using Lasso for selection and ordinary least squares (OLS) for estimation performs particularly well, and it chooses a smoothing parameter based on combined performance of Lasso and OLS.

scholarly journals Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds

2018 ◽  
Vol 43 (5) ◽  
pp. 540-567 ◽  
Author(s):  
Jiannan Lu ◽  
Peng Ding ◽  
Tirthankar Dasgupta
Keyword(s):  
Causal Effect ◽  
Real Life ◽  
Causal Effects ◽  
Potential Outcomes ◽  
Randomized Experiments ◽  
Sharp Bounds ◽  
Average Causal Effect ◽  
Behavioral Studies ◽  
And Control ◽  
Potential Outcomes Framework

Assessing the causal effects of interventions on ordinal outcomes is an important objective of many educational and behavioral studies. Under the potential outcomes framework, we can define causal effects as comparisons between the potential outcomes under treatment and control. However, unfortunately, the average causal effect, often the parameter of interest, is difficult to interpret for ordinal outcomes. To address this challenge, we propose to use two causal parameters, which are defined as the probabilities that the treatment is beneficial and strictly beneficial for the experimental units. However, although well-defined for any outcomes and of particular interest for ordinal outcomes, the two aforementioned parameters depend on the association between the potential outcomes and are therefore not identifiable from the observed data without additional assumptions. Echoing recent advances in the econometrics and biostatistics literature, we present the sharp bounds of the aforementioned causal parameters for ordinal outcomes, under fixed marginal distributions of the potential outcomes. Because the causal estimands and their corresponding sharp bounds are based on the potential outcomes themselves, the proposed framework can be flexibly incorporated into any chosen models of the potential outcomes and is directly applicable to randomized experiments, unconfounded observational studies, and randomized experiments with noncompliance. We illustrate our methodology via numerical examples and three real-life applications related to educational and behavioral research.

scholarly journals Insights on Variance Estimation for Blocked and Matched Pairs Designs

2020 ◽  
pp. 107699862094627
Author(s):  
Nicole E. Pashley ◽  
Luke W. Miratrix
Keyword(s):  
Variance Estimation ◽  
Control Unit ◽  
Potential Outcomes ◽  
Matched Pairs ◽  
Randomized Experiments ◽  
Multiple Treatment ◽  
Variance Estimators ◽  
Control Units ◽  
Average Impact ◽  
And Control

Evaluating blocked randomized experiments from a potential outcomes perspective has two primary branches of work. The first focuses on larger blocks, with multiple treatment and control units in each block. The second focuses on matched pairs, with a single treatment and control unit in each block. These literatures not only provide different estimators for the standard errors of the estimated average impact, but they are also built on different sets of assumptions. Neither literature handles cases with blocks of varying size that contain singleton treatment or control units, a case which can occur in a variety of contexts, such as with different forms of matching or poststratification. In this article, we reconcile the literatures by carefully examining the performance of variance estimators under several different frameworks. We then use these insights to derive novel variance estimators for experiments containing blocks of different sizes.

scholarly journals Randomization Tests that Condition on Non-Categorical Covariate Balance

2019 ◽  
Vol 7 (1) ◽  
Author(s):  
Zach Branson ◽  
Luke W. Miratrix
Keyword(s):  
Randomization Test ◽  
Simulation Studies ◽  
Randomized Experiments ◽  
Randomization Tests ◽  
Test Statistic ◽  
Control Groups ◽  
Unbiased Estimators ◽  
Covariate Balance ◽  
Conditional Tests ◽  
And Control

AbstractA benefit of randomized experiments is that covariate distributions of treatment and control groups are balanced on average, resulting in simple unbiased estimators for treatment effects. However, it is possible that a particular randomization yields covariate imbalances that researchers want to address in the analysis stage through adjustment or other methods. Here we present a randomization test that conditions on covariate balance by only considering treatment assignments that are similar to the observed one in terms of covariate balance. Previous conditional randomization tests have only allowed for categorical covariates, while our randomization test allows for any type of covariate. Through extensive simulation studies, we find that our conditional randomization test is more powerful than unconditional randomization tests and other conditional tests. Furthermore, we find that our conditional randomization test is valid (1) unconditionally across levels of covariate balance, and (2) conditional on particular levels of covariate balance. Meanwhile, unconditional randomization tests are valid for (1) but not (2). Finally, we find that our conditional randomization test is similar to a randomization test that uses a model-adjusted test statistic.

Conjugated Linoleic Acid Causes a Marked Increase in Liver α-Tocopherol and Liver α-Tocopherol Transfer Protein in C57BL/6 J Mice

2010 ◽  
Vol 80 (1) ◽  
pp. 65-73 ◽  
Author(s):  
Pei-Min Chao ◽  
Wan-Hsuan Chen ◽  
Chun-Huei Liao ◽  
Huey-Mei Shaw
Keyword(s):  
Antioxidant Enzymes ◽  
Linoleic Acid ◽  
Conjugated Linoleic Acid ◽  
Thiobarbituric Acid ◽  
Control Group ◽  
Thiobarbituric Acid Reactive Substances ◽  
Control Groups ◽  
Protein Levels ◽  
Transfer Protein ◽  
And Control

Conjugated linoleic acid (CLA) is a collective term for the positional and geometric isomers of a conjugated diene of linoleic acid (C18:2, n-6). The aims of the present study were to evaluate whether levels of hepatic α-tocopherol, α-tocopherol transfer protein (α-TTP), and antioxidant enzymes in mice were affected by a CLA-supplemented diet. C57BL/6 J mice were divided into the CLA and control groups, which were fed, respectively, a 5 % fat diet with or without 1 g/100 g of CLA (1:1 mixture of cis-9, trans-11 and trans-10, cis-12) for four weeks. α-Tocopherol levels in plasma and liver were significantly higher in the CLA group than in the control group. Liver α-TTP levels were also significantly increased in the CLA group, the α-TTP/β-actin ratio being 2.5-fold higher than that in control mice (p<0.01). Thiobarbituric acid-reactive substances were significantly decreased in the CLA group (p<0.01). There were no significant differences between the two groups in levels of three antioxidant enzymes (superoxide dismutase, glutathione peroxidase, and catalase). The accumulation of liver α-tocopherol seen with the CLA diet can be attributed to the antioxidant potential of CLA and the ability of α-TTP induction. The lack of changes in antioxidant enzyme protein levels and the reduced lipid peroxidation in the liver of CLA mice are due to α-tocopherol accumulation.

The Prospects and Risks Connected with Increasing the Level of Self-Organisation in Students

2020 ◽  
pp. 75-81
Author(s):  
Svetlana Alexandrovna Kosareva ◽  
Keyword(s):  
Higher Education ◽  
Professional Competence ◽  
Initial Level ◽  
Education Institution ◽  
Diagnostic Methods ◽  
Study Guide ◽  
Learning Approaches ◽  
Control Groups ◽  
Self Organisation ◽  
And Control

The paper describes the method for increasing the level of self-organisation in students which has been developed by the author. It also contains the method testing results and presents the prospects and risks teachers could face while applying the method in a higher education institution. The purpose of this study is to find out the prospects and risks of applying the method for increasing the level of self-organisation in students and to determine the ways of reducing the risks. Methodology. The author points out the learning approaches which were the basis of developing the method and describes diagnostic methods for determining students’ self-organisation levels. The work focused on increasing each student’s initial level consists of a theoretical and a practical part and includes project activities on creating a study guide. The results of the study. The method developed proved to be effective. It was established by diagnosing the final level of self-organisation in students in the experimental and control groups. The paper considers the advantages of the method among which there is universal character, flexibility, improvements to teacher’s and students’ professional competence, etc. At the same time it is necessary to be aware of the risks due to the increased amount of teacher’s work and the fact that students’ work within the project tends to be monotonous. In conclusion, the prospects of the method for increasing the level of self-organisation in students are related to its advantages and the final results of the work. The risks of its use can be reduced with the help of the measures proposed in the paper.

Sri Lankan Cleft Lip and Palate Study Model Analysis: Clefts of the Secondary Palate

1993 ◽  
Vol 30 (2) ◽  
pp. 227-230 ◽  
Author(s):  
Andrew Mccance ◽  
David Roberts-Harry ◽  
Martyn Sherriff ◽  
Michael Mars ◽  
William J.B. Houston
Keyword(s):  
Cleft Lip ◽  
Cleft Lip And Palate ◽  
Model Analysis ◽  
Control Group ◽  
Tooth Size ◽  
Sri Lankan ◽  
Control Groups ◽  
Secondary Palate ◽  
Comparable Control ◽  
And Control

The study models of a group of adult Sri Lankan patients with clefts of the secondary palate were investigated. Tooth-size and arch-dimension comparisons were made with a comparable control group. Significant differences were found between the cleft and control groups in tooth sizes, chord lengths, and arch widths. The cleft group dimensions were generally smaller than those of the control group. Overjets were larger in the cleft group.

Sign in / Sign up

Export Citation Format

Share Document