scholarly journals Randomization Tests that Condition on Non-Categorical Covariate Balance

2019 ◽  
Vol 7 (1) ◽  
Author(s):  
Zach Branson ◽  
Luke W. Miratrix
Keyword(s):  
Randomization Test ◽  
Simulation Studies ◽  
Randomized Experiments ◽  
Randomization Tests ◽  
Test Statistic ◽  
Control Groups ◽  
Unbiased Estimators ◽  
Covariate Balance ◽  
Conditional Tests ◽  
And Control

AbstractA benefit of randomized experiments is that covariate distributions of treatment and control groups are balanced on average, resulting in simple unbiased estimators for treatment effects. However, it is possible that a particular randomization yields covariate imbalances that researchers want to address in the analysis stage through adjustment or other methods. Here we present a randomization test that conditions on covariate balance by only considering treatment assignments that are similar to the observed one in terms of covariate balance. Previous conditional randomization tests have only allowed for categorical covariates, while our randomization test allows for any type of covariate. Through extensive simulation studies, we find that our conditional randomization test is more powerful than unconditional randomization tests and other conditional tests. Furthermore, we find that our conditional randomization test is valid (1) unconditionally across levels of covariate balance, and (2) conditional on particular levels of covariate balance. Meanwhile, unconditional randomization tests are valid for (1) but not (2). Finally, we find that our conditional randomization test is similar to a randomization test that uses a model-adjusted test statistic.

scholarly journals A Conditional Randomization Test to Account for Covariate Imbalance in Randomized Experiments

2016 ◽  
Vol 4 (1) ◽  
pp. 61-80 ◽  
Author(s):  
Jonathan Hennessy ◽  
Tirthankar Dasgupta ◽  
Luke Miratrix ◽  
Cassandra Pattanayak ◽  
Pradipta Sarkar
Keyword(s):  
Null Distribution ◽  
Randomization Test ◽  
Covariate Adjustment ◽  
Randomized Experiments ◽  
Randomization Tests ◽  
Test Statistic ◽  
Significance Level ◽  
Covariate Balance ◽  
Covariate Information ◽  
Added Benefit

AbstractWe consider the conditional randomization test as a way to account for covariate imbalance in randomized experiments. The test accounts for covariate imbalance by comparing the observed test statistic to the null distribution of the test statistic conditional on the observed covariate imbalance. We prove that the conditional randomization test has the correct significance level and introduce original notation to describe covariate balance more formally. Through simulation, we verify that conditional randomization tests behave like more traditional forms of covariate adjustment but have the added benefit of having the correct conditional significance level. Finally, we apply the approach to a randomized product marketing experiment where covariate information was collected after randomization.

scholarly journals Lasso adjustments of treatment effect estimates in randomized experiments

2016 ◽  
Vol 113 (27) ◽  
pp. 7383-7390 ◽  
Author(s):  
Adam Bloniarz ◽  
Hanzhong Liu ◽  
Cun-Hui Zhang ◽  
Jasjeet S. Sekhon ◽  
Bin Yu
Keyword(s):  
Treatment Effect ◽  
Multivariate Regression ◽  
Asymptotic Variance ◽  
Ordinary Least Squares ◽  
Randomized Experiments ◽  
Control Groups ◽  
The Difference ◽  
Selection Operator ◽  
Simple Difference ◽  
And Control

We provide a principled way for investigators to analyze randomized experiments when the number of covariates is large. Investigators often use linear multivariate regression to analyze randomized experiments instead of simply reporting the difference of means between treatment and control groups. Their aim is to reduce the variance of the estimated treatment effect by adjusting for covariates. If there are a large number of covariates relative to the number of observations, regression may perform poorly because of overfitting. In such cases, the least absolute shrinkage and selection operator (Lasso) may be helpful. We study the resulting Lasso-based treatment effect estimator under the Neyman–Rubin model of randomized experiments. We present theoretical conditions that guarantee that the estimator is more efficient than the simple difference-of-means estimator, and we provide a conservative estimator of the asymptotic variance, which can yield tighter confidence intervals than the difference-of-means estimator. Simulation and data examples show that Lasso-based adjustment can be advantageous even when the number of covariates is less than the number of observations. Specifically, a variant using Lasso for selection and ordinary least squares (OLS) for estimation performs particularly well, and it chooses a smoothing parameter based on combined performance of Lasso and OLS.

Design-Based Covariate Adjustments in Paired Experiments

2020 ◽  
pp. 107699862094146
Author(s):  
Edward Wu ◽  
Johann A. Gagnon-Bartsch
Keyword(s):  
Random Forests ◽  
Potential Outcomes ◽  
Prediction Algorithm ◽  
Randomized Experiments ◽  
Control Groups ◽  
Trade Off ◽  
And Control

In paired experiments, participants are grouped into pairs with similar characteristics, and one observation from each pair is randomly assigned to treatment. The resulting treatment and control groups should be well-balanced; however, there may still be small chance imbalances. Building on work for completely randomized experiments, we propose a design-based method to adjust for covariate imbalances in paired experiments. We leave out each pair and impute its potential outcomes using any prediction algorithm such as lasso or random forests. This method addresses a unique trade-off that exists for paired experiments. By addressing this trade-off, the method has the potential to improve precision over existing methods.

scholarly journals Using the half normal distribution to quantify covariate balance in cluster-randomized pragmatic trials

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Jin Huang ◽  
David L. Roth
Keyword(s):  
Simulation Studies ◽  
Pragmatic Trials ◽  
Covariate Balance ◽  
Site Characteristics ◽  
Cluster Randomized Controlled Trials ◽  
The Mean ◽  
Cluster Randomized ◽  
And Control ◽  
Final Selection ◽  
Selection Of

Abstract Background Pragmatic trials often consist of cluster-randomized controlled trials (C-RCTs), where staff of existing clinics or sites deliver interventions and randomization occurs at the site level. Covariate-constrained randomization (CCR) methods are often recommended to minimize imbalance on important site characteristics across intervention and control arms because sizable imbalances can occur by chance in simple randomizations when the number of units to be randomized is relatively small. CCR methods involve multiple random assignments initially, an assessment of balance achieved on site-level covariates from each randomization, and the final selection of an allocation that produces acceptable balance. However, no clear consensus exists on how to assess imbalance or identify allocations with sufficient balance. In this article, we describe an overall imbalance index (I) that is based on the mean of the absolute value of the standardized differences in means on the site characteristics. Methods We derive the theoretical distribution of I, then conduct simulation studies to examine its empirical properties under the varying covariate distributions and inter-correlations. Results I has an expected value of 0.798 and, assuming independent site characteristics, a variance of 0.363/k, where k is the number of site characteristics being balanced. Simulations indicated that the properties of I are robust under varying covariate circumstances as long as k is greater than 3 and the covariates are not too highly inter-correlated. Conclusions We recommend that values of I below the 10th percentile indicate sufficient overall site balance in CCRs. Definitions of acceptable randomizations might also include individual covariate criteria specified in advance, in addition to overall balance criteria.

Validity of Randomization Tests for one-Subject Experiments

1980 ◽  
Vol 5 (3) ◽  
pp. 235-251 ◽  
Author(s):  
Eugene S. Edgington
Keyword(s):  
Statistical Tests ◽  
Test Procedure ◽  
Randomization Test ◽  
Statistical Test ◽  
Random Assignment ◽  
Randomization Tests ◽  
Test Statistic ◽  
Wide Range ◽  
Α Level

Valid Statistical tests for one-subject experiments are necessary to justify Statistical inferences and to ensure the acceptability of research reports to a wide range of journals and readers. The validity of randomization tests for one-subject experiments is examined in this paper. A randomization test is a procedure for determining significance in the following manner. A test statistic (e.g., t or F) is computed for a set of research data. The value of the test statistic is called the “obtained test statistic value.” The data are then divided repeatedly, and the test statistic is computed for each data division. If the proportion of the data divisions giving a test statistic value as large as the obtained test statistic value is no greater than α, the test statistic is significant at the α level. Any Statistical test is said to be a randomization test when the significance of its test statistic is determined by the randomization test procedure. Determination of significance by the randomization test procedure permits the valid application of any Statistical test, whether it be as simple as a t test or as complex as factorial multivariate analysis of variance, for one-subject as well as multiple-subject experiments. For the randomization test procedure to be valid for a one-subject experiment, there must be random assignment of treatment times to treatments (i.e., random determination of when each treatment is to be given); specification, before observing the data, of the test statistic and the criterion to be employed for discarding data; and, in the determination of significance, division of the data in a manner consistent with the random assignment procedure.

Conjugated Linoleic Acid Causes a Marked Increase in Liver α-Tocopherol and Liver α-Tocopherol Transfer Protein in C57BL/6 J Mice

2010 ◽  
Vol 80 (1) ◽  
pp. 65-73 ◽  
Author(s):  
Pei-Min Chao ◽  
Wan-Hsuan Chen ◽  
Chun-Huei Liao ◽  
Huey-Mei Shaw
Keyword(s):  
Antioxidant Enzymes ◽  
Linoleic Acid ◽  
Conjugated Linoleic Acid ◽  
Thiobarbituric Acid ◽  
Control Group ◽  
Thiobarbituric Acid Reactive Substances ◽  
Control Groups ◽  
Protein Levels ◽  
Transfer Protein ◽  
And Control

Conjugated linoleic acid (CLA) is a collective term for the positional and geometric isomers of a conjugated diene of linoleic acid (C18:2, n-6). The aims of the present study were to evaluate whether levels of hepatic α-tocopherol, α-tocopherol transfer protein (α-TTP), and antioxidant enzymes in mice were affected by a CLA-supplemented diet. C57BL/6 J mice were divided into the CLA and control groups, which were fed, respectively, a 5 % fat diet with or without 1 g/100 g of CLA (1:1 mixture of cis-9, trans-11 and trans-10, cis-12) for four weeks. α-Tocopherol levels in plasma and liver were significantly higher in the CLA group than in the control group. Liver α-TTP levels were also significantly increased in the CLA group, the α-TTP/β-actin ratio being 2.5-fold higher than that in control mice (p<0.01). Thiobarbituric acid-reactive substances were significantly decreased in the CLA group (p<0.01). There were no significant differences between the two groups in levels of three antioxidant enzymes (superoxide dismutase, glutathione peroxidase, and catalase). The accumulation of liver α-tocopherol seen with the CLA diet can be attributed to the antioxidant potential of CLA and the ability of α-TTP induction. The lack of changes in antioxidant enzyme protein levels and the reduced lipid peroxidation in the liver of CLA mice are due to α-tocopherol accumulation.

The Prospects and Risks Connected with Increasing the Level of Self-Organisation in Students

2020 ◽  
pp. 75-81
Author(s):  
Svetlana Alexandrovna Kosareva ◽  
Keyword(s):  
Higher Education ◽  
Professional Competence ◽  
Initial Level ◽  
Education Institution ◽  
Diagnostic Methods ◽  
Study Guide ◽  
Learning Approaches ◽  
Control Groups ◽  
Self Organisation ◽  
And Control

The paper describes the method for increasing the level of self-organisation in students which has been developed by the author. It also contains the method testing results and presents the prospects and risks teachers could face while applying the method in a higher education institution. The purpose of this study is to find out the prospects and risks of applying the method for increasing the level of self-organisation in students and to determine the ways of reducing the risks. Methodology. The author points out the learning approaches which were the basis of developing the method and describes diagnostic methods for determining students’ self-organisation levels. The work focused on increasing each student’s initial level consists of a theoretical and a practical part and includes project activities on creating a study guide. The results of the study. The method developed proved to be effective. It was established by diagnosing the final level of self-organisation in students in the experimental and control groups. The paper considers the advantages of the method among which there is universal character, flexibility, improvements to teacher’s and students’ professional competence, etc. At the same time it is necessary to be aware of the risks due to the increased amount of teacher’s work and the fact that students’ work within the project tends to be monotonous. In conclusion, the prospects of the method for increasing the level of self-organisation in students are related to its advantages and the final results of the work. The risks of its use can be reduced with the help of the measures proposed in the paper.

Sri Lankan Cleft Lip and Palate Study Model Analysis: Clefts of the Secondary Palate

1993 ◽  
Vol 30 (2) ◽  
pp. 227-230 ◽  
Author(s):  
Andrew Mccance ◽  
David Roberts-Harry ◽  
Martyn Sherriff ◽  
Michael Mars ◽  
William J.B. Houston
Keyword(s):  
Cleft Lip ◽  
Cleft Lip And Palate ◽  
Model Analysis ◽  
Control Group ◽  
Tooth Size ◽  
Sri Lankan ◽  
Control Groups ◽  
Secondary Palate ◽  
Comparable Control ◽  
And Control

The study models of a group of adult Sri Lankan patients with clefts of the secondary palate were investigated. Tooth-size and arch-dimension comparisons were made with a comparable control group. Significant differences were found between the cleft and control groups in tooth sizes, chord lengths, and arch widths. The cleft group dimensions were generally smaller than those of the control group. Overjets were larger in the cleft group.

THE EFFECT OF APPLYING ANIMAL CARTOON PICTURES ON STUDENTS’ ACHIEVEMENT IN WRITING NARRATIVE TEXT

2017 ◽  
Vol 6 (1) ◽  
Author(s):  
Hasanul Arifin Zul And Masitowarni Siregar
Keyword(s):  
T Test ◽  
Narrative Text ◽  
Control Group ◽  
First Year ◽  
Writing Achievement ◽  
Control Groups ◽  
Post Test ◽  
Written Test ◽  
And Control ◽  
Experimental Group

This thesis is focused on the investigation of the effect of applying animal cartoon pictures on students’ achievement in writing narrative text. This study aims to find whether applying animal cartoon pictures significantly affect the students’ writing achievement or not. The data in this study were obtained by administering a written test. The population was the 2015/2016 first year (grade XI) of SMA Swasta Nusantara Lubuk Pakam and 66 students were taken as the sample by using random sampling. The sample was divided into two groups, experimental and control groups. The experimental group was taught by applying animal cartoon pictures while the control group without animal cartoon pictures (x = lecturing). The data were taken the scores from the pre-test and post-test to both experimental and control groups. These data were analyzed by using t-test. The result of computing the t-test obviously showed that t-observed is higher than t-table (5.21 >1,67) with the degree of freedom 64 (df =N-2) at the level significance 0,05 one tail test. It showed that the application of animal cartoon pictures significantly affected the students of SMA Swasta Nusantara Lubuk Pakam achievement in writing narrative text.

Sign in / Sign up

Export Citation Format

Share Document