scholarly journals Optimal Allocation of Interviews to Baseline and Endline Surveys in Place-Based Randomized Trials and Quasi-Experiments

2018 ◽  
Vol 42 (4) ◽  
pp. 391-422 ◽  
Author(s):  
Donald P. Green ◽  
Winston Lin ◽  
Claudia Gerber
Keyword(s):  
Random Sample ◽  
Treatment Effect ◽  
Randomized Trials ◽  
Optimal Allocation ◽  
Average Treatment Effect ◽  
Baseline Survey ◽  
Baseline Measure ◽  
Panel Surveys ◽  
Planning Stage ◽  
Average Treatment

Background: Many place-based randomized trials and quasi-experiments use a pair of cross-section surveys, rather than panel surveys, to estimate the average treatment effect of an intervention. In these studies, a random sample of individuals in each geographic cluster is selected for a baseline (preintervention) survey, and an independent random sample is selected for an endline (postintervention) survey. Objective: This design raises the question, given a fixed budget, how should a researcher allocate resources between the baseline and endline surveys to maximize the precision of the estimated average treatment effect? Results: We formalize this allocation problem and show that although the optimal share of interviews allocated to the baseline survey is always less than one-half, it is an increasing function of the total number of interviews per cluster, the cluster-level correlation between the baseline measure and the endline outcome, and the intracluster correlation coefficient. An example using multicountry survey data from Africa illustrates how the optimal allocation formulas can be combined with data to inform decisions at the planning stage. Another example uses data from a digital political advertising experiment in Texas to explore how precision would have varied with alternative allocations.

scholarly journals Generalizability of Randomized Trial Results to Target Populations

2017 ◽  
Vol 28 (5) ◽  
pp. 532-537 ◽  
Author(s):  
Elizabeth A. Stuart ◽  
Benjamin Ackerman ◽  
Daniel Westreich
Keyword(s):  
Social Work ◽  
Randomized Trial ◽  
Treatment Effect ◽  
Randomized Trials ◽  
Internal Validity ◽  
Target Population ◽  
Average Treatment Effect ◽  
Social Work Research ◽  
Average Treatment ◽  
Social Work Program

Randomized trials play an important role in estimating the effect of a policy or social work program in a given population. While most trial designs benefit from strong internal validity, they often lack external validity, or generalizability, to the target population of interest. In other words, one can obtain an unbiased estimate of the study sample average treatment effect from a randomized trial; however, this estimate may not equal the target population average treatment effect if the study sample is not fully representative of the target population. This article provides an overview of existing strategies to assess and improve upon the generalizability of randomized trials, both through statistical methods and study design, as well as recommendations on how to implement these ideas in social work research.

Clarifying selection bias in cluster randomized trials

2021 ◽  
pp. 174077452110568
Author(s):  
Fan Li ◽  
Zizhong Tian ◽  
Jennifer Bobb ◽  
Georgia Papadogeorgou ◽  
Fan Li
Keyword(s):  
Selection Bias ◽  
Treatment Effect ◽  
Treatment Effects ◽  
Randomized Trials ◽  
Additional Data ◽  
Average Treatment Effect ◽  
Causal Effects ◽  
Cluster Randomized Trials ◽  
Average Treatment ◽  
Cluster Randomized

Background In cluster randomized trials, patients are typically recruited after clusters are randomized, and the recruiters and patients may not be blinded to the assignment. This often leads to differential recruitment and consequently systematic differences in baseline characteristics of the recruited patients between intervention and control arms, inducing post-randomization selection bias. We aim to rigorously define causal estimands in the presence of selection bias. We elucidate the conditions under which standard covariate adjustment methods can validly estimate these estimands. We further discuss the additional data and assumptions necessary for estimating causal effects when such conditions are not met. Methods Adopting the principal stratification framework in causal inference, we clarify there are two average treatment effect (ATE) estimands in cluster randomized trials: one for the overall population and one for the recruited population. We derive analytical formula of the two estimands in terms of principal-stratum-specific causal effects. Furthermore, using simulation studies, we assess the empirical performance of the multivariable regression adjustment method under different data generating processes leading to selection bias. Results When treatment effects are heterogeneous across principal strata, the average treatment effect on the overall population generally differs from the average treatment effect on the recruited population. A naïve intention-to-treat analysis of the recruited sample leads to biased estimates of both average treatment effects. In the presence of post-randomization selection and without additional data on the non-recruited subjects, the average treatment effect on the recruited population is estimable only when the treatment effects are homogeneous between principal strata, and the average treatment effect on the overall population is generally not estimable. The extent to which covariate adjustment can remove selection bias depends on the degree of effect heterogeneity across principal strata. Conclusion There is a need and opportunity to improve the analysis of cluster randomized trials that are subject to post-randomization selection bias. For studies prone to selection bias, it is important to explicitly specify the target population that the causal estimands are defined on and adopt design and estimation strategies accordingly. To draw valid inferences about treatment effects, investigators should (1) assess the possibility of heterogeneous treatment effects, and (2) consider collecting data on covariates that are predictive of the recruitment process, and on the non-recruited population from external sources such as electronic health records.

scholarly journals Evaluación de impacto académico y social en egresados universitarios

2019 ◽  
Vol 30 (3) ◽  
pp. 695-712
Author(s):  
Gabriel González ◽  
Luisa Díez-Echavarría ◽  
Elkin Zapa ◽  
Danilo Eusse
Keyword(s):  
Treatment Effect ◽  
Average Treatment Effect ◽  
Educación Superior ◽  
Educacion Superior ◽  
Average Treatment

Las instituciones de educación superior deben formar a sus estudiantes según requerimientos del contexto en que se desenvuelven, ya que, sobre la base de su desempeño, es donde se medirá si las políticas de desarrollo socioeconómico son efectivas. Para lograrlo, es necesario identificar el impacto de esa educación en sus egresados, y hacer los ajustes necesarios que generen mejora continua. El objetivo de este artículo es estimar el impacto académico y social de egresados del Instituto Tecnológico Metropolitano – Medellín, a través de un análisis multivariado y la estimación del modelo Average Treatment Effect (ATE). Se encontró que la educación ofrecida a esta población ha generado un impacto académico, asociado a los estudios de actualización, y dos impactos sociales, asociados a la situación laboral y al nivel de ingresos percibidos por los egresados. Se recomienda usar esta metodología en otras instituciones, ya que suele arrojar resultados más informativos y precisos que los estudios tradicionales de caracterización, y se puede medir el efecto de cualquier estrategia.

The Effect of Extension Services and Credit on Agricultural Production in Bolivia, Peru, and Colombia

2021 ◽  
Author(s):  
Mateus C. R. Neves ◽  
Felipe De Figueiredo Silva ◽  
Carlos Otávio Freitas
Keyword(s):  
Travel Time ◽  
Treatment Effect ◽  
Agricultural Production ◽  
Treatment Effects ◽  
Geographic Information ◽  
Average Treatment Effect ◽  
Extension Services ◽  
Average Treatment ◽  
Andean Countries ◽  
Average Treatment Effects

In this paper we estimate the average treatment effect from access to extension services and credit on agricultural production in selected Andean countries (Bolivia, Peru, and Colombia). More specifically, we want to identify the effect of accessibility, here represented as travel time to the nearest area with 1,500 or more inhabitants per square kilometer or at least 50,000 inhabitants, on the likelihood of accessing extension and credit. To estimate the treatment effect and identify the effect of accessibility on these variables, we use data from the Colombian and Bolivian Agricultural Censuses of 2013 and 2014, respectively; a national agricultural survey from 2017 for Peru; and geographic information on travel time. We find that the average treatment effect for extension is higher compared to that of credit for farms in Bolivia and Peru, and lower for Colombia. The average treatment effects of extension and credit for Peruvian farms are $2,387.45 and $3,583.42 respectively. The average treatment effect for extension and credit are $941.92 and $668.69, respectively, while in Colombia are $1,365.98 and $1,192.51, respectively. We also find that accessibility and the likelihood of accessing these services are nonlinearly related. Results indicate that higher likelihood is associated with lower travel time, especially in the analysis of credit.

OLS and 2SLS in Randomized and Conditionally Randomized Experiments

2018 ◽  
Vol 238 (3-4) ◽  
pp. 243-293 ◽  
Author(s):  
Jason Ansel ◽  
Han Hong ◽  
and Jessie Li
Keyword(s):  
Treatment Effect ◽  
Average Treatment Effect ◽  
Randomized Experiments ◽  
Adaptive Randomization ◽  
Working Paper ◽  
Average Treatment ◽  
Local Average Treatment Effect ◽  
Effect Parameter ◽  
Multiple Treatments ◽  
Local Average

Abstract We investigate estimation and inference of the (local) average treatment effect parameter when a binary instrumental variable is generated by a randomized or conditionally randomized experiment. Under i.i.d. sampling, we show that adding covariates and their interactions with the instrument will weakly improve estimation precision of the (local) average treatment effect, but the robust OLS (2SLS) standard errors will no longer be valid. We provide an analytic correction that is easy to implement and demonstrate through Monte Carlo simulations and an empirical application the interacted estimator’s efficiency gains over the unadjusted estimator and the uninteracted covariate adjusted estimator. We also generalize our results to covariate adaptive randomization where the treatment assignment is not i.i.d., thus extending the recent contributions of Bugni, F., I.A. Canay, A.M. Shaikh (2017a), Inference Under Covariate-Adaptive Randomization. Working Paper and Bugni, F., I.A. Canay, A.M. Shaikh (2017b), Inference Under Covariate-Adaptive Randomization with Multiple Treatments. Working Paper to allow for the case of non-compliance.

Sign in / Sign up

Export Citation Format

Share Document