scholarly journals Impact Visualisation in Educational Interventions

2017 ◽  
Author(s):  
ZhiMin Xiao ◽  
Steve Higgins ◽  
Adetayo Kasim
Keyword(s):  
Variable Selection ◽  
Treatment Effect ◽  
Effect Size ◽  
Evidence Synthesis ◽  
Educational Interventions ◽  
Average Treatment Effect ◽  
Average Treatment ◽  
The Impact ◽  
Analytical Approaches ◽  
Size Estimates

Reporting of research data analysis often resorts to numerical summaries, such as effect size estimates in Randomised Controlled Trials (RCTs). Summary statistics are helpful and important for evidence synthesis and decision making. However, they can be unstable and inconsistent due to diversity in research designs and variability in analytical specifications. They also mask the dynamics of individual responses to a certain intervention by focusing on average treatment effect on the treated, even though the variation in impact may be crucial information for policy makers. To establish stability and consistency of impact estimates and to reveal the dynamics of individual responses in RCTs, we conduct variable selection, harness the power of noise, implement Cumulative Quantile Analysis (CQA), and devise umbrella plots of loss and gain in this study, using real datasets from over 30 educational interventions funded by the Education Endowment Foundation (EEF) in England. For the purpose of comparison, which is essential in data visualisation, all the aforementioned methods are built upon multiple analytical approaches. We show that the importance of an intervention can be ordered through variable selection, and that the power of noise or the bias induced by inappropriate variables, can be utilised to assess the stability of an impact estimate. We also demonstrate that estimates of average treatment effect cannot fully capture the impact of an intervention on sub-groups of participants with varying levels of attainment at baseline, not to mention individual responses to the intervention. Using CQA and umbrella plots, we are able to supplement what common effect size estimates lack in educational interventions. We argue that the impact of an intervention is often more complex than the average treatment effect suggests, and that until a summary is more informative and able to speak directly to the eye, evidence-based policy and practice cannot be fully achieved.

scholarly journals Adoption of Conservation Practices: Its Impact on Input Use and Performance in the Northern Region of Ghana

2018 ◽  
Vol 11 (5) ◽  
pp. 149
Author(s):  
Mavis Boimah ◽  
Akwasi Mensah-Bonsu ◽  
Yaw Osei-Asare ◽  
Daniel B. Sarpong
Keyword(s):  
Treatment Effect ◽  
Environmental Sustainability ◽  
Technical Assistance ◽  
Negative Impact ◽  
Maize Yield ◽  
Average Treatment Effect ◽  
Inorganic Fertilizer ◽  
Switching Regression ◽  
Average Treatment ◽  
The Impact

Conservation Agriculture (CA) is promoted worldwide on the basis of its contribution to economic, social, and environmental sustainability of agricultural production. In Ghana, despite the increasing interest in the promotion of CA and its practices, its rate of adoption is still low, mainly due to the conflicting evidences regarding its effectiveness. This paper contributes to the numerous debates by examining the impact of CA practices on hired labour, rates of inorganic fertilizers applied by adopters, maize yield, and profit of adopters. Using a cross-sectional data, a multinomial endogenous switching regression (MESR) model was employed to compute the Average Treatment Effect (ATE) and Average Treatment Effect on Treated (ATET) for yield, hired labour, inorganic fertilizer rate, and profit of adopters of CA practices. The study reveals that CA practices impact positively on hired labour employed on the farm, but have a negative impact on profits of adopters. No impact whatsoever of adoption of CA practices is observed on maize yield and also inorganic fertilizer application rates. Technical assistance, and training of farmers on strategies that minimize costs of production must be intensified to raise profits of adopters.

Propensity score specification for optimal estimation of average treatment effect with binary response

2020 ◽  
Vol 29 (12) ◽  
pp. 3623-3640
Author(s):  
John A Craycroft ◽  
Jiapeng Huang ◽  
Maiying Kong
Keyword(s):  
Propensity Score ◽  
Treatment Effect ◽  
Propensity Scores ◽  
Optimal Estimation ◽  
Theoretical Explanation ◽  
Average Treatment Effect ◽  
Adaptive Lasso ◽  
Confounding Bias ◽  
Average Treatment ◽  
The Impact

Propensity score methods are commonly used in statistical analyses of observational data to reduce the impact of confounding bias in estimations of average treatment effect. While the propensity score is defined as the conditional probability of a subject being in the treatment group given that subject’s covariates, the most precise estimation of average treatment effect results from specifying the propensity score as a function of true confounders and predictors only. This property has been demonstrated via simulation in multiple prior research articles. However, we have seen no theoretical explanation as to why this should be so. This paper provides that theoretical proof. Furthermore, this paper presents a method for performing the necessary variable selection by means of elastic net regression, and then estimating the propensity scores so as to obtain optimal estimates of average treatment effect. The proposed method is compared against two other recently introduced methods, outcome-adaptive lasso and covariate balancing propensity score. Extensive simulation analyses are employed to determine the circumstances under which each method appears most effective. We applied the proposed methods to examine the effect of pre-cardiac surgery coagulation indicator on mortality based on a linked dataset from a retrospective review of 1390 patient medical records at Jewish Hospital (Louisville, KY) with the Society of Thoracic Surgeons database.

Impact of the production of underutilized vegetables on the livelihood of farmers in Southwestern Nigeria

2017 ◽  
Vol 44 (12) ◽  
pp. 1669-1682
Author(s):  
Oluwatosin Adejoke Oyedele ◽  
Kemisola O. Adenegan
Keyword(s):  
Treatment Effect ◽  
Nutritive Value ◽  
Average Treatment Effect ◽  
Current Status ◽  
Special Focus ◽  
Vegetable Production ◽  
Southwestern Nigeria ◽  
Content Type ◽  
Average Treatment ◽  
The Impact

Purpose African indigenous vegetables have high nutritive value which contains high levels of minerals. The current status of indigenous vegetable production in developing countries shows that these crops are “under-recognized” and “underutilized” with respect to nutritional value and opportunities for food security. The purpose of this paper is to examine the impact of the production of underutilized vegetables on the livelihood of farmers in South Western Nigeria. Design/methodology/approach The population for the study includes all the vegetable farmers in South Western Nigeria with a special focus on farmers’ groups formed by the NICANVEG project in Osun, Oyo, Ondo and Ekiti states. Descriptive statistics and propensity score matching (PSM) was used to analyze the objective. Findings Perceptions on individual household income reveal that the majority of participating respondents perceived higher production and harvesting density. This is due to the fact that harvesting is done by cutting the stems of the vegetables. The probability score shows that the dependent variables have an average effect of 44.6 percent on the probability of farmers participating in NICANVEG project. The PSM results reveal that average treatment effect on the treated is ₦269,254.87. Average treatment on the untreated is ₦11,990.63 while average treatment effect is ₦139,336.43. The total income of the participants from all the various livelihood strategies is increased by 29.73 percent because of their participation in the NICANVEG project. Originality/value This work has not been carried out by any other person before. This work will add to the existing knowledge on the impact of evaluation in agricultural economics.

scholarly journals A Text-As-Data Approach for Using Open-Ended Responses as Manipulation Checks

2020 ◽  
Author(s):  
Jeffrey Ziegler
Keyword(s):  
Treatment Effect ◽  
Average Treatment Effect ◽  
Continuous Measure ◽  
Causal Inferences ◽  
Online Surveys ◽  
Manipulation Check ◽  
Potential Variation ◽  
Average Treatment ◽  
The Impact

Participants that complete online surveys and experiments may be inattentive, which can hinder researchers’ ability to draw substantive or causal inferences. As such, many practitioners include multiple factual or informational closed-ended manipulation checks to identify low-attention respondents. However, closed-ended manipulation checks are either correct or incorrect, which allows participants to more easily guess and it reduces the potential variation in attention between respondents. In response to these shortcomings, I develop an automatic and standardized methodology to measure attention that relies on the text that respondents provide in an open-ended manipulation check. There are multiple benefits to this approach. First, it provides a continuous measure of attention, which allows for greater variation between respondents. Second, it reduces the reliance on subjective, paid humans to analyze open-ended responses. Last, I outline how to diagnose the impact of inattentive workers on the overall results, including how to assess the average treatment effect of those respondents that likely received the treatment. I provide easy-to-use software in R to implement these suggestions for open-ended manipulation checks.

scholarly journals Instruments with Heterogeneous Effects: Bias, Monotonicity, and Localness

2020 ◽  
Vol 8 (1) ◽  
pp. 182-208
Author(s):  
Nick Huntington-Klein
Keyword(s):  
Treatment Effect ◽  
Treatment Effects ◽  
Weighted Average ◽  
Small Sample ◽  
Average Treatment Effect ◽  
Average Treatment ◽  
Local Average Treatment Effect ◽  
Heterogeneous Effects ◽  
The Impact ◽  
Local Average

AbstractIn Instrumental Variables (IV) estimation, the effect of an instrument on an endogenous variable may vary across the sample. In this case, IV produces a local average treatment effect (LATE), and if monotonicity does not hold, then no effect of interest is identified. In this paper, I calculate the weighted average of treatment effects that is identified under general first-stage effect heterogeneity, which is generally not the average treatment effect among those affected by the instrument. I then describe a simple set of data-driven approaches to modeling variation in the effect of the instrument. These approaches identify a Super-Local Average Treatment Effect (SLATE) that weights treatment effects by the corresponding instrument effect more heavily than LATE. Even when first-stage heterogeneity is poorly modeled, these approaches considerably reduce the impact of small-sample bias compared to standard IV and unbiased weak-instrument IV methods, and can also make results more robust to violations of monotonicity. In application to a published study with a strong instrument, the preferred approach reduces error by about 19% in small (N ≈ 1, 000) subsamples, and by about 13% in larger (N ≈ 33, 000) subsamples.

scholarly journals The Effect of IL-6 Inhibitors on Mortality Among Hospitalized COVID-19 Patients: A Multicenter Study

2020 ◽  
Author(s):  
Pranay Sinha ◽  
S Reza Jafarzadeh ◽  
Sabrina A Assoumou ◽  
Catherine G Bielick ◽  
Bethanne Carpenter ◽  
...  
Keyword(s):  
Confidence Interval ◽  
Hospital Mortality ◽  
Treatment Effect ◽  
Effect Size ◽  
Odds Ratio ◽  
Safety Net ◽  
Average Treatment Effect ◽  
Polymerase Chain Reaction Test ◽  
Average Treatment ◽  
Safety Net Hospital

Abstract Background The effectiveness of interleukin-6 inhibitors (IL-6i) in ameliorating coronavirus disease 2019 (COVID-19) remains uncertain. Methods We analyzed data for patients aged ≥18 years admitted with a positive severe acute respiratory syndrome coronavirus 2 polymerase chain reaction test at 4 safety-net hospital systems with diverse populations and high rates of medical comorbidities in 3 US regions. We used inverse probability of treatment weighting via machine learning for confounding adjustment by demographics, comorbidities, and disease severity markers. We estimated the average treatment effect, the odds of IL-6i effect on in-hospital mortality from COVID-19, using a logistic marginal structural model. Results Of 516 patients, 104 (20.1%) received IL-6i. Estimate of the average treatment effect adjusted for confounders suggested a 37% reduction in odds of in-hospital mortality in those who received IL-6i compared with those who did not, although the confidence interval included the null value of 1 (odds ratio = 0.63; 95% confidence interval, .29–1.38). A sensitivity analysis suggested that potential unmeasured confounding would require a minimum odds ratio of 2.55 to nullify our estimated IL-6i effect size. Conclusions Despite low precision, our findings suggested a relatively large effect size of IL-6i in reducing the odds of COVID-19–related in-hospital mortality.

scholarly journals A Text-As-Data Approach for Using Open-Ended Responses as Manipulation Checks

2021 ◽  
pp. 1-9
Author(s):  
Jeffrey Ziegler
Keyword(s):  
Treatment Effect ◽  
Average Treatment Effect ◽  
Continuous Measure ◽  
Causal Inferences ◽  
Online Surveys ◽  
Manipulation Check ◽  
Potential Variation ◽  
Average Treatment ◽  
The Impact

Abstract Participants that complete online surveys and experiments may be inattentive, which can hinder researchers’ ability to draw substantive or causal inferences. As such, many practitioners include multiple factual or instructional closed-ended manipulation checks to identify low-attention respondents. However, closed-ended manipulation checks are either correct or incorrect, which allows participants to more easily guess and it reduces the potential variation in attention between respondents. In response to these shortcomings, I develop an automatic and standardized methodology to measure attention that relies on the text that respondents provide in an open-ended manipulation check. There are multiple benefits to this approach. First, it provides a continuous measure of attention, which allows for greater variation between respondents. Second, it reduces the reliance on subjective, paid humans to analyze open-ended responses. Last, I outline how to diagnose the impact of inattentive workers on the overall results, including how to assess the average treatment effect of those respondents that likely received the treatment. I provide easy-to-use software in R to implement these suggestions for open-ended manipulation checks.

Sign in / Sign up

Export Citation Format

Share Document