Anti-Corruption Campaign and Firm Financial Performance: Evidence From Vietnam Firms

Background Corruption affects businesses in various ways. Anti-corruption, on the other hand, can improve the institutions of the country as well as business operations. Vietnam, as a socialist-oriented country with an ongoing high-profile anti-corruption campaign, provides us a unique setting to evaluate the impacts of anti-corruption on corporate performance. Objectives We address two questions: (1) what is the effect of anti-corruption on the performance of private-owned firms in Vietnam? and (2) how does anti-corruption influence the performance of firms with state ownership (FSOs) in Vietnam? Research design To investigate the impact of anti-corruption on performance of firms with different ownership settings, we use the establishment of the Central Anti-Corruption Steering Committee of Vietnam as a quasi-natural experiment for difference-in-differences analysis. We generate treatment effects of private holding and the state block ownership. To validate the findings, we construct a novel news-based anti-corruption index from Vietnamese online newspapers and use it in a robustness test to evaluate anti-corruption’s impacts on firm performance. Results and Conclusions We find a positive impact of the anti-corruption campaign on private firms’ performance, supporting the social norm perspective of how corruption affects businesses. The empirical results indicate a negative impact of the campaign on FSOs’ performance. The findings suggest that anti-corruption benefits private firms via improving the institutional quality of the country while improving the financial transparency of FSOs. Our study provides a method for measuring anti-corruption which is virtually unobservable and absent in the literature. The findings have implications for policymaking in contemporary Vietnam.

An Empirical Validation of the Regression Point Displacement Design Using Within-Study Comparison Logic: Emerging Possibilities and Cautions

Background The regression point displacement (RPD) design is a quasi-experiment (QE) that aims to control many threats to internal validity. Though it has existed for several decades, RPD has only recently begun to answer applied research questions in lieu of stronger QEs. Objectives Our primary objective was to implement within-study comparison (WSC) logic to create RPD replicates and to determine conditions under which RPD might provide estimates comparable to those found in validating experiments. Research Design We utilize three randomized controlled trials (two cluster-level, one individual-level), artificially decomposing or creating cluster structures, to create multiple RPDs. We compare results in each RPD treatment group to a fixed set of control groups to gauge the congruence of these repeated RPD realizations with results found in these three RCTs. Results RPD’s performance was uneven. Using multiple criteria, we found that RPDs successfully predicted the direction of the RCT’s intervention effect but inconsistently fell within the .10 SD threshold. A scant 13% of RPD results were statistically significant at either the .05 or .01 alpha-level. RPD results were within the 95% confidence interval of RCTs around half the time, and false negative rates were substantially higher than false positive rates. Conclusions RPD consistently underestimates treatment effects in validating RCTs. We analyze reasons for this insensitivity and offer practical suggestions to improve the chances RPD will correctly identify favorable results. We note that the synthetic, “decomposition of cluster RCTs,” WSC design represents a prototype for evaluating other QEs.

Did the Hospital Readmissions Reduction Program Reduce Readmissions? An Assessment of Prior Evidence and New Estimates

In this article, we provide a comprehensive, empirical assessment of the hypothesis that the Hospital Readmissions Reduction Program (HRRP) affected hospital readmissions. In doing so, we provide evidence as to the validity of prior empirical approaches used to evaluate the HRRP and we present results from a previously unused approach to study this research question—a regression-kink design. Results of our analysis document that the empirical approaches used in most prior research assessing the efficacy of the HRRP often lack internal validity. Therefore, results from these studies may not be informative about the causal consequences of the HRRP. Results from our regression-kink analysis, which we validate, suggest that the HRRP had little effect on hospital readmissions. This finding contrasts with the results of most prior studies, which report that the HRRP significantly reduced readmissions. Our finding is consistent with conceptual considerations related to the assumptions underlying HRRP penalty: in particular, the difficulty of identifying preventable readmissions, the highly imperfect risk adjustment that affects the penalty determination, and the absence of proven tools to reduce readmissions.

Finite Mixture Modeling for Program Evaluation: Resampling and Pre-processing Approaches

Background Finite mixture models cluster individuals into latent subgroups based on observed traits. However, inaccurate enumeration of clusters can have lasting implications on policy decisions and allocations of resources. Applied and methodological researchers accept no obvious best model fit statistic, and different measures could suggest different numbers of latent clusters. Objectives The purpose of this article is to evaluate and compare different cluster enumeration techniques. Research Design Study I demonstrates how recently proposed resampling methods result in no precise number of clusters on which all fit statistics agree. We recommend the pre-processing method in Study II as an alternative. Both studies used nationally representative data on working memory, cognitive flexibility, and inhibitory control. Conclusions The data plus priors method shows promise to address inconsistencies among fit measures and help applied researchers using finite mixture models in the future.

Assessing the Impact of Holocaust Education on Adolescents’ Civic Values: Experimental Evidence from Arkansas

Background American adults overwhelmingly agree that the Holocaust should be taught in schools, yet few studies investigate the potential benefits of Holocaust education. Objectives We evaluate the impact of a Holocaust education conference on knowledge of the Holocaust and several civic outcomes, including “upstander” efficacy (willingness to intervene on behalf of others), likelihood of exercising civil disobedience, empathy for the suffering of others, and tolerance of others with different values and lifestyles. Research Design We recruit two cohorts of students from three local high schools and randomize access to the Arkansas Holocaust Education Conference, where students have the chance to hear from a Holocaust survivor and to participate in breakout sessions led by Holocaust experts. Results We find evidence that the conference increased participants’ upstander efficacy, but fail to reject the null hypothesis that the conference would increase participants’ knowledge or other civic attitudes.

Repetita Iuvant? A Repeated Randomized Controlled Trial on the Effectiveness of an At-Scale Teacher Professional Development Program

Background A recent stream of literature recognizes the impact of good/poor implementation on the effectiveness of programs. However, implementation is often disregarded in randomized controlled trials (RCTs) because they are run on a small scale. Replicated RCTs, although rare, provide a unique opportunity to study the relevance of implementation for program effectiveness. Objectives Evaluating the effectiveness of an at-scale professional development program for lower secondary school math teachers through two repeated RCTs. Research Design The program lasts a full school year and provides innovative methods for teaching math. The evaluation was conducted on two cohorts of teachers in the 2009/10 and 2010/11 school years. The program and RCTs were held at scale. Participating teachers and their classes were followed for 3 years. Impact is estimated by comparing the math scores of treatment and control students. Subjects The evaluation involved 195 teachers and their 3940 students (first cohort) and 146 teachers and their 2858 students (second cohort). Measures The key outcome is students’ math achievement, measured through standardized assessment. Results In the first wave, the program did not impact on students’ achievement, while in the second wave, a positive, persistent, and not insignificant effect was found. After excluding other sources of change, different findings across waves are interpreted in the light of improvements in the program implementation, such as enrollment procedure, teacher collaboration, and integration of innovation in daily teaching. Conclusions Repeated assessment of interventions already at-scale provides the opportunity to better identify and correct sources of weak implementation, potentially improving effectiveness.

Promoting Readiness of Minors in Supplemental Security Income (PROMISE): Early Impacts from a Multi-Site Random Assignment Evaluation

Background PROMISE was a federal initiative to support youth receiving Supplemental Security Income (SSI) during the transition to adulthood. Objectives This article presents estimates of the impacts of the six PROMISE projects on youth and family outcomes as of 18 months after enrolling in PROMISE. Research Design The study uses a randomized controlled trial design. Subjects The six PROMISE projects each enrolled a minimum of 2000 treatment and control youth (and their parents) residing in their service areas who were aged 14 to 16 and receiving SSI. Measures We estimated impacts on outcomes related to youth and family service use, school enrollment, training, employment, earnings, and federal disability program participation using survey and administrative data. Results The projects succeeded in connecting more youth to transition services and more families to support services during the 18 months after enrollment, and most increased the likelihood that youth applied for state vocational rehabilitation services. On average, there was no impact on youth’s school enrollment, but there were favorable impacts on youth’s receipt of job-related training, employment, earnings, and total income. The projects did not affect parents’ employment, earnings, or income, on average. For most outcomes PROMISE affected, the impacts varied substantially across the projects. Conclusions The positive short-term impacts of PROMISE on youth’s use of transition services, youth employment, and families’ use of services are consistent with the program logic model and suggest there might be potential for longer-term favorable impacts on youth and family outcomes.

The Role of Sample Size to Attain Statistically Comparable Groups – A Required Data Preprocessing Step to Estimate Causal Effects With Observational Data

Propensity score methods provide data preprocessing tools to remove selection bias and attain statistically comparable groups – the first requirement when attempting to estimate causal effects with observational data. Although guidelines exist on how to remove selection bias when groups in comparison are large, not much is known on how to proceed when one of the groups in comparison, for example, a treated group, is particularly small, or when the study also includes lots of observed covariates (relative to the treated group’s sample size). This article investigates whether propensity score methods can help us to remove selection bias in studies with small treated groups and large amount of observed covariates. We perform a series of simulation studies to study factors such as sample size ratio of control to treated units, number of observed covariates and initial imbalances in observed covariates between the groups of units in comparison, that is, selection bias. The results demonstrate that selection bias can be removed with small treated samples, but under different conditions than in studies with large treated samples. For example, a study design with 10 observed covariates and eight treated units will require the control group to be at least 10 times larger than the treated group, whereas a study with 500 treated units will require at least, only, two times bigger control group. To confirm the usefulness of simulation study results for practice, we carry out an empirical evaluation with real data. The study provides insights for practice and directions for future research.

Stakeholder Involvement in Evaluation: Does it Affect Observers’ Perceptions of an Evaluation? And Which Stakeholder Group(s) Do People Think Should to Participate?

Background Stakeholders are often involved in evaluation, such as in the selection of specific research questions and the interpretation of results. Except for the topic of whether stakeholder involvement increases use, a paucity of research exists to guide practice regarding stakeholders. Objectives We address two questions: (1) If a third-party observer knows stakeholders were involved in an evaluation, does that affect the perceived credibility, fairness, and relevance of the evaluation? (2) Among individuals with a possible stake in an evaluation, which stakeholder group(s) do they want to see participate; in particular, do they prefer that multiple stakeholder groups, rather than a single group, participate? Research Design Six studies are reported. All studies address the former question, while Studies 3 to 5 also focus on the latter question. To study effects of stakeholder involvement on third-party views, participants read summaries of ostensible evaluations, with stakeholder involvement noted or not. To examine a priori preferences among potential stakeholders, participants completed a survey about alternative stakeholder group involvement in an evaluation in which they would likely have an interest. Results and Conclusions Across studies, effects of reported stakeholder participation on third-parties’ views were not robust; however, small effects on perceived fairness sometimes, but not always, occurred after stakeholder involvement and its rationales had been made salient. All surveys showed a large preference for the involvement of multiple, rather than single stakeholder groups. We discuss implications for research and practice regarding stakeholder involvement, and for research on evaluation more generally.