The Rhetorical Superiority of Poor Economics

This textual analysis highlights the rhetorical prowess of Poor Economics and argues that it is part of the puzzling success of RCTs in economics. It shows how Banerjee and Duflo combine effectively (1) logos (rational discourse, figures, extensive use of numbers), (2) pathos (striking anecdotes to move the reader, personified figures) and (3) ethos (the narrators exhibit wisdom, excellence, and good will). Despite their explicit discard of anecdotes, the authors make ubiquitous use of them. This becomes less paradoxical when considering their manifold persuasive functions. Anecdotes additionally have an understated but inchoate heuristic role. This chapter also scrutinizes two impactful rhetorical schemes: the sensible middle ground between two extremes, and the rhetoric of small measures producing big effects, which magnifies the micro and minimizes the macro. This canny, partly manipulative, rhetoric should not overshadow the thinness of its storytelling and the extent of blind spots in RCTs revealed by what is out-of-discourse.

Microfinance RCTs in Development

Microcredit has long stood as a flagship topic for RCTs in development, starting with the publication of a special issue in a leading economics journal on six RCTs conducted in different world regions. This special issue was hailed as the first rigorous and conceivably definitive study on the impacts of microcredit. However, a detailed exploration of the implementation of these six RCTs reveals many limitations with respect to internal and external validity, ethics, and interpretation. This chapter uses analytical tools from statistics, political economy, and development anthropology to discuss the extent to which the entire RCT chain strays from the ideal RCT principles (from sampling, data collection, data entry and recoding, estimates and interpretation to publication and dissemination of results). It also raises questions about the disparity between the academic and political success of this special issue and the many inconsistencies of method.

Reducing the Knowledge Gap in Global Health Delivery

Randomized control trials (RCTs) are considered to be the gold standard for impact evaluation in international development and they are associated with a new era of evidence-based global health policies. However, there are inherent challenges in using RCTs to answer some of the most important questions in global health: why, if solutions are known, affordable at scale, and supported by existing evidence, do hundreds of millions of people lack access to essential health services? A lack of clarity on appropriate research methods for strengthening health systems has corresponded to a lack of investment in more complex and adaptive systems of integrated care delivery. This chapter reviews the use of RCTs in global health, highlighting major contributions, and addressing some pressing priorities in implementation research at a time when the Sustainable Development Goals emphasize the importance of sector-wide approaches, such as integrated primary care and universal health coverage.

Epilogue: Randomization and Social Policy Evaluation Revisited

This chapter examines the case for randomized controlled trials in economics. It revisits the author’s previous paper “Randomization and Social Policy Evaluation” and updates its message. The chapter presents a brief summary of the history of randomization in economics. It identifies two waves of enthusiasm for the method as “Two Awakenings” because of the near-religious zeal associated with each wave. The First Wave substantially contributed to the development of microeconometrics because of the awed nature of the experimental evidence. The Second Wave has improved experimental designs to avoid some of the technical statistical issues identified by econometricians in the wake of the First Wave. However, the deep conceptual issues about parameters estimated, and the economic interpretation and the policy relevance of the experimental results have not been addressed in the Second Wave.

Are the “Randomistas” Evaluators?

In an article published in 2012 the author concluded that the surge of enthusiasm in randomization was bound to be short lived. But he had underestimated the public appeal of RCTs and their alignment with the evolving demands of a contemporary evaluation market dominated by vested interests. By now, it has become clear that the randomization bubble will not burst any time soon. Grounded in deep historical roots, favored by power-holders and considered uniquely rigorous by an ill-informed public, RCTs will continue to be commissioned even though they are severely constrained by statistical imperatives and ethical concerns, can only tackle narrow social research questions and are ineffective as tools of organization accountability and learning.

Using Priors in Experimental Design

I argue that a big issue facing all academic studies is whether and how to leverage decision-makers’ priors in study design. Both RCTs and non-RCTs could more efficiently inform policy-making if priors were taken into consideration. Non-RCTs can exploit deterministic assignment to maximize the information value of the study. On the other hand, non-RCTs also appear more subject to specification searching, which should lead to decision-makers being more uncertain about how to interpret a study’s results and consequentially perhaps being more convinced by evidence from RCTs and willing to experiment more with leveraging prior beliefs. I discuss these issues and explore the potential value of information benefits to using priors in study design by simulation.

Ethics of RCTs

Equipoise is defined by Freedman (1987: 141) as a “state of genuine uncertainty on the part of the clinical investigator regarding the comparative therapeutic merits of each arm in a trial.” This principle is grounded in the ethical motivation that any ex-ante preference for a given option would undermine the interests of those who are offered another. Randomized controlled trials (RCTs) in development economics disregard the equipoise requirement by typically disadvantaging the control group. This chapter investigates how the equipoise principle is formalized in the medical literature and discusses whether and how it should be taken into consideration by economists. It argues that equipoise is especially relevant when double (or even single) blindness is excluded and when the control group includes already vulnerable individuals. More generally, this chapter advocates for developing a vibrant ethics conversation on the design and fairness of RCTs in social sciences.

Randomizing Development

An important argument for the increased use of randomized control trial methods in development is that the evidence from these studies will encourage the uptake of effective programs and projects (both through discouraging ineffective projects and improving design of new projects) and this will lead to reduced poverty and improved human well-being. However, cross-national evidence shows that the four-fold transformation of national development, to higher productivity economies, to more responsive states, the more capable organizations and administration and to more equal social treatment produces gains in poverty and human well-being that are orders of magnitude bigger than the best that can be hoped from better programs. Arguments that RCT research is a good (much less “best”) investment depend on both believing in an implausibly low likelihood that non-RCT research can improve progress national development and believing in an implausibly large likelihood that RCT evidence improves outcomes.

Interviews

Rémy Rioux, you head an institution (AFD) with a mission to fund development projects, programmes and policies. What does evaluation entail in your institution and what role does it play in your and your partners’ work? To clearly understand the role ascribed to evaluation, let’s first situate it in the Agence Française de Développement’s work cycle in the light of what I refer to in my book ...

Trials and Tribulations

This chapter introduces the WASH (Water Sanitation and Health) sector’s conversations about randomized intervention studies and draws lessons for development policy, more generally. Sanitation is a valuable case because, on the one hand, improving sanitation is widely recognized as a critical part of the development process, but, on the other hand, WASH interventions are often less well suited for randomized intervention evidence than other topics in health science or development economics. The chapter discusses a recent set of randomized trials which, far from definitively settling the important questions of rural sanitation policy, have renewed confusion and debate in the sector. Because even flawlessly designed and implemented sanitation interventions are likely to have different effects from one another and in different contexts, facts and theories from non-RCT sources are necessary (in addition to RCTs) to provide full and timely answers to sanitation policy questions. Finally, a case is made for the increased use of RCTs in the WASH sector where they might be more likely to help: to answer questions about behavior, rather than about health.