Importance and Utilization of Theory-Based Evaluations in the Context of Sustainable Development and Social-Ecological Systems

Numerous challenges confront the task of evaluating sustainable development—its complex nature, complementary evaluation criteria, and the difficulty of evaluation at the nexus of human and natural systems. Theory-based evaluation, drawn from critical realism, is well suited to this task. When constructing a program theory/theory of change for evaluating sustainable development, concepts of socioecological systems and coupled human and natural systems are useful. The chapter discusses four modes of inference and the application of different theory-based evaluation approaches. It introduces the CHANS (coupled human and natural systems) framework, a holistic, analytical framework that is useful in evaluating such complex, social-ecological systems and resonates with the challenging elements of sustainable development evaluation.

Transformational Change for Achieving Scale: Lessons for a Greener Recovery

Achieving transformational changes that can be then effectively scaled up requires ambition in design, a supportive policy environment, sound project design and implementation, partnerships, and multistakeholder participation. This chapter presents a framework that can be applied at the design stage to plan for change and scaling up and provides relevant lessons based on GEF interventions. Achieving change and scale can be an iterative and a continuous process until impacts are generated at the magnitude and scope of the targeted scale. Successful transformations typically adopt a systems approach and address multiple constraints to attain environmental and other socioeconomic impacts.

Evaluating Transformational Adaptation in Smallholder Farming: Insights from an Evidence Review

The literature on smallholder farming and climate change adaptation (CCA) has predominantly investigated the barriers to and determinants of farmer uptake of adaptation interventions. Although useful, this evidence fails to highlight the changes or persistence of adaptation responses over time. Studies usually adopt a narrow focus on incremental actions that provide limited insights into transformative adaptation pathways and how fundamental shifts in policy can address the root causes of vulnerability across different sectors and dimensions. Drawing on an evidence synthesis commissioned by the International Fund for Agricultural Development’s Independent Office of Evaluation, this chapter outlines how lessons from CCA interventions can be transferred via three learning domains that are essential for transformational change: scaling-up (in its multiple forms), knowledge management, and the human-environment nexus. We discuss the implications of our findings on monitoring, evaluation, and learning, highlighting the challenges that evaluators may face in capturing (a) the persistence or durability of transformational pathways, (b) the complexity of “super-wicked” problems, and (c) the relevance of context-dependent dynamics, within a landscape setting. We also address the contribution of evidence reviews to contemporary debates around development policy linked to climate change and agriculture, and the implications and value of such reviews to provide independent scientific rigor and robustness to conventional programmatic evaluations.

Assessing the Evaluability of Adaptation-Focused Interventions: Lessons from the Adaptation Fund

Evaluability assessments (EAs) have differing definitions, focus on various aspects of evaluation, and have been implemented inconsistently in the last several decades. Climate change adaptation (CCA) programming presents particular challenges for evaluation given shifting baselines, variable time horizons, adaptation as a moving target, and uncertainty inherent to climate change and its extreme and varied effects. The Adaptation Fund Technical Evaluation Reference Group (AF-TERG) developed a framework to assess the extent to which the Fund’s portfolio of projects has in place structures, processes, and resources capable of supporting credible and useful monitoring, evaluation, and learning (MEL). The framework was applied on the entire project portfolio to determine the level of evaluability and make recommendations for improvement. This chapter explores the assessment’s findings on designing programs and projects to help minimize the essential challenges in the field. It discusses how the process of EA can help identify opportunities for strengthening both evaluability and a project’s MEL more broadly. A key conclusion was that the strength and quality of a project’s overall approach to MEL is a major determinant of a project’s evaluability. Although the framework was used retroactively, EAs could also be used prospectively as quality assurance tools at the pre-implementation stage.

Evaluation for Transformational Change: Learning from Practice

The COVID-19 crisis has challenged the evaluation profession by altering the framework within which it operates. Evaluators must embrace new realities and respond to changes while not altering their principles, norms, and standards.A review of how evaluation networks and offices have responded to changing demands showed lack of recognition for the long-term implications to the profession. Commissioners and users of evaluation now have new priorities, and nontraditional actors have entered the traditional evaluation space, offering similar expertise and meeting the demands of evaluation commissioners and users. The extensive development challenges posed by COVID-19 require a comprehensive response capacity from evaluation if it is to be transformative as a profession.This chapter draws on national and international case studies, examining the concept of transformation from a contextual perspective and noting the relativism in the concept. It draws links between aspects, suggesting that this period is an opportunity for evaluators to learn from practice around transformation, and suggests that flexibility provides an opportunity to remain relevant and advance transformational goals.

Assessing Sustainability in Development Interventions

Sustainability is often claimed as an impact in development interventions although there is rarely a shared understanding of what it means, how to design for it, and especially how to assess the likelihood that intended streams of benefits will continue. This chapter asserts that to design and later to evaluate an intervention with sustainable impacts, the intervention must deepen indigenous capabilities to manage the program, to solve problems, and to innovate. The design and implementation also must operate within environmental boundaries, not extracting resources beyond the ability to regenerate or degrading environmental services—that is, design and implementation must incorporate the primacy of the environment. A postprogram evaluation 3–10 years after a program has ended provides evidence on whether the program is likely to have sustainable impacts. A case study of an asset transfer program in Malawi highlights the criteria for evaluating sustainability: deepened capabilities and social capital, reinvestment in program activities, and the development of backward and forward linkages catalyzing growing economic opportunities.

Transformational Change for People and the Planet: Evaluating Environment and Development – Introduction

The world is facing multiple crises as manifested in runaway climate change, a global pandemic, loss of ecosystems and biological species, and rapidly growing inequality. These are all closely interlinked as recognized in the 2030 Agenda for Sustainable Development. Addressing them will require broad transformational change that encompasses the economy, institutions, and how we interact with the natural environment. This chapter introduces the book that is intended to highlight how evaluation can contribute to such transformations. The chapter first reviews the state of development evaluation. It then briefly introduces the state of the global environment before discussing the implications of this context for evaluation, and how evaluation as a profession and practice must change in order to respond to the challenges of sustainability. The chapter ends by explaining the flow of the book in its four parts that focus on: transformational change, drivers of sustainability, climate change mitigation and adaptation, and evaluation approaches.

Evaluation at the Endgame: Evaluating Sustainability and the SDGs by Moving Past Dominion and Institutional Capture

Three facts underlay this chapter. First, the human system and all our ambitions for improving the human system depend on sustainable natural systems. Second, we do not have much time. On track to fall well short of all sustainability goals, the climate and sustainability crises grow and extinction looms. Third, up to this point evaluation has shown little interest in sustainability, yet evaluation potentially addresses the very questions that are central to informing and guiding rapid adaptation of human behavior to successfully surmounting extinction.Business-as-usual evaluation will not suffice. At the endgame with extinction looming, we need an evaluation that is more nimble, keeps up with rapidly accelerating knowledge, is relentlessly use-seeking and that guides the way to joined-up approaches. The evaluation we need will systematically mainstream sustainability across all evaluations and interventions, in all evaluation criteria and standards. For this, all evaluations will always address nexus where human and natural systems join and incorporate knowledge and methods from both systems. Existing evaluation knowledge is well suited to this task, as are knowledges in biophysical sciences. We know and promote knowledge processes for integrative evaluation and are starting to shift toward the requirements for evaluation at the nexus. As this chapter shows, the anchors holding us back are political, not technical.

Sustainability After Project Completion: Evidence from the GEF

This chapter examines the extent to which completed GEF projects are sustainable and the factors affecting sustainability. We considered only those projects that were covered through postcompletion evaluation at least 2 years after implementation completion, and where the evaluation reports provided adequate information related to observed sustainability during the postcompletion period. We assessed 62 projects to meet the selection criteria, then completed a desk review of the postcompletion evaluation reports and other relevant documents for these projects to assess the extent to which the project outcome was sustainable.We found that the projects covered through postcompletion evaluations were generally sustainable, with the sustainability outlook deteriorating for some projects while improving for others. The incidence of the catalytic processes that enhance sustainability—sustaining, mainstreaming, replication, scaling-up, and market change—was higher at postcompletion evaluation, as the passage of time allows long-term project outcomes to manifest. At the project level, we observed these catalytic processes in a wider set of activities at postcompletion evaluation than at implementation completion. Factors such as financial support for follow-up, political support, follow-up by and capacities of the executing agency, stakeholder buy-in, and project design seem to play a crucial role in determining project sustainability.

Pathway to the Transformative Policy of Agenda 2030: Evaluation of Finland’s Sustainable Development Policy

Finland has been the first country in the world to conduct a comprehensive evaluation of the national implementation the Agenda 2030. The purpose of the evaluation was to support efficient implementation of the agenda by producing information on the nation’s sustainability work for all administrative branches. The evaluation results are used for coherence in the policies and long-term sustainable development activities. The evaluation produced concrete recommendations on future directions for sustainable development policy. It also proposed future evaluation approaches.In this chapter, the authors present the evaluation approach and discuss the key results and their usage. They identify the essential elements of the utility of the evaluation in contributing to national progress of sustainable development policy. The Agenda 2030 evaluation approach was developmentally oriented and conducted in a very participatory manner. The authors reflect on the evaluative lessons learned and future options. They encourage emphasis on learning throughout the evaluation process even more in policy-level evaluations, and special attention to usefulness of the evaluation results already in evaluation design. Designing inclusive evaluation processes is a crucial precondition for evidence-informed learning and decision making in promoting transformative policy in the country context.