Background: Across a range of policy areas and geographies, governments and philanthropists are increasingly looking to adopt a social outcomes contracting (SOC) approach. Under this model, an agreement is made that a provider of services must achieve specific, measurable social and/or environmental outcomes and payments are only made when these outcomes have been achieved. Despite this growing interest, there is currently a paucity of evidence in relation to the tangible improvement in outcomes associated with the implementation of these approaches. Although promising, evidence suggests that there are risks (especially around managing perverse incentives).[1] The growing interest in SOC has been accompanied by research of specific programmes, policy domains or geographies, but there has not been a systematic attempt to synthetise this emerging evidence. To address this gap, this systematic review aims to surface the best evidence on when and where effects have been associated with SOC.
Methods: This mixed-methods systematic review protocol has been prepared using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA-P) guidelines (Additional File 2) (Shamseer et al., 2010). The review aims to consult policymakers throughout the evidence synthesis process, by adopting a user-involved research process. This will include the establishment and involvement of a Policy Advisory Group (PAG). The PAG will consist of a large, diverse, international group of policy makers who are or have been actively involved in funding and shaping social outcomes contracts (Additional File 3). The following electronic databases will be searched: ABI/INFORM Global, Applied Social Sciences Index & Abstracts (ASSIA), Scopus, International Bibliography of the Social Sciences (IBSS), PAIS Index, PolicyFile Index, Proquest Dissertations and Theses, ProQuest Social Science, Social Services Abstracts, Web of Science, Worldwide Political Science Abstracts and PsycINFO. We will also conduct a comprehensive search of grey literature sources. Studies will be imported into Covidence and screened (after de-duplication) independently by two reviewers, using explicit inclusion/exclusion criteria. We will conduct risk of bias and quality assessment using recommended tools and we will extract data using a pre-piloted, standardised data extraction form. If meta-synthesis cannot be conducted for the effectiveness component, we will carry out a descriptive narrative synthesis of the quantitative evidence, categorised by type of intervention, type of outcome/s, population characteristics and/or policy sector. The qualitative studies will be synthesised using thematic content analysis (Thomas and Harden 2008). If possible, we will also analyse the available economic data to understand the costs and benefits associated with SOC. Finally, we will conduct a cross-study synthesis, which will involve bringing together the findings from the effectiveness review, economic review and qualitative review. We recognise that the proposed conventional effectiveness review method may lead to inconclusive or partial findings given the complexity of the intervention, the likely degree of heterogeneity and the under-developed evidence base. We see a traditional systematic review as an important foundation to describe the evidence landscape. We will use this formal review as a starting point and then explore more contextually rooted review work in future.
Discussion: We will use the systematic review findings to produce accessible and reliable empirical insights on whether, when, and where (and if possible, how) SOC approaches deliver improved impact when compared to more conventional funding arrangements. The outputs will support policymakers to make informed decisions in relation to commissioning and funding approaches.
Systematic review registration: This systematic review was registered with the International Prospective Register of Systematic Reviews (PROSPERO), on 20th November 2020 and was last updated on 21 January 2021: (registration number PROSPERO CRD42020215207).
[1] A perverse incentive in an outcomes-based contract is an incentive that has unintended and undesirable results. For instance, a poorly designed welfare-to-work scheme could create incentives for service providers to prioritise clients who are easier to help and to ‘park’ those who are harder to assist (NAO 2015).
Evidentiary function of systematic reviews of scientific literature: Epistemological foundations and methodological derivatives
