First Things First: How to Elicit the Initial Program Theory for a Realist Evaluation of Complex Integrated Care Programs

Abstract Background One of the Sustainable Development Goals is to reduce the global maternal mortality ratio to less than 70 per 100,000 live births by 2030. In South Africa, the flagship National Department of Health MomConnect program was launched in 2014 to strengthen the quality of maternal and child health (MCH) services and improve mortality outcomes. MomConnect was rapidly rolled out with a limited understanding of how and why the program was expected to work even though studies had shown the effectiveness of the MomConnect program in improving the uptake of MCH services. This study aimed to unearth the initial program theory of the MomConnect program based on explicit and implicit assumptions of how the program was organized and expected to work. Methods We conducted a document analysis using design- and implementation-related documents of the MomConnect program guided by the principles of Theory of Change (ToC) and Realist Evaluation (RE). Content and thematic analysis approaches were deductively applied to analyze the documents toward constructing ToC and RE-informed models. Abductive thinking and retroduction were further applied to the realist-informed approach to link program context, mechanisms, and outcomes to construct the initial program theory. Results ToC and RE-informed models illustrated how the MomConnect program was organized and expected to work. The process of constructing the ToC provided the platform for the development of the initial program theory, which identified three critical elements: (1) the central modalities of the MomConnect program; (2) the intended outcomes; and (3) the tentative causal links indicating, in a stepwise manner of, how the outcomes were intended to be achieved. The RE approach ‘enhanced’ the causal links by identifying relevant programmatic contexts and linking the postulated mechanisms of action (empowerment, encouragement, motivation, and knowledge acquisition) to program outcomes. Conclusion The application of ToC and RE provided an explicitly cumulative approach to knowledge generation in unveiling the initial program theory of MomConnect rather than delivering answers to questions of program effectiveness.

Download Full-text

The Implementation of Healthy Start-Départ Santé intervention in Canadian Childcare Centres: A Realist Evaluation

10.21203/rs.3.rs-267547/v1 ◽

2021 ◽

Author(s):

Nazeem Muhajarine ◽

MA Fiona Fick ◽

Daphne N McRae

Keyword(s):

Physical Activity ◽

Behavioural Change ◽

Realist Evaluation ◽

Healthy Start ◽

Program Theory ◽

Physically Active ◽

Childcare Centre ◽

Positive Attitudes ◽

Initial Program ◽

Nutritional Needs

Abstract BackgroundHealthy Start-Départ Santé (HSDS) is a 10-month, bilingual (English and French) program which aims to educate childcare centre staff about the activity and nutritional needs of young children, as well as teaching them how to incorporate healthy practices into their daily routines. This study examines “How, for whom, and in what contexts is the HSDS intervention effectively implemented and sustained in early learning childcare centers?”MethodsWe conducted a realist evaluation to assess the HSDS program. After formulating an initial program theory (described in a previously published paper) we tested and refined the theory using interview data from childcare centre directors and staff. Purposive sampling was used to recruit centres with differing characteristics (levels of engagement with HSDS, length of time since training, and geographic and cultural diversity in clientele). Twenty-five interviews from 23 centres were completed. Participants were asked questions directly related to the program theory (e.g., Does the program line up with your personal values and goals around healthy child development?) to confirm, refute, or refine elements of the initial theory. Interviews were coded by context, mechanisms, and outcomes and separated according to the stages of Training, Implementation, and Sustainability. Codes specific to physical activity and nutrition were created. NVivo 11 was used for data management. ResultsOur study found that the HSDS program was effectively implemented and sustained when staff had positive attitudes toward the program, identified with its goals and values, and were willing to actively participate in activities. Other key components for staff included a sense of responsibility for child health, increased understanding of the benefits of promoting healthy behaviours, and access to resources in support of behavioural change. For children, the program was successful when they had multiple opportunities to be physically active and try a variety of foods, and when they could develop competence in physical activities. ConclusionsThe HSDS final program theory can be tailored to various contexts and/or similar programs to facilitate improved physical activity and healthy eating in childcare centres.

Download Full-text

Understanding integrated care pathways in palliative care using realist evaluation: a mixed methods study protocol: Table 1

BMJ Open ◽

10.1136/bmjopen-2012-001533 ◽

2012 ◽

Vol 2 (4) ◽

pp. e001533 ◽

Cited By ~ 16

Author(s):

Sonia Michelle Dalkin ◽

Diana Jones ◽

Monique Lhussier ◽

Bill Cunningham

Keyword(s):

Palliative Care ◽

Mixed Methods ◽

Integrated Care ◽

Study Protocol ◽

Realist Evaluation ◽

Mixed Methods Study ◽

Care Pathways ◽

Integrated Care Pathways

Download Full-text

A realist evaluation of local networks designed to achieve more integrated care

International Journal of Integrated Care ◽

10.5334/ijic.s3632 ◽

2019 ◽

Vol 19 (4) ◽

pp. 632

Author(s):

Harry Rea ◽

Jacqueline Cumming ◽

Megan Pledger ◽

Lesley Middleton

Keyword(s):

Integrated Care ◽

Realist Evaluation ◽

Local Networks

Download Full-text

Facebook as a Medium for the Support and Enhancement of Ambulatory Care for People With Diabetes: Qualitative Realist Evaluation of a Real-World Trial

JMIR Diabetes ◽

10.2196/18146 ◽

2020 ◽

Vol 5 (3) ◽

pp. e18146

Author(s):

Bryan Cleal ◽

Ingrid Willaing ◽

Mette T Hoybye ◽

Henrik H Thomsen

Keyword(s):

Social Media ◽

Data Gathering ◽

Realist Evaluation ◽

Program Theory ◽

Primary Data ◽

Provider Communication ◽

New Developments ◽

Evaluation Approach ◽

Diabetes Technology ◽

Patient Provider Communication

Background There is a growing focus on the potential uses, benefits, and limitations of social media in the context of health care communication. In this study, we have sought to evaluate an initiative pioneered at a hospital in Denmark that uses Facebook to support and enhance patient-provider communication about diabetes. Objective This paper aims to evaluate the success of the trial according to its initial objectives and to assess its potential scalability. Methods The study was undertaken in a clinic for diabetes and hormonal diseases at a large regional hospital in Denmark. Using a realist evaluation approach, we identified 4 key components in the program theory of the initiative, which we formulated as context-mechanism-outcome configurations (eg, complex and iterative chains of causality). These configurations informed data gathering and analysis. Primary data sources were the activity and content in the Facebook group, in the form of posts, likes, and comments, and interviews with patients (n=26) and staff (n=6) at the clinic. Results New developments in diabetes technology were the most popular posts in the forum, judged by number of likes and comments. Otherwise, information specific to the clinic received the most attention. All 4 components of the program theory were compromised to varying degrees, either as a result of failings in the anticipated mechanisms of change or contextual factors derived from the mode of implementation. Conclusions Social media serves well as a conduit for imagining positive change, but this can be a strength and weakness when attempting to enact change via concrete interventions, where stakeholder expectations may be unreasonably high or incompatible. Nonetheless, such initiatives may possess intangible benefits difficult to measure in terms of cost-effectiveness.

Download Full-text

Characterizing performance improvement in primary care systems in Mesoamerica: A realist evaluation protocol

Gates Open Research ◽

10.12688/gatesopenres.12782.1 ◽

2018 ◽

Vol 2 ◽

pp. 1 ◽

Cited By ~ 5

Author(s):

Wolfgang Munar ◽

Syed S. Wahid ◽

Leslie Curry

Keyword(s):

Primary Care ◽

El Salvador ◽

Large Scale ◽

Grey Literature ◽

Realist Evaluation ◽

Program Theory ◽

Necessary Condition ◽

Observation Data ◽

Care Systems ◽

Preliminary Program

Background. Improving performance of primary care systems in low- and middle-income countries (LMICs) may be a necessary condition for achievement of universal health coverage in the age of Sustainable Development Goals. The Salud Mesoamerica Initiative (SMI), a large-scale, multi-country program that uses supply-side financial incentives directed at the central-level of governments, and continuous, external evaluation of public, health sector performance to induce improvements in primary care performance in eight LMICs. This study protocol seeks to explain whether and how these interventions generate program effects in El Salvador and Honduras. Methods. This study presents the protocol for a study that uses a realist evaluation approach to develop a preliminary program theory that hypothesizes the interactions between context, interventions and the mechanisms that trigger outcomes. The program theory was completed through a scoping review of relevant empirical, peer-reviewed and grey literature; a sense-making workshop with program stakeholders; and content analysis of key SMI documents. The study will use a multiple case-study design with embedded units with contrasting cases. We define as a case the two primary care systems of Honduras and El Salvador, each with different context characteristics. Data will be collected through in-depth interviews with program actors and stakeholders, documentary review, and non-participatory observation. Data analysis will use inductive and deductive approaches to identify causal patterns organized as ‘context, mechanism, outcome’ configurations. The findings will be triangulated with existing secondary, qualitative and quantitative data sources, and contrasted against relevant theoretical literature. The study will end with a refined program theory. Findings will be published following the guidelines generated by the Realist and Meta-narrative Evidence Syntheses study (RAMESES II). This study will be performed contemporaneously with SMI’s mid-term stage of implementation. Of the methods described, the preliminary program theory has been completed. Data collection, analysis and synthesis remain to be completed.

Download Full-text

A realist evaluation of local networks designed to achieve more integrated care

10.26686/wgtn.14265725.v1 ◽

2021 ◽

Author(s):

Harry Rea ◽

Jacqueline Cumming ◽

Megan Pledger ◽

Lesley Middleton

Keyword(s):

Integrated Care ◽

Realist Evaluation ◽

Local Networks

No description supplied

Download Full-text

Developing an Initial Program Theory to Explain How Patient-Reported Outcomes Are Used in Health Care Settings: Methodological Process and Lessons Learned

International Journal of Qualitative Methods ◽

10.1177/1609406920916299 ◽

2020 ◽

Vol 19 ◽

pp. 160940692091629

Author(s):

Rachel Flynn ◽

Kara Schick-Makaroff ◽

Adrienne Levay ◽

Joanne Greenhalgh

Keyword(s):

Health Care ◽

Patient Reported Outcomes ◽

Research Question ◽

Complex Interventions ◽

Lessons Learned ◽

Program Theory ◽

Health Care Settings ◽

Central Aspect ◽

Patient Reported ◽

Initial Program

A central aspect of any theory-driven realist investigation (synthesis or evaluation) is to develop an initial program theory (IPT). An IPT can be used to frame and understand how, for whom, why, and under what contexts complex interventions work or not. Despite well-established evidence that IPTs are a central aspect to any realist investigation, there is wide variation and a lack of methodological discussion on how to develop an IPT. In this article, we present the approach that we used to develop an IPT of how patient-reported outcomes (PROs) are used in health care settings. Specifically, we completed a systematic review to extract tacit theories reported in the literature. The benefit of this approach was that it provided a rigorous review of the literature in the development of IPTs. The challenges included (1) rediscovering what is already well established in the theoretical literature, (2) generating an overabundance of partial candidate theories, and (3) extensive use of time and resources for what was the first stage to our larger funded research study. Our recommendations to other scholars considering this approach are to ensure that they (1) live within their means and (2) narrow the scope of the research question and/or develop a conceptual framework using middle-range theories. These methodological insights are highly relevant to researchers embarking on a realist investigation, tasked with developing an IPT.

Download Full-text

Realism and rhetoric in the evaluation of a new care model

Journal of Integrated Care ◽

10.1108/jica-11-2018-0067 ◽

2019 ◽

Vol 27 (3) ◽

pp. 204-214 ◽

Cited By ~ 1

Author(s):

Tom Grimwood

Keyword(s):

Integrated Care ◽

Large Scale ◽

Realist Evaluation ◽

Scale Transformation ◽

Care Model ◽

Content Type ◽

Realist Approach ◽

Dynamic Form ◽

Multiple Audiences ◽

Objective System

Purpose The purpose of this paper is to discuss the methodological challenges to evaluating one of the 50 vanguard sites of the new care model (NCM) programme for integrated care in England, and make the case for a modified realist approach to this kind of evaluation. Design/methodology/approach The paper considers three challenges to evaluating the NCM in this particular vanguard: complexity, strategy and rhetoric. It reflects on how the realist approach negotiates these philosophical challenges to delivering integrated care, in order to provide contextualised accounts of who a programme works for, in what context, and why. Findings The paper argues that, in the case of this particular vanguard site, the tangible benefit of the realist approach was not in providing a firm epistemological basis for evaluation, but rather in drawing out and articulating the ontological rhetoric of such large-scale transformation programmes. By understanding the work of the NCM less as an objective “system”, and more as a dynamic form of persuasion, aimed at securing the “adherence of minds” (Perelman and Olbrechts-Tyteca, 2008, p. 8) in multiple audiences, the paper suggests that realist evaluation can be used to address both the systematic issues and localised successes the NCMs encountered. Originality/value The paper identifies a number of aspects of new models of integrated care for evaluators to consider. It offers ways of negotiating the challenges to conventional outcome-focused evaluation, by drawing attention to the need for contextualised, time-situated and audience-sensitive value of NCMs.

Download Full-text