Saving 20 000 Days and Beyond: a realist evaluation of two quality improvement campaigns to manage hospital demand in a New Zealand District Health Board

BackgroundThe current paper reports on a realist evaluation of two consecutive quality improvement campaigns based on the Institute for Healthcare Improvement’s Breakthrough Series. The campaigns were implemented by a District Health Board to manage hospital demand in South Auckland, New Zealand. A realist evaluation design was adopted to investigate what worked in the two campaigns and under what conditions.MethodsA mixed-methods approach was used, involving three phases of data collection. During the first phase, a review of campaign materials and relevant literature, as well as key informant interviews were undertaken to generate an initial logic model of how the campaign was expected to achieve its objective. In phase II, the model was tested against the experiences of participants in the first campaign via a questionnaire to all campaign participants, interviews with campaign sponsors and collaborative team leaders and a review of collaborative team dashboards. In phase III, the refined model was tested further against the experiences of participants in the second campaign through interviews with collaborative team leaders, case studies of four collaborative teams and a review of the overall system-level dashboard.ResultsThe evaluation identified four key mechanisms through which the campaigns’ outcomes were achieved. These were characterised as ‘an organisational preparedness to change’, ‘enlisting the early adopters’, ‘strong collaborative teams’ and ‘learning from measurement’. Contextual factors that both enabled and constrained the operation of these mechanisms were also identified.ConclusionsBy focusing on the explication of a theory of how the campaigns achieved their outcomes and under what circumstances, the realist evaluation reported in this paper provides some instructive lessons for future evaluations of quality improvement initiatives.