A qualitative and quantitative evaluation of the Advancing Quality pay-for-performance programme in the NHS North West

BackgroundAdvancing Quality (AQ) is a voluntary programme providing financial incentives for improvement in the quality of care provided to NHS patients in the north-west of England.Objectives(1) To identify the impact of AQ on key stakeholders and clinical practice; (2) to assess its cost-effectiveness; (3) to identify key factors that assist or impede its successful implementation; and (4) to provide lessons for the wider implementation of pay-for-performance schemes across the NHS.DesignWe tested whether or not the financial incentives of AQ had an impact on mortality using two methods: a between-region difference-in-differences analysis comparing the North West region and the rest of England for the incentivised and non-incentivised conditions and a triple-difference analysis comparing performance on the incentivised conditions, as well as the non-incentivised conditions, in the North West region and the rest of England. A cost-effectiveness analysis of AQ based on the first 18 months of the programme was also undertaken. We used interviews and observation to explore how and why changes occurred.ResultsRisk-adjusted mortality rates for all three of the conditions we studied (pneumonia, heart failure and myocardial infarction) decreased in both the North West region and the rest of England during the first 18 months of the scheme. The reduction in mortality for incentivised conditions was greater in the North West region than in the rest of England. Compared with non-incentivised conditions within the North West region, there was a significant reduction in overall mortality for incentivised conditions, comprising a statistically significant reduction in pneumonia and non-significant reductions in the other two conditions. Comparing mortality for the incentivised conditions with mortality for these conditions in other regions, there was a significant reduction in overall mortality in the North West region, again made up of individually significant reductions in pneumonia and non-significant reductions in the other two conditions. The reduction in mortality over the 18-month period studied for non-incentivised conditions was not significantly different between the North West region and the rest of England. The between-region difference-in-differences analysis after 42 months showed that risk-adjusted mortality for the incentivised conditions fell in the rest of England and the North West region. This reduction in the rest of England was significantly larger than in the North West region and was concentrated in pneumonia. However, the reductions in mortality were larger for the non-incentivised conditions in the North West region than in the rest of England between these periods. For incentivised conditions, the triple-difference analysis shows a larger reduction in mortality for the rest of England than in the North West region between the short- and long-term periods.ConclusionsBased on the first 18 months, the AQ programme was a relatively effective and cost-effective intervention. However, findings at 42 months are open to interpretation. One interpretation is that the short-term improvements were not sustained and that the observed improvements in mortality in the non-incentivised conditions within hospitals participating in AQ were unrelated to the programme. An alternative interpretation is that these improvements are related to the positive spillover effect of AQ. Further research should be undertaken to determine the explanation for the findings.FundingThe National Institute for Health Research Health Services and Delivery Research programme.