Evaluation of a national surveillance system for mortality alerts: a mixed-methods study

BackgroundSince 2007, Imperial College London has generated monthly mortality alerts, based on statistical process control charts and using routinely collected hospital administrative data, for all English acute NHS hospital trusts. The impact of this system has not yet been studied.ObjectivesTo improve understanding of mortality alerts and evaluate their impact as an intervention to reduce mortality.DesignMixed methods.SettingEnglish NHS acute hospital trusts.ParticipantsEleven trusts were included in the case study. The survey involved 78 alerting trusts.Main outcome measuresRelative risk of mortality and perceived efficacy of the alerting system.Data sourcesHospital Episodes Statistics, published indicators on quality and safety, Care Quality Commission (CQC) reports, interviews and documentary evidence from case studies, and a national evaluative survey.MethodsDescriptive analysis of alerts; association with other measures of quality; associated change in mortality using an interrupted time series approach; in-depth qualitative case studies of institutional response to alerts; and a national cross-sectional evaluative survey administered to describe the organisational structure for mortality governance and perceptions of efficacy of alerts.ResultsA total of 690 mortality alerts generated between April 2007 and December 2014. CQC pursued 75% (154/206) of alerts sent between 2011 and 2013. Patient care was cited as a factor in 70% of all investigations and in 89% of sepsis alerts. Alerts were associated with indicators on bed occupancy, hospital mortality, staffing, financial status, and patient and trainee satisfaction. On average, the risk of death fell by 58% during the 9-month lag following an alert, levelling afterwards and reaching an expected risk within 18 months of the alert. Acute myocardial infarction (AMI) and sepsis alerts instigated institutional responses across all the case study sites, although most sites were undertaking some parallel activities at a more general level to address known problems in care in these and other areas. Responses included case note review and coding improvements, changes in patient pathways, changes in diagnosis of sepsis and AMI, staff training in case note write-up and coding, greater transparency in patient deterioration, and infrastructure changes. Survey data revealed that 86% of responding trusts had a dedicated trust-level lead for mortality reduction and 92% had a dedicated trust-level mortality group or committee in place. Trusts reported that mortality reduction was a high priority and that there was strong senior leadership support for mortality monitoring. The weakest areas reported concerned the accuracy of coding, the quality of specialty-level mortality data and understanding trends in specialty-level mortality data.LimitationsOwing to the correlational nature of our analysis, we could not ascribe a causal link between mortality alerts and reductions in mortality. The complexity of the institutional context and behaviour hindered our capacity to attribute locally reported changes specifically to the effects of the alerts rather than to ongoing institutional strategy.ConclusionsThe mortality alert surveillance system reflects aspects of quality care and is valued by trusts. Alerts were considered a useful focus for identifying problems and implementing interventions around mortality.Future workA further analysis of site visits and survey material, the application of evaluative framework to other interventions, a blinded case note review and the dissemination of findings.FundingThe National Institute for Health Research Health Services and Delivery Research programme.