A novel method to assess data quality in large medical registries and databases

Background There is no gold standard to assess data quality in large medical registries. Data auditing may be impeded by data protection regulations. Objective To explore the applicability and usefulness of funnel plots as a novel tool for data quality control in critical care registries. Method The Swiss ICU-Registry from all 77 certified adult Swiss ICUs (2014 and 2015) was subjected to quality assessment (completeness/accuracy). For the analysis of accuracy, a list of logical rules and cross-checks was developed. Type and number of errors (true coding errors or implausible data) were calculated for each ICU, along with noticeable error rates (>mean + 3 SD in the variable’s summary measure, or >99.8% CI in the respective funnel-plot). Results We investigated 164 415 patient records with 31 items each (37 items: trauma diagnosis). Data completeness was excellent; trauma was the only incomplete item in 1495 of 9871 records (0.1%, 0.0%–0.6% [median, IQR]). In 15 572 patients records (9.5%), we found 3121 coding errors and 31 265 implausible situations; the latter primarily due to non-specific information on patients’ provenance/diagnosis or supposed incoherence between diagnosis and treatments. Together, the error rate was 7.6% (5.9%–11%; median, IQR). Conclusions The Swiss ICU-Registry is almost complete and data quality seems to be adequate. We propose funnel plots as suitable, easy to implement instrument to assist in quality assurance of such a registry. Based on our analysis, specific feedback to ICUs with special-cause variation is possible and may promote such ICUs to improve the quality of their data.