One Size Doesn’t Fit All: Administrative Data Quality Frameworks for Production of Official Statistics

Background with rationaleThe use of administrative data is key to achieving the UK Statistics Authority’s strategy of Better Statistics, Better Decisions. Integrating administrative data into official statistics can benefit policy decisions by allowing the possibility of greater granularity and improved timeliness in outputs, while delivering efficiency gains and reducing respondent burden. Quality assessment and communicating uncertainty of administrative data sources is critical to their effective integration into official statistical outputs. Main AimThis presentation will discuss the main challenges of quality assuring statistical outputs containing administrative data. The differences in existing quality frameworks and identified quality metrics will be discussed. In addition, the presentation will cover the need to tailor quality assessment to answer a specific research question that an identified source is being used for and the considerations required. Methods/ApproachA comprehensive literature review was carried out, bringing together existing quality frameworks and metrics from National Statistical Institutes (NSIs) and academia for production of statistics using administrative data sources. ResultsThe main challenges and considerations faced when quality assuring outputs produced using administrative sources have been identified. The quality requirements for different outputs across social, business and census statistics were summarised and a general quality framework for admin data developed. This framework draws on international best practices for use in the UK statistical system. ConclusionIntegrating administrative data presents challenges can’t be solved by a one-size fits all framework. Through unifying available guidance, an adaptable quality assurance methodology has been created, enabling the use of public data for the public good.

Download Full-text

A Quality Framework for Statistics based on Administrative Data Sources using the Example of the Austrian Census 2011

Austrian Journal of Statistics ◽

10.17713/ajs.v39i4.251 ◽

2016 ◽

Vol 39 (4) ◽

Cited By ~ 2

Author(s):

Christopher Berka ◽

Stefan Humer ◽

Manuela Lenk ◽

Mathias Moser ◽

Henrik Rechta ◽

...

Keyword(s):

Quality Assessment ◽

Administrative Data ◽

Quality Measures ◽

Data Sources ◽

Structural Quality ◽

Methodological Framework ◽

Process Flow ◽

Quality Framework ◽

Key Aspects

Along with the implementation of a register-based census we develop a methodological framework to assess administrative data sources for statistical use. Key aspects for the quality of these data are identified in the context of hyperdimensions and embedded into a process flow. Based on this approach we develop a structural quality framework and suggest a concept for quality assessment and several quality measures.

Download Full-text

Linked government administrative data: Public or Private?

International Journal for Population Data Science ◽

10.23889/ijpds.v4i3.1173 ◽

2019 ◽

Vol 4 (3) ◽

Author(s):

Joel Stafford

Keyword(s):

Public Good ◽

Communication Networks ◽

Administrative Data ◽

Public Service ◽

Policy Development ◽

Data Linkage ◽

The Public ◽

Government Services ◽

Service Contracts ◽

Public Data

Background with rationaleIt is commonplace in policy discussions concerning administrative data linkage to presuppose that the data referred to is government services data. But this is not always the case. Much of the data public services hold is now collected via intermediaries, such as Non-Government Organisations, operating under service contracts with one or multiple government departments. Nor are these the only administrative data holdings applicable to clients of government services. There are also vast private administrative data holdings – including utility data, and consumer behaviour data. Creating and amending legislation that governs public service practices in this domain is increasingly made complex when private companies partner with governments agencies on policy development and evaluation work. Understanding the concept of public data for public good in light of this expanded sense of administrative data opens the door to deeper questions about the role linked data can play in government decision making. Main aimThe paper problematizes how legislation governing the linking of government administrative data is scoped and discusses how public service work can be affected by the opaque communication networks that increasingly span the public-private sector divide. Methods/ApproachAfter contextualising the challenge of legislating for administrative data linkage in the current work of the Office of the National Data Commissioner (ONDC) in Australia, this paper tests aspects of the proposed legislation against the extent to which it permits the possibility of ‘data laundering’. ResultsThe presentation demonstrates the need for greater sophistication in the specification of data linkage and sharing legislation in service of the public good. Conclusions This paper indicates that contemporary practices governing the linkage of government administrative data holdings is porous to the aims of extra-governmental organisations and may benefit by better incorporating legislative structures that govern private analytical services entities.

Download Full-text

The control and ‘fitness for purpose’ of UK official statistics

Data in Society ◽

10.1332/policypress/9781447348214.003.0010 ◽

2019 ◽

pp. 119-132

Author(s):

David Rhind

Keyword(s):

New Technologies ◽

Data Sets ◽

Official Statistics ◽

Politics And Government ◽

The Public ◽

Economic Statistics ◽

Institutional Structures ◽

The Uk ◽

Government Statistics ◽

The Impact

This chapter describes the evolution of UK Official Statistics over an 80 year period under the influence of personalities, politics and government policies, new user needs and changing technology. These have led to changing institutional structures – such as the Statistics Commission - and periodic oscillations in what statistics are created and the ease of their accessibility by the public. The chapter concludes with the impact of the first major statistical legislation for 60 years, particularly as a consequence of its creation of the UK Statistics Authority. This has included major investment in quality assurance of National and Official Statistics and in professional resourcing. These changes are very welcome, as is the statutory specification of government statistics as a public good by the 2007 Statistics and Registration Service Act. But problems of access to some data sets and the pre-release of key economic statistics to selected groups of users remain. Given the widespread societal consequences of the advent of new technologies, what we collect and how we do it will inevitably continue to change rapidly.

Download Full-text

Canadian Approaches to Optimizing Quality of Administrative Data for Health System Use, Research, and Linkage

International Journal for Population Data Science ◽

10.23889/ijpds.v3i4.982 ◽

2018 ◽

Vol 3 (4) ◽

Author(s):

Catherine Eastwood ◽

Keith Denny ◽

Maureen Kelly ◽

Hude Quan

Keyword(s):

Data Collection ◽

Data Quality ◽

Quality Assessment ◽

Administrative Data ◽

Information Quality ◽

Health Data ◽

Data Sources ◽

University Of Calgary ◽

Quality Assessment And Improvement

Theme: Data and Linkage QualityObjectives: To define health data quality from clinical, data science, and health system perspectives To describe some of the international best practices related to quality and how they are being applied to Canada’s administrative health data. To compare methods for health data quality assessment and improvement in Canada (automated logical checks, chart quality indicators, reabstraction studies, coding manager perspectives) To highlight how data linkage can be used to provide new insights into the quality of original data sources To highlight current international initiatives for improving coded data quality including results from current ICD-11 field trials Dr. Keith Denny: Director of Clinical Data Standards and Quality, Canadian Insititute for Health Information (CIHI), Adjunct Research Professor, Carleton University, Ottawa, ON. He provides leadership for CIHI’s information quality initiatives and for the development and application of clinical classifications and terminology standards. Maureen Kelly: Manager of Information Quality at CIHI, Ottawa, ON. She leads CIHI’s corporate quality program that is focused on enhancing the quality of CIHI’s data sources and information products and to fostering CIHI’s quality culture. Dr. Cathy Eastwood: Scientific Manager, Associate Director of Alberta SPOR Methods & Development Platform, Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB. She has expertise in clinical data collection, evaluation of local and systemic data quality issues, disease classification coding with ICD-10 and ICD-11. Dr. Hude Quan: Professor, Community Health Sciences, Cumming School of Medicine, University of Calgary, Director Alberta SPOR Methods Platform; Co-Chair of Hypertension Canada, Co-Chair of Person to Population Health Collaborative of the Libin Cardiovascular Institute in Calgary, AB. He has expertise in assessing, validating, and linking administrative data sources for conducting data science research including artificial intelligence methods for evaluating and improving data quality. Intended Outcomes:“What is quality health data?” The panel of experts will address this common question by discussing how to define high quality health data, and measures being taken to ensure that they are available in Canada. Optimizing the quality of clinical-administrative data, and their use-value, first requires an understanding of the processes used to create the data. Subsequently, we can address the limitations in data collection and use these data for diverse applications. Current advances in digital data collection are providing more solutions to improve health data quality at lower cost. This panel will describe a number of quality assessment and improvement initiatives aimed at ensuring that health data are fit for a range of secondary uses including data linkage. It will also discuss how the need for the linkage and integration of data sources can influence the views of the data source’s fitness for use. CIHI content will include: Methods for optimizing the value of clinical-administrative data CIHI Information Quality Framework Reabstraction studies (e.g. physician documentation/coders’ experiences) Linkage analytics for data quality University of Calgary content will include: Defining/measuring health data quality Automated methods for quality assessment and improvement ICD-11 features and coding practices Electronic health record initiatives

Download Full-text

Combination of evidence from multiple administrative data sources: quality assessment of the Austrian register-based Census 2011

Statistica Neerlandica ◽

10.1111/j.1467-9574.2011.00506.x ◽

2011 ◽

Vol 66 (1) ◽

pp. 18-33 ◽

Cited By ~ 5

Author(s):

Christopher Berka ◽

Stefan Humer ◽

Mathias Moser ◽

Manuela Lenk ◽

Henrik Rechta ◽

...

Keyword(s):

Quality Assessment ◽

Administrative Data ◽

Data Sources

Download Full-text

Using Administrative Data to Explore the Effect of Survey Nonresponse in the UK Employment Retention and Advancement Demonstration

Evaluation Review ◽

10.1177/0193841x18807986 ◽

2019 ◽

pp. 0193841X1880798

Author(s):

Richard Dorsett ◽

Richard Hendra ◽

Philip K. Robins

Keyword(s):

Administrative Data ◽

Large Scale ◽

Data Sources ◽

Survey Nonresponse ◽

Full Sample ◽

Employment Retention ◽

The United Kingdom ◽

Data Collection And Analysis ◽

Research Designs ◽

The Uk

Background: Even a well-designed randomized control trial (RCT) study can produce ambiguous results. This article highlights a case in which full sample results from a large-scale RCT in the United Kingdom differ from results for a subsample of survey respondents. Objectives: Our objective is to ascertain the source of the discrepancy in inferences across data sources and, in doing so, to highlight important threats to the reliability of the causal conclusions derived from even the strongest research designs. Research design: The study analyzes administrative data to shed light on the source of the differences between the estimates. We explore the extent to which heterogeneous treatment impacts and survey nonresponse might explain these differences. We suggest checks which assess the external validity of survey measured impacts, which in turn provides an opportunity to test the effectiveness of different weighting schemes to remove bias. The subjects included 6,787 individuals who participated in a large-scale social policy experiment. Results: Our results were not definitive but suggest nonresponse bias is the main source of the inconsistent findings. Conclusions: The results caution against overconfidence in drawing conclusions from RCTs and highlight the need for great care to be taken in data collection and analysis. Particularly, given the modest size of impacts expected in most RCTs, small discrepancies in data sources can alter the results. Survey data remain important as a source of information on outcomes not recorded in administrative data. However, linking survey and administrative data is strongly recommended whenever possible.

Download Full-text

Advancing cross-centre research networks: learning from experience, looking to the future

International Journal for Population Data Science ◽

10.23889/ijpds.v3i4.609 ◽

2018 ◽

Vol 3 (4) ◽

Author(s):

Kerina Jones ◽

Sharon Heys ◽

Helen Daniels

Keyword(s):

Information Sharing ◽

Data Sharing ◽

Virtual Teams ◽

Administrative Data ◽

Large Scale ◽

Data Availability ◽

Research Networks ◽

The Public ◽

Information Governance ◽

The Uk

IntroductionMany jurisdictions have programmes for the large-scale reuse of health and administrative data that would benefit from greater cross-centre working. The Advancing Cross centre Research Networks (ACoRN) project considered barriers and drivers for joint working and information sharing using the UK Farr Institute as a case study, and applicable widely. Objectives and ApproachACoRN collected information from researchers, analysts, academics and the public to gauge the acceptability of sharing data across institutions and jurisdictions. It considered international researcher experiences and evidence from a variety of cross centre projects to reveal barriers and potential solutions to joint working. It reviewed the legal and regulatory provisions that surround data sharing and cross-centre working, including issues of information governance to provide the context and backdrop. The emerging issues were grouped into five themes and used to propose a set of recommendations. ResultsThe five themes identified were: organisational structures and legal entities; people and culture; information governance; technology and infrastructure; and finance and strategic planning. Recommendations within these included: standardised terms and conditions including agreements and contractual templates; performance indicators for frequency of dataset sharing; communities of practice and virtual teams to develop cooperation; standardised policies and procedures to underpin data sharing; an accredited quality seal for organisations sharing data; a dashboard for data availability and sharing; and adequate resource to move towards greater uniformity and to drive data sharing initiatives. Conclusion/ImplicationsThe challenges posed by cross-centre information sharing are considerable but the public benefits associated with the greater use of health and administrative data are inestimable, particularly as novel and emerging data become increasingly available. The proposed recommendations will assist in achieving the benefits of cross-centre working.

Download Full-text

Using Administrative Data to Explore the Effect of Survey Nonresponse in the UK Employment Retention and Advancement Demonstration

Evaluation Review ◽

10.1177/0193841x16674395 ◽

2016 ◽

Vol 42 (5-6) ◽

pp. 491-514 ◽

Cited By ~ 5

Author(s):

Richard Dorsett ◽

Richard Hendra ◽

Philip K. Robins

Keyword(s):

Administrative Data ◽

Large Scale ◽

Data Sources ◽

Survey Nonresponse ◽

Full Sample ◽

Employment Retention ◽

The United Kingdom ◽

Data Collection And Analysis ◽

Research Designs ◽

The Uk

Download Full-text

Trust, security and public interest: Striking the balance

International Journal for Population Data Science ◽

10.23889/ijpds.v5i3.1368 ◽

2020 ◽

Vol 5 (3) ◽

Cited By ~ 1

Author(s):

Elizabeth Waind

Keyword(s):

Administrative Data ◽

Public Interest ◽

Public Attitudes ◽

Public Support ◽

Minimum Standard ◽

Privacy And Security ◽

The Public ◽

Core Conditions ◽

The Uk ◽

Very High

This literature review explores previous work in relation to the UK public’s attitudes towards the sharing, linking and use of administrative data for research. It finds the public is broadly supportive of administrative data research if three core conditions are met: public interest, privacy and security, and trust and transparency. None of these three conditions is sufficient in isolation; rather, the literature shows public support is underpinned by a minimum standard of all three. However, it also shows that in certain cases where the standard of one condition is very high – for example, public interest – this could mean that of another – for example, privacy and security – may, if necessary, be lower. An appropriate balance must be struck, and the proposed benefit must outweigh the potential risk. Broad, conditional support for the use of administrative data in research has not only been found consistently, but has also been held over time, with data collection for the 16 studies included spanning more than a decade from 2006-2018. It is therefore, at this time, appropriate to move beyond widescale, general consultation on the use of administrative data for research and build upon existing knowledge by delving into specific areas of research. The purpose of such an approach would not be to consult on whether research using administrative data should be done – as has been the focus of previous literature – but rather to guide how, why and when it is done. Nevertheless, it is important to continue to monitor and respond to any changes to public attitudes and adapt approaches if necessary.

Download Full-text

Custodian Legal Culture - Enabling Multi-Stakeholder Research Collaborations Between Academia and Government in the UK

International Journal for Population Data Science ◽

10.23889/ijpds.v3i2.476 ◽

2018 ◽

Vol 3 (2) ◽

Author(s):

Stergios Aidinlis

Keyword(s):

Data Sharing ◽

Administrative Data ◽

Public Interest ◽

Driving Forces ◽

Social Research ◽

Academic Community ◽

Structured Interviews ◽

The Public ◽

The Uk ◽

Public Bodies

BackgroundEmpirical studies suggest that some public bodies in England are very reluctant to grant access to administrative data for various purposes. This poster presents the conclusions drawn in my so-far research on the driving forces of administrative discretion in respect of data sharing for social research in the public interest in England. ObjectivesThis poster aims to work towards answering a fundamental question for methodological models for engagement and research co-production between academia and government. This question is: what are the driving forces behind the exercise of data custodian discretion when it comes to deciding whether they will disclose it or not for research purposes? Methods (including data)First, this poster presents the findings of a qualitative case-study involving semi-structured interviews with individuals working for three different public bodies in England, two data providers and a body facilitating administrative data sharing for research. Second, it integrates a pilot survey which will aim to elicit the perspectives of ADR conference attendees, both admin data researchers and other stakeholders, on the crucial questions that revolve around the disclosure of data for research by different providers across the UK. FindingsI propose a distinction between structural (e.g. the law/ infrastructural decision-making models) and cultural (e.g. perceptions of data ownership / trust-distrust in data sharing collaborators) influences, claiming that the latter are more salient in steering custodian discretion to share administrative data for research in practice than the former. I identify five main candidate cultural drivers and elaborate on them. ConclusionsWithout a sound socio-legal understanding of the driving forces of discretionary legal powers to share data on behalf of the providers, building bridges between them and the academic community in the interest of promoting social research in the public interest will remain a resilient challenge.

Download Full-text