Between the Spreadsheets

2021 ◽  
Author(s):  
Susan Walsh
Keyword(s):  
Data Quality ◽  
Deep Understanding ◽  
Data Classification ◽  
Dirty Data ◽  
Quality Issues ◽  
Book Covers ◽  
Level Of Experience ◽  
The Impact

Dirty data is a problem that costs businesses thousands, if not millions, every year. In organisations large and small across the globe you will hear talk of data quality issues. What you will rarely hear about is the consequences or how to fix it.<br><br><i>Between the Spreadsheets: Classifying and Fixing Dirty Data</i> draws on classification expert Susan Walsh's decade of experience in data classification to present a fool-proof method for cleaning and classifying your data. The book covers everything from the very basics of data classification to normalisation, taxonomies and presents the author's proven <b>COAT</b> methodology, helping ensure an organisation's data is <b>Consistent</b>, <b>Organised</b>, <b>Accurate</b> and <b>Trustworthy</b>. A series of data horror stories outlines what can go wrong in managing data, and if it does, how it can be fixed. <br><br>After reading this book, regardless of your level of experience, not only will you be able to work with your data more efficiently, but you will also understand the impact the work you do with it has, and how it affects the rest of the organisation.<br><br>Written in an engaging and highly practical manner, <i>Between the Spreadsheets</i> gives readers of all levels a deep understanding of the dangers of dirty data and the confidence and skills to work more efficiently and effectively with it.

scholarly journals Data Quality Issues With Physician-Rating Websites: Systematic Review (Preprint)

2019 ◽  
Author(s):  
Pavankumar Mulgund ◽  
Raj Sharman ◽  
Priya Anand ◽  
Shashank Shekhar ◽  
Priya Karadi
Keyword(s):  
Data Quality ◽  
English Language ◽  
Quality Of Data ◽  
Contextual Data ◽  
Patient Reported ◽  
Physician Rating ◽  
Quality Framework ◽  
Quality Issues ◽  
The Impact

BACKGROUND In recent years, online physician-rating websites have become prominent and exert considerable influence on patients’ decisions. However, the quality of these decisions depends on the quality of data that these systems collect. Thus, there is a need to examine the various data quality issues with physician-rating websites. OBJECTIVE This study’s objective was to identify and categorize the data quality issues afflicting physician-rating websites by reviewing the literature on online patient-reported physician ratings and reviews. METHODS We performed a systematic literature search in ACM Digital Library, EBSCO, Springer, PubMed, and Google Scholar. The search was limited to quantitative, qualitative, and mixed-method papers published in the English language from 2001 to 2020. RESULTS A total of 423 articles were screened. From these, 49 papers describing 18 unique data quality issues afflicting physician-rating websites were included. Using a data quality framework, we classified these issues into the following four categories: intrinsic, contextual, representational, and accessible. Among the papers, 53% (26/49) reported intrinsic data quality errors, 61% (30/49) highlighted contextual data quality issues, 8% (4/49) discussed representational data quality issues, and 27% (13/49) emphasized accessibility data quality. More than half the papers discussed multiple categories of data quality issues. CONCLUSIONS The results from this review demonstrate the presence of a range of data quality issues. While intrinsic and contextual factors have been well-researched, accessibility and representational issues warrant more attention from researchers, as well as practitioners. In particular, representational factors, such as the impact of inline advertisements and the positioning of positive reviews on the first few pages, are usually deliberate and result from the business model of physician-rating websites. The impact of these factors on data quality has not been addressed adequately and requires further investigation.

scholarly journals Developing a standardised approach to the aggregation of inpatient episodes into person-based spells in all specialties and psychiatric specialties

2019 ◽  
Vol 19 (1) ◽  
Author(s):  
Sarah Rees ◽  
Ashley Akbari ◽  
Huw Collins ◽  
Sze Chim Lee ◽  
Amanda Marchant ◽  
...  
Keyword(s):  
Data Quality ◽  
Health Record ◽  
Discharge Destination ◽  
Patient Journey ◽  
Single Episode ◽  
Successful Method ◽  
Admission Source ◽  
Quality Issues ◽  
Transfers Of Care ◽  
The Impact

Abstract Background Electronic health record (EHR) data are available for research in all UK nations and cross-nation comparative studies are becoming more common. All UK inpatient EHRs are based around episodes, but episode-based analysis may not sufficiently capture the patient journey. There is no UK-wide method for aggregating episodes into standardised person-based spells. This study identifies two data quality issues affecting the creation of person-based spells, and tests four methods to create these spells, for implementation across all UK nations. Methods Welsh inpatient EHRs from 2013 to 2017 were analysed. Phase one described two data quality issues; transfers of care and episode sequencing. Phase two compared four methods for creating person spells. Measures were mean length of stay (LOS, expressed in days) and number of episodes per person spell for each method. Results 3.5% of total admissions were transfers-in and 3.1% of total discharges were transfers-out. 68.7% of total transfers-in and 48.7% of psychiatric transfers-in had an identifiable preceding transfer-out, and 78.2% of total transfers-out and 59.0% of psychiatric transfers-out had an identifiable subsequent transfer-in. 0.2% of total episodes and 4.0% of psychiatric episodes overlapped with at least one other episode of any specialty. Method one (no evidence of transfer required; overlapping episodes grouped together) resulted in the longest mean LOS (4.0 days for all specialties; 48.5 days for psychiatric specialties) and the fewest single episode person spells (82.4% of all specialties; 69.7% for psychiatric specialties). Method three (evidence of transfer required; overlapping episodes separated) resulted in the shortest mean LOS (3.7 days for all specialties; 45.8 days for psychiatric specialties) and the most single episode person spells; (86.9% for all specialties; 86.3% for psychiatric specialties). Conclusions Transfers-in appear better recorded than transfers-out. Transfer coding is incomplete, particularly for psychiatric specialties. The proportion of episodes that overlap is small but psychiatric episodes are disproportionately affected. The most successful method for grouping episodes into person spells aggregated overlapping episodes and required no evidence of transfer from admission source/method or discharge destination codes. The least successful method treated overlapping episodes as distinct and required transfer coding. The impact of all four methods was greater for psychiatric specialties.

scholarly journals Quantifying the Effect of Data Quality on the Validity of an eMeasure

2017 ◽  
Vol 08 (04) ◽  
pp. 1012-1021 ◽  
Author(s):  
Steven Johnson ◽  
Stuart Speedie ◽  
Gyorgy Simon ◽  
Vipin Kumar ◽  
Bonnie Westra
Keyword(s):  
Data Quality ◽  
Best Practice ◽  
Synthetic Data ◽  
Birth Date ◽  
Healthcare Organizations ◽  
Healthcare Data ◽  
Quality Issue ◽  
Assessment Approach ◽  
Quality Issues ◽  
The Impact

Objective The objective of this study was to demonstrate the utility of a healthcare data quality framework by using it to measure the impact of synthetic data quality issues on the validity of an eMeasure (CMS178—urinary catheter removal after surgery). Methods Data quality issues were artificially created by systematically degrading the underlying quality of EHR data using two methods: independent and correlated degradation. A linear model that describes the change in the events included in the eMeasure quantifies the impact of each data quality issue. Results Catheter duration had the most impact on the CMS178 eMeasure with every 1% reduction in data quality causing a 1.21% increase in the number of missing events. For birth date and admission type, every 1% reduction in data quality resulted in a 1% increase in missing events. Conclusion This research demonstrated that the impact of data quality issues can be quantified using a generalized process and that the CMS178 eMeasure, as currently defined, may not measure how well an organization is meeting the intended best practice goal. Secondary use of EHR data is warranted only if the data are of sufficient quality. The assessment approach described in this study demonstrates how the impact of data quality issues on an eMeasure can be quantified and the approach can be generalized for other data analysis tasks. Healthcare organizations can prioritize data quality improvement efforts to focus on the areas that will have the most impact on validity and assess whether the values that are reported should be trusted.

scholarly journals A Systematic Framework for Analyzing Observation Data in Patient-Centered Registries: Case Study for Patients With Depression (Preprint)

2020 ◽  
Author(s):  
Maryam Zolnoori ◽  
Mark D Williams ◽  
William B Leasure ◽  
Kurt B Angstman ◽  
Che Ngufor
Keyword(s):  
Data Quality ◽  
Quality Data ◽  
Patient Centered ◽  
High Quality ◽  
Data Set ◽  
High Quality Data ◽  
Research Questions ◽  
Quality Issues ◽  
The Impact ◽  
Systematic Framework

BACKGROUND Patient-centered registries are essential in population-based clinical care for patient identification and monitoring of outcomes. Although registry data may be used in real time for patient care, the same data may further be used for secondary analysis to assess disease burden, evaluation of disease management and health care services, and research. The design of a registry has major implications for the ability to effectively use these clinical data in research. OBJECTIVE This study aims to develop a systematic framework to address the data and methodological issues involved in analyzing data in clinically designed patient-centered registries. METHODS The systematic framework was composed of 3 major components: visualizing the multifaceted and heterogeneous patient-centered registries using a data flow diagram, assessing and managing data quality issues, and identifying patient cohorts for addressing specific research questions. RESULTS Using a clinical registry designed as a part of a collaborative care program for adults with depression at Mayo Clinic, we were able to demonstrate the impact of the proposed framework on data integrity. By following the data cleaning and refining procedures of the framework, we were able to generate high-quality data that were available for research questions about the coordination and management of depression in a primary care setting. We describe the steps involved in converting clinically collected data into a viable research data set using registry cohorts of depressed adults to assess the impact on high-cost service use. CONCLUSIONS The systematic framework discussed in this study sheds light on the existing inconsistency and data quality issues in patient-centered registries. This study provided a step-by-step procedure for addressing these challenges and for generating high-quality data for both quality improvement and research that may enhance care and outcomes for patients. INTERNATIONAL REGISTERED REPORT DERR1-10.2196/18366

scholarly journals A Systematic Framework for Analyzing Observation Data in Patient-Centered Registries: Case Study for Patients With Depression

10.2196/18366 ◽  
2020 ◽  
Vol 9 (10) ◽  
pp. e18366
Author(s):  
Maryam Zolnoori ◽  
Mark D Williams ◽  
William B Leasure ◽  
Kurt B Angstman ◽  
Che Ngufor
Keyword(s):  
Data Quality ◽  
Quality Data ◽  
Patient Centered ◽  
High Quality ◽  
Data Set ◽  
High Quality Data ◽  
Research Questions ◽  
Quality Issues ◽  
The Impact ◽  
Systematic Framework

Background Patient-centered registries are essential in population-based clinical care for patient identification and monitoring of outcomes. Although registry data may be used in real time for patient care, the same data may further be used for secondary analysis to assess disease burden, evaluation of disease management and health care services, and research. The design of a registry has major implications for the ability to effectively use these clinical data in research. Objective This study aims to develop a systematic framework to address the data and methodological issues involved in analyzing data in clinically designed patient-centered registries. Methods The systematic framework was composed of 3 major components: visualizing the multifaceted and heterogeneous patient-centered registries using a data flow diagram, assessing and managing data quality issues, and identifying patient cohorts for addressing specific research questions. Results Using a clinical registry designed as a part of a collaborative care program for adults with depression at Mayo Clinic, we were able to demonstrate the impact of the proposed framework on data integrity. By following the data cleaning and refining procedures of the framework, we were able to generate high-quality data that were available for research questions about the coordination and management of depression in a primary care setting. We describe the steps involved in converting clinically collected data into a viable research data set using registry cohorts of depressed adults to assess the impact on high-cost service use. Conclusions The systematic framework discussed in this study sheds light on the existing inconsistency and data quality issues in patient-centered registries. This study provided a step-by-step procedure for addressing these challenges and for generating high-quality data for both quality improvement and research that may enhance care and outcomes for patients. International Registered Report Identifier (IRRID) DERR1-10.2196/18366

scholarly journals Quality Matters: Biocuration Experts on the Impact of Duplication and Other Data Quality Issues in Biological Databases

2019 ◽  
Author(s):  
Qingyu Chen ◽  
Ramona Britto ◽  
Ivan Erill ◽  
Constance J. Jeffery ◽  
Arthur Liberzon ◽  
...  
Keyword(s):  
Data Quality ◽  
Biological Database ◽  
Detection Methods ◽  
Correct Identification ◽  
Biological Databases ◽  
Biological Sequence ◽  
The Face ◽  
Quality Issues ◽  
Negative Impacts ◽  
The Impact

AbstractThe volume of biological database records is growing rapidly, populated by complex records drawn from heterogeneous sources. A specific challenge is duplication, that is, the presence of redundancy (records with high similarity) or inconsistency (dissimilar records that correspond to the same entity). The characteristics (which records are duplicates), impact (why duplicates are significant), and solutions (how to address duplication), are not well understood. Studies on the topic are neither recent nor comprehensive. In addition, other data quality issues, such as inconsistencies and inaccuracies, are also of concern in the context of biological databases. A primary focus of this paper is to present and consolidate the opinions of over 20 experts and practitioners on the topic of duplication in biological sequence databases. The results reveal that survey participants believe that duplicate records are diverse; that the negative impacts of duplicates are severe, while positive impacts depend on correct identification of duplicates; and that duplicate detection methods need to be more precise, scalable, and robust. A secondary focus is to consider other quality issues. We observe that biocuration is the key mechanism used to ensure the quality of this data, and explore the issues through a case study of curation in UniProtKB/Swiss-Prot as well as an interview with an experienced biocurator. While biocuration is a vital solution for handling of data quality issues, a broader community effort is needed to provide adequate support for thorough biocuration in the face of widespread quality concerns.

scholarly journals Data Quality Issues With Physician-Rating Websites: Systematic Review

10.2196/15916 ◽  
2020 ◽  
Vol 22 (9) ◽  
pp. e15916
Author(s):  
Pavankumar Mulgund ◽  
Raj Sharman ◽  
Priya Anand ◽  
Shashank Shekhar ◽  
Priya Karadi
Keyword(s):  
Data Quality ◽  
English Language ◽  
Quality Of Data ◽  
Contextual Data ◽  
Patient Reported ◽  
Physician Rating ◽  
Quality Framework ◽  
Quality Issues ◽  
The Impact

Background In recent years, online physician-rating websites have become prominent and exert considerable influence on patients’ decisions. However, the quality of these decisions depends on the quality of data that these systems collect. Thus, there is a need to examine the various data quality issues with physician-rating websites. Objective This study’s objective was to identify and categorize the data quality issues afflicting physician-rating websites by reviewing the literature on online patient-reported physician ratings and reviews. Methods We performed a systematic literature search in ACM Digital Library, EBSCO, Springer, PubMed, and Google Scholar. The search was limited to quantitative, qualitative, and mixed-method papers published in the English language from 2001 to 2020. Results A total of 423 articles were screened. From these, 49 papers describing 18 unique data quality issues afflicting physician-rating websites were included. Using a data quality framework, we classified these issues into the following four categories: intrinsic, contextual, representational, and accessible. Among the papers, 53% (26/49) reported intrinsic data quality errors, 61% (30/49) highlighted contextual data quality issues, 8% (4/49) discussed representational data quality issues, and 27% (13/49) emphasized accessibility data quality. More than half the papers discussed multiple categories of data quality issues. Conclusions The results from this review demonstrate the presence of a range of data quality issues. While intrinsic and contextual factors have been well-researched, accessibility and representational issues warrant more attention from researchers, as well as practitioners. In particular, representational factors, such as the impact of inline advertisements and the positioning of positive reviews on the first few pages, are usually deliberate and result from the business model of physician-rating websites. The impact of these factors on data quality has not been addressed adequately and requires further investigation.

scholarly journals Quality Matters: Biocuration Experts on the Impact of Duplication and Other Data Quality Issues in Biological Databases

2020 ◽  
Vol 18 (2) ◽  
pp. 91-103 ◽  
Author(s):  
Qingyu Chen ◽  
Ramona Britto ◽  
Ivan Erill ◽  
Constance J. Jeffery ◽  
Arthur Liberzon ◽  
...  
Keyword(s):  
Data Quality ◽  
Biological Databases ◽  
Quality Matters ◽  
Quality Issues ◽  
The Impact

An Investigation of the Impact of Organization Size on Data Quality Issues

2005 ◽  
Vol 16 (3) ◽  
pp. 58-71 ◽  
Author(s):  
G. Daryl Nord ◽  
Jeretta Horn Nord ◽  
Hongjiang Xu
Keyword(s):  
Data Quality ◽  
Organization Size ◽  
Quality Issues ◽  
The Impact

scholarly journals Beyond Data Validation – Advanced Strategies for Assessing Data Quality for Oil Spill Investigations

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Linda Cook ◽  
Laurie Benton ◽  
Melanie Edwards
Keyword(s):  
Data Quality ◽  
Oil Spill ◽  
Analytical Data ◽  
Data Validation ◽  
Quality Of Data ◽  
Validation Process ◽  
Chemistry Data ◽  
Quality Issues ◽  
The Impact ◽  
Analytical Approaches

ABSTRACT Field sampling investigations in response to oil spill incidents are growing increasingly more complex with analytical data collected by a variety of interested parties over many years and with different investigative purposes. For the Deepwater Horizon (DWH) Oil Spill, the analytical chemistry data and toxicity study data were required to be validated in accordance with U.S. Environmental Protection Agency's (EPA's) data validation for Superfund program methods. The process of validating data according to EPA guidelines is a manual and time-consuming process focused on chemistry results for individual samples within a single data package to assess if data meet quality control criteria. In hindsight, the burden of validating all of the chemistry data appears to be excessive, and for some parameters unnecessary, which was costly and slowed the process of disseminating data. Depending on the data use (e.g., assessing human and ecological risk, qualitative oil tracking, or forensic fingerprinting), data validation may not be needed in every circumstance or for every data type. Publicly available water column, sediment, and oil chemistry analytical data associated with the DWH Oil Spill, obtained from the Gulf of Mexico Research Initiative Information and Data Cooperative data portal were evaluated to understand the impact, effort, accuracy, and benefit of the data validation process. Questions explored include: What data changed based on data validation reviews?How would these changes affect the associated data evaluation findings?Did data validation introduce additional errors?What data quality issues did the data validation process miss?What statistical and data analytical approaches would more efficiently identify potential data quality issues? Based on our evaluation of the chemical data associated with the DWH Oil Spill, new strategies to assess the quality of data associated with oil spill investigations will be presented.

Sign in / Sign up

Export Citation Format

Share Document