A Context-based Crowd Sourcing Tool for Quality Assurance of SNOMED CT

2021 ◽  
Author(s):  
Kashifuddin Qazi ◽  
Ankur Agrawal
Keyword(s):  
Quality Assurance ◽  
Crowd Sourcing ◽  
Snomed Ct

scholarly journals Quality assurance and enrichment of biological and biomedical ontologies and terminologies

2020 ◽  
Vol 20 (S10) ◽  
Author(s):  
Ankur Agrawal ◽  
Licong Cui
Keyword(s):  
Quality Assurance ◽  
Cancer Registries ◽  
Supplement Issue ◽  
Biomedical Ontologies ◽  
Snomed Ct ◽  
Unified Medical Language System ◽  
Domain Specific ◽  
Healthcare Settings ◽  
Domain Specific Knowledge ◽  
Enrichment Techniques

AbstractBiological and biomedical ontologies and terminologies are used to organize and store various domain-specific knowledge to provide standardization of terminology usage and to improve interoperability. The growing number of such ontologies and terminologies and their increasing adoption in clinical, research and healthcare settings call for effective and efficient quality assurance and semantic enrichment techniques of these ontologies and terminologies. In this editorial, we provide an introductory summary of nine articles included in this supplement issue for quality assurance and enrichment of biological and biomedical ontologies and terminologies. The articles cover a range of standards including SNOMED CT, National Cancer Institute Thesaurus, Unified Medical Language System, North American Association of Central Cancer Registries and OBO Foundry Ontologies.

scholarly journals Quality Assurance of UMLS Semantic Type Assignments Using SNOMED CT Hierarchies

2016 ◽  
Vol 55 (02) ◽  
pp. 158-165 ◽  
Author(s):  
Y. Chen ◽  
Z. He ◽  
M. Halper ◽  
L. Chen ◽  
H. Gu
Keyword(s):  
Quality Assurance ◽  
Error Rate ◽  
Semantic Network ◽  
Snomed Ct ◽  
Semantic Type ◽  
Domain Experts ◽  
Unified Medical Language System ◽  
Type Assignment ◽  
Semantic Types ◽  
High Level

SummaryBackground: The Unified Medical Language System (UMLS) is one of the largest biomedical terminological systems, with over 2.5 million concepts in its Metathesaurus repository. The UMLS’s Semantic Network (SN) with its collection of 133 high-level semantic types serves as an abstraction layer on top of the Metathesaurus. In particular, the SN elaborates an aspect of the Metathesaurus’s concepts via the assignment of one or more types to each concept. Due to the scope and complexity of the Metathesaurus, errors are all but inevitable in this semantic-type assignment process.Objectives: To develop a semi-automated methodology to help assure the quality of semantic-type assignments within the UMLS.Methods: The methodology uses a cross- validation strategy involving SNOMED CT’s hierarchies in combination with UMLS se -mantic types. Semantically uniform, disjoint concept groups are generated programmatically by partitioning the collection of all concepts in the same SNOMED CT hierarchy according to their respective semantic-type assignments in the UMLS. Domain experts are then called upon to review the concepts in any group having a small number of concepts. It is our hypothesis that a semantic-type assignment combination applicable only to a very small number of concepts in a SNOMED CT hierarchy is an indicator of potential problems.Results: The methodology was applied to the UMLS 2013AA release along with the SNOMED CT from January 2013. An overall error rate of 33% was found for concepts proposed by the quality-assurance methodology. Supporting our hypothesis, that number was four times higher than the error rate found in control samples.Conclusion: The results show that the quality-assurance methodology can aid in effective and efficient identification of UMLS semantic-type assignment errors.

scholarly journals A tribal abstraction network for SNOMED CT target hierarchies without attribute relationships

2014 ◽  
Vol 22 (3) ◽  
pp. 628-639 ◽  
Author(s):  
Christopher Ochs ◽  
James Geller ◽  
Yehoshua Perl ◽  
Yan Chen ◽  
Ankur Agrawal ◽  
...  
Keyword(s):  
Quality Assurance ◽  
Description Logic ◽  
Snomed Ct ◽  
Hierarchical Relationships ◽  
Similar Description ◽  
Systematized Nomenclature Of Medicine

Abstract Objective Large and complex terminologies, such as Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT), are prone to errors and inconsistencies. Abstraction networks are compact summarizations of the content and structure of a terminology. Abstraction networks have been shown to support terminology quality assurance. In this paper, we introduce an abstraction network derivation methodology which can be applied to SNOMED CT target hierarchies whose classes are defined using only hierarchical relationships (ie, without attribute relationships) and similar description-logic-based terminologies. Methods We introduce the tribal abstraction network (TAN), based on the notion of a tribe—a subhierarchy rooted at a child of a hierarchy root, assuming only the existence of concepts with multiple parents. The TAN summarizes a hierarchy that does not have attribute relationships using sets of concepts, called tribal units that belong to exactly the same multiple tribes. Tribal units are further divided into refined tribal units which contain closely related concepts. A quality assurance methodology that utilizes TAN summarizations is introduced. Results A TAN is derived for the Observable entity hierarchy of SNOMED CT, summarizing its content. A TAN-based quality assurance review of the concepts of the hierarchy is performed, and erroneous concepts are shown to appear more frequently in large refined tribal units than in small refined tribal units. Furthermore, more erroneous concepts appear in large refined tribal units of more tribes than of fewer tribes. Conclusions In this paper we introduce the TAN for summarizing SNOMED CT target hierarchies. A TAN was derived for the Observable entity hierarchy of SNOMED CT. A quality assurance methodology utilizing the TAN was introduced and demonstrated.

SNOMED CT Implementation

2012 ◽  
Vol 51 (06) ◽  
pp. 529-538 ◽  
Author(s):  
K. Rosenbeck Gøeg ◽  
A. Randorff Højen
Keyword(s):  
Quality Assurance ◽  
Clinical Practice ◽  
Electronic Health Record ◽  
Quality Criteria ◽  
Clinical Information ◽  
Health Record ◽  
Snomed Ct ◽  
Depth Analysis ◽  
Systematized Nomenclature Of Medicine ◽  
Future Work

SummaryClinical practice as well as research and quality-assurance benefit from unambiguous clinical information resulting from the use of a common terminology like the Systematized Nomenclature of Medicine – Clinical Terms (SNOMED CT). A common terminology is a necessity to enable consistent reuse of data, and supporting semantic interoperability. Managing use of terminology for large cross specialty Electronic Health Record systems (EHR systems) or just beyond the level of single EHR systems requires that mappings are kept consistent. The objective of this study is to provide a clear methodology for SNOMED CT mapping to enhance applicability of SNOMED CT despite incompleteness and redundancy. Such mapping guidelines are presented based on an in depth analysis of 14 different EHR templates retrieved from five Danish and Swedish EHR systems. Each mapping is assessed against defined quality criteria and mapping guidelines are specified. Future work will include guideline validation.

scholarly journals Analysis of readability and structural accuracy in SNOMED CT

2020 ◽  
Vol 20 (S10) ◽  
Author(s):  
Francisco Abad-Navarro ◽  
Manuel Quesada-Martínez ◽  
Astrid Duque-Ramos ◽  
Jesualdo Tomás Fernández-Breis
Keyword(s):  
Gene Ontology ◽  
Quality Assurance ◽  
Best Practices ◽  
Natural Language ◽  
Biomedical Research ◽  
Snomed Ct ◽  
Quantitative Metrics ◽  
Label Content

Abstract Background The increasing adoption of ontologies in biomedical research and the growing number of ontologies available have made it necessary to assure the quality of these resources. Most of the well-established ontologies, such as the Gene Ontology or SNOMED CT, have their own quality assurance processes. These have demonstrated their usefulness for the maintenance of the resources but are unable to detect all of the modelling flaws in the ontologies. Consequently, the development of efficient and effective quality assurance methods is needed. Methods Here, we propose a series of quantitative metrics based on the processing of the lexical regularities existing in the content of the ontology, to analyse readability and structural accuracy. The readability metrics account for the ratio of labels, descriptions, and synonyms associated with the ontology entities. The structural accuracy metrics evaluate how two ontology modelling best practices are followed: (1) lexically suggest locally define (LSLD), that is, if what is expressed in natural language for humans is available as logical axioms for machines; and (2) systematic naming, which accounts for the amount of label content of the classes in a given taxonomy shared. Results We applied the metrics to different versions of SNOMED CT. Both readability and structural accuracy metrics remained stable in time but could capture some changes in the modelling decisions in SNOMED CT. The value of the LSLD metric increased from 0.27 to 0.31, and the value of the systematic naming metric was around 0.17. We analysed the readability and structural accuracy in the SNOMED CT July 2019 release. The results showed that the fulfilment of the structural accuracy criteria varied among the SNOMED CT hierarchies. The value of the metrics for the hierarchies was in the range of 0–0.92 (LSLD) and 0.08–1 (systematic naming). We also identified the cases that did not meet the best practices. Conclusions We generated useful information about the engineering of the ontology, making the following contributions: (1) a set of readability metrics, (2) the use of lexical regularities to define structural accuracy metrics, and (3) the generation of quality assurance information for SNOMED CT.

scholarly journals Scalable quality assurance for large SNOMED CT hierarchies using subject-based subtaxonomies

2015 ◽  
Vol 22 (3) ◽  
pp. 507-518 ◽  
Author(s):  
Christopher Ochs ◽  
James Geller ◽  
Yehoshua Perl ◽  
Yan Chen ◽  
Junchuan Xu ◽  
...  
Keyword(s):  
Quality Assurance ◽  
Clinical Finding ◽  
Control Sample ◽  
Snomed Ct ◽  
Domain Experts ◽  
Significant Challenge ◽  
Innovative Technique ◽  
Innovative Methodology ◽  
Large Hierarchy

Abstract Objective Standards terminologies may be large and complex, making their quality assurance challenging. Some terminology quality assurance (TQA) methodologies are based on abstraction networks (AbNs), compact terminology summaries. We have tested AbNs and the performance of related TQA methodologies on small terminology hierarchies. However, some standards terminologies, for example, SNOMED, are composed of very large hierarchies. Scaling AbN TQA techniques to such hierarchies poses a significant challenge. We present a scalable subject-based approach for AbN TQA. Methods An innovative technique is presented for scaling TQA by creating a new kind of subject-based AbN called a subtaxonomy for large hierarchies. New hypotheses about concentrations of erroneous concepts within the AbN are introduced to guide scalable TQA. Results We test the TQA methodology for a subject-based subtaxonomy for the Bleeding subhierarchy in SNOMED's large Clinical finding hierarchy. To test the error concentration hypotheses, three domain experts reviewed a sample of 300 concepts. A consensus-based evaluation identified 87 erroneous concepts. The subtaxonomy-based TQA methodology was shown to uncover statistically significantly more erroneous concepts when compared to a control sample. Discussion The scalability of TQA methodologies is a challenge for large standards systems like SNOMED. We demonstrated innovative subject-based TQA techniques by identifying groups of concepts with a higher likelihood of having errors within the subtaxonomy. Scalability is achieved by reviewing a large hierarchy by subject. Conclusions An innovative methodology for scaling the derivation of AbNs and a TQA methodology was shown to perform successfully for the largest hierarchy of SNOMED.

scholarly journals Mining non-lattice subgraphs for detecting missing hierarchical relations and concepts in SNOMED CT

2017 ◽  
Vol 24 (4) ◽  
pp. 788-798 ◽  
Author(s):  
Licong Cui ◽  
Wei Zhu ◽  
Shiqiang Tao ◽  
James T Case ◽  
Olivier Bodenreider ◽  
...  
Keyword(s):  
Quality Assurance ◽  
Random Sample ◽  
Visual Inspection ◽  
Snomed Ct ◽  
Structural Part ◽  
Domain Experts ◽  
Frequent Type ◽  
Hierarchical Relations ◽  
Objective Quality

Abstract Objective: Quality assurance of large ontological systems such as SNOMED CT is an indispensable part of the terminology management lifecycle. We introduce a hybrid structural-lexical method for scalable and systematic discovery of missing hierarchical relations and concepts in SNOMED CT. Material and Methods: All non-lattice subgraphs (the structural part) in SNOMED CT are exhaustively extracted using a scalable MapReduce algorithm. Four lexical patterns (the lexical part) are identified among the extracted non-lattice subgraphs. Non-lattice subgraphs exhibiting such lexical patterns are often indicative of missing hierarchical relations or concepts. Each lexical pattern is associated with a potential specific type of error. Results: Applying the structural-lexical method to SNOMED CT (September 2015 US edition), we found 6801 non-lattice subgraphs that matched these lexical patterns, of which 2046 were amenable to visual inspection. We evaluated a random sample of 100 small subgraphs, of which 59 were reviewed in detail by domain experts. All the subgraphs reviewed contained errors confirmed by the experts. The most frequent type of error was missing is-a relations due to incomplete or inconsistent modeling of the concepts. Conclusions: Our hybrid structural-lexical method is innovative and proved effective not only in detecting errors in SNOMED CT, but also in suggesting remediation for these errors.

Sign in / Sign up

Export Citation Format

Share Document