Developing an Ontology Evaluation Methodology: Cognitive Measure of Quality

<p>Ontologies are formal specifications of shared conceptualizations of a domain. Important applications of ontologies include distributed knowledge based systems, such as the semantic web, and the evaluation of modelling languages, e.g. for business process or conceptual modelling. These applications require formal ontologies of good quality. In this thesis, we present a multi-method ontology evaluation methodology, which consists of two techniques (sentence verification task and recall) based on principles of cognitive psychology, to test how well a specification of a formal ontology corresponds to the ontology users' conceptualization of a domain. Two experiments were conducted, each evaluating the SUMO ontology and WordNet with an experimental technique, as demonstrations of the multi-method evaluation methodology. We also tested the applicability of the two evaluation techniques by conducting a replication study for each. The replication studies obtained findings that point towards the same direction as the original studies, although no significance was achieved. Overall, the evaluation using the multi-method methodology suggests that neither of the two ontologies we examined is a good specification of the conceptualization of the domain. Both the terminology and the structure of the ontologies, may benefit from improvement.</p>

Developing an Ontology Evaluation Methodology: Cognitive Measure of Quality

<p>Ontologies are formal specifications of shared conceptualizations of a domain. Important applications of ontologies include distributed knowledge based systems, such as the semantic web, and the evaluation of modelling languages, e.g. for business process or conceptual modelling. These applications require formal ontologies of good quality. In this thesis, we present a multi-method ontology evaluation methodology, which consists of two techniques (sentence verification task and recall) based on principles of cognitive psychology, to test how well a specification of a formal ontology corresponds to the ontology users' conceptualization of a domain. Two experiments were conducted, each evaluating the SUMO ontology and WordNet with an experimental technique, as demonstrations of the multi-method evaluation methodology. We also tested the applicability of the two evaluation techniques by conducting a replication study for each. The replication studies obtained findings that point towards the same direction as the original studies, although no significance was achieved. Overall, the evaluation using the multi-method methodology suggests that neither of the two ontologies we examined is a good specification of the conceptualization of the domain. Both the terminology and the structure of the ontologies, may benefit from improvement.</p>

A Survey of Ontologies and Their Applications in e-Learning Environments

Ontology technology has been investigated in a wide range of areas and is currently being utilized in many fields. In the e-learning context, many studies have used ontology to address problems such as the interoperability in learning objects, modeling and enriching learning resources, and personalizing educational content recommendations. We systematically reviewed research on ontology for e-learning from 2008 to 2020. The review was guided by 3 research questions: “How is ontology used for knowledge modeling in the context of e-learning?”, “What are the design principles, building methods, scale, level of semantic richness, and evaluation of current educational ontologies?”, and “What are the various ontology-based applications for e-learning?” We classified current educational ontologies into 6 types and analyzed them by 5 measures: design methodology, building routine, scale of ontology, level of semantic richness, and ontology evaluation. Furthermore, we reviewed 4 types of ontology-based e-learning applications and systems. The observations obtained from this survey can benefit researchers in this area and help to guide future research.

Social engineering in cybersecurity: a domain ontology and knowledge graph application examples

Social engineering has posed a serious threat to cyberspace security. To protect against social engineering attacks, a fundamental work is to know what constitutes social engineering. This paper first develops a domain ontology of social engineering in cybersecurity and conducts ontology evaluation by its knowledge graph application. The domain ontology defines 11 concepts of core entities that significantly constitute or affect social engineering domain, together with 22 kinds of relations describing how these entities related to each other. It provides a formal and explicit knowledge schema to understand, analyze, reuse and share domain knowledge of social engineering. Furthermore, this paper builds a knowledge graph based on 15 social engineering attack incidents and scenarios. 7 knowledge graph application examples (in 6 analysis patterns) demonstrate that the ontology together with knowledge graph is useful to 1) understand and analyze social engineering attack scenario and incident, 2) find the top ranked social engineering threat elements (e.g. the most exploited human vulnerabilities and most used attack mediums), 3) find potential social engineering threats to victims, 4) find potential targets for social engineering attackers, 5) find potential attack paths from specific attacker to specific target, and 6) analyze the same origin attacks.

A Framework for Building a Multilingual Industrial Ontology: Methodology and a Case Study for Building Smartphone English-Arabic Ontology

As Web 3.0 is blooming, ontologies augment semantic Web with semi–structured knowledge. Industrial ontologies can help in improving online commercial communication and marketing. In addition, conceptualizing the enterprise knowledge can improve information retrieval for industrial applications. Having ontologies combine multiple languages can help in delivering the knowledge to a broad sector of Internet users. In addition, multi-lingual ontologies can also help in commercial transactions. This research paper provides a framework model for building industrial multilingual ontologies which include Corpus Determination, Filtering, Analysis, Ontology Building, and Ontology Evaluation. It also addresses factors to be considered when modeling multilingual ontologies. A case study for building a bilingual English-Arabic ontology for smart phones is presented. The ontology was illustrated using an ontology editor and visualization tool. The built ontology consists of 67 classes and 18 instances presented in both Arabic and English. In addition, applications for using the ontology are presented. Future research directions for the built industrial ontology are presented.

Delta: A Modular Ontology Evaluation System

Ontologies are widely used nowadays. However, the plethora of ontologies currently available online, makes it really difficult to identify which ontologies are appropriate for a given task and to decide on their quality characteristics. This is further complicated by the fact that multiple quality criteria have been proposed for ontologies, making it even more difficult to decide which ontology to adopt. In this context, in this paper we present Delta, a modular online tool for analyzing and evaluating ontologies. The interested user can upload an ontology to the tool, which then automatically analyzes it and graphically visualizes numerous statistics, metrics, and pitfalls. Those visuals presented include a diverse set of quality dimensions, further guiding users to understand the benefits and the drawbacks of each individual ontology and how to properly develop and extend it.

Thai Tattoo Wisdom’s Representation of Knowledge by Ontology

Sak Yan Ontology (SYO) models knowledge derived from Thai tattoos in the design of cultural heritage preservation planning. Ontology Development 101 is a technique of ontology model creation. The aims of this study are to share the performance of ontology development and ontology evaluation. The study is specifically focused on validation from domain experts and automation evaluated using the OOPS! tools (OntOlogy Pitfall Scanner is a tool that helps detect some of the most common pitfalls appearing when developing ontologies). The results obtained from OOPS! show that SYO is devoid of critical errors; however, it does have one critical, three important, and three minor problems. Four of the problems are fixed, whereas the others are continuous. The combination of automatic and human validation methodologies improves the quality of the ontology being modeled. The tools enhance the traditional methodology with quicker, easier, and smaller amounts of subjective analysis. In conclusion, for the reparation movement, solutions for the above problems are suggested.

Ontology Development for Motorcycle Semantic Knowledge Representation

Transportation is currently a basic necessity in supporting daily life, starting from supporting economic activities and various other things. One of the most commonly used means of transportation is a motorcycle because it is very practical to use. However, the development of motorcycles is currently very fast, confusing the community in choosing a motorcycle that suits their needs. The solution used to overcome this problem can be overcome by using the concept of semantic ontology. The method of building the ontology model used is METHONTOLOGY. This method is one of the methods of building an ontology model that can reuse the built ontology for further system development. The motorcycle ontology development model generates 16 classes, 13 object properties, 11 data properties, and 69 individuals. The ontology evaluation process by performing SPARQL queries also provides appropriate results.

Ontology Fixing by Using Software Engineering Technology

This paper presents OntologyFixer, a web-based tool that supports a methodology to build, assess, and improve the quality of ontology web language (OWL) ontologies. Using our software, knowledge engineers are able to fix low-quality OWL ontologies (such as those created from natural language documents using ontology learning processes). The fixing process is guided by a set of metrics and fixing mechanisms provided by the tool, and executed primarily through automated changes (inspired by quick fix actions used in the software engineering domain). To evaluate the quality, the tool supports numerical and graphical quality assessments, focusing on ontology content and structure attributes. This tool follows principles, and provides features, typical of scientific software, including user parameter requests, logging, multithreading execution, and experiment repeatability, among others. OntologyFixer architecture takes advantage of model view controller (MVC), strategy, template, and factory design patterns; and decouples graphical user interfaces (GUI) from ontology quality metrics, ontology fixing, and REST (REpresentational State Transfer) API (Application Programming Interface) components (used for pitfall identification, and ontology evaluation). We also separate part of the OntologyFixer functionality into a new package called OntoMetrics, which focuses on the identification of symptoms and the evaluation of the quality of ontologies. Finally, OntologyFixer provides mechanisms to easily develop and integrate new quick fix methods.