Efficient processing of complex XSD using Hive and Spark

The eXtensible Markup Language (XML) files are widely used by the industry due to their flexibility in representing numerous kinds of data. Multiple applications such as financial records, social networks, and mobile networks use complex XML schemas with nested types, contents, and/or extension bases on existing complex elements or large real-world files. A great number of these files are generated each day and this has influenced the development of Big Data tools for their parsing and reporting, such as Apache Hive and Apache Spark. For these reasons, multiple studies have proposed new techniques and evaluated the processing of XML files with Big Data systems. However, a more usual approach in such works involves the simplest XML schemas, even though, real data sets are composed of complex schemas. Therefore, to shed light on complex XML schema processing for real-life applications with Big Data tools, we present an approach that combines three techniques. This comprises three main methods for parsing XML files: cataloging, deserialization, and positional explode. For cataloging, the elements of the XML schema are mapped into root, arrays, structures, values, and attributes. Based on these elements, the deserialization and positional explode are straightforwardly implemented. To demonstrate the validity of our proposal, we develop a case study by implementing a test environment to illustrate the methods using real data sets provided from performance management of two mobile network vendors. Our main results state the validity of the proposed method for different versions of Apache Hive and Apache Spark, obtain the query execution times for Apache Hive internal and external tables and Apache Spark data frames, and compare the query performance in Apache Hive with that of Apache Spark. Another contribution made is a case study in which a novel solution is proposed for data analysis in the performance management systems of mobile networks.

Crossref 4.4.2 XML Elements and Attributes

For anyone trying to understand both the basics and the full range of options available when making a DOI metadata submission to Crossref, this linked table of XML element and attribute descriptions gives one small publisher’s best understanding of the most recent version of Crossref’s metadata submission elements and attributes. As of April 2021, the most recent version of Crossref XML files is 4.4.2. This table provides definitions for the six Crossref XML Schema Definition (xsd) files that include the most commonly used description elements of a DOI submission: crossref4.4.2.xsd, common4.4.2.xsd, fundref.xsd, AccessIndicators.xsd, clinicaltrials.xsd, and relations.xsd. The table also includes a brief description of the main features of the externally defined jats:abstract (JATS) element. This table focuses not on XML syntax but on the intent and structure of the elements from a small publisher perspective. This table is one small publisher’s interpretation of Crossref XML and is not authoritative in any way. It will inevitably contain errors, and the author takes no responsibility for its use, which is necessarily and entirely at your own risk. Any submissions created with information from this table should be verified for correctness against the official automated documentation and tools at the Crossref submission site. Note, however, that occasional errors and inconsistencies in those Crossref XML files were uncovered during the creation of this table. Every effort has been made here both to document inconsistencies in the original files and in this interpretation of those files. Important links to Crossref documentation, including comment on the apparent status of Crossref web pages, are provided in the References section after the table on the last document page.

Methods for Formalizing Cognitive Graphics and Visual Models using XML Schemas

The paper analyses methods of formalising cognitive graphics and visual models using promising data storage formats. We describe the primary visual design techniques and note that they appear to be rather disconnected. We show that ensuring the coupling of data and knowledge in visual models featuring various levels of detail is the main problem in integrated usage of visual modelling tools. We analyse approaches to solving the semantic discontinuity problem, that is, provided we meet the condition under which the properties of objects, systems and processes under consideration are only input once, it is necessary to ensure that data from models corresponding to different levels of abstraction (expertise) is interconnected. One should assume that the main drawback of existing approaches to visualising complex systems is that these approaches are fragmented and isolated, which means that they will only be effective locally. The paper proposes several approaches to formalising visual models employing XML schemas, which ensures that development processes concerning visual models of various levels of abstraction are synchronised and interconnected. We use a BPMN (Business, Process, Model and Notation) visual model as an example that shows the principles of representing visual model elements by means of XML schemas. The paper provides a detailed analysis of the principles behind layer interaction in the BPMN model through flexible XML description. We show that the BPMN data structure boasts its own XML schema containing all the parameters of a class or an element. The paper presents several examples and a technique of applying an XML schema to a BPMN model, including a further generalisation aimed at formally representing the process models of complex systems

Querying multi-source heterogeneous fuzzy spatiotemporal data

With the rapid development of the environmental, meteorological and marine data management, fuzzy spatiotemporal data has received considerable attention. Even though some achievements in querying aspect have been made, there are still some unsolved problems. Semantic and structural heterogeneity may exist among different data sources, which will lead to incomplete results. In addition, there are ambiguous query intentions and conditions when the user queries the data. This paper proposes a fuzzy spatiotemporal data semantic model. Based on this model, the RDF local semantic models are converted into a RDF global semantic model after mapping relational data and XML data to RDF local semantic models. The existing methods mainly convert relational data to RDF Schema directly. But our approach converts relational data to XML Schema and then converts it to RDF, which utilizes the semi-structured feature of XML schema to solve the structural heterogeneity between different data sources. The integration process enables us to perform global queries against different data sources. In the proposed query algorithms, the query conditions inputted are converted into exact queries before the results are returned. Finally, this paper has carried out extensive experiments, calculated the recall, precision and F-Score of the experimental results, and compared with other state-of-the-art query methods. It shows the importance of the data integration method and the effectiveness of the query method proposed in this paper.

Evaluation of XML Schema Support in Knowledge Management

Well organized data contributes extensively to the classification possibilities and quality of Knowledge Management. XML schemas play an important role in data organization activities, and provide basic foundations for companies and organizations dealing with large amounts of data. In times where knowledge represents the greatest advantage in a competitive economy and is relatively simple to find through different web providers, the quality of internal data structures and efficient management of a company’s valuable information is of the utmost importance. XML schemas are one of the mechanisms that can provide a data organization system in a qualitative manner, and efficient knowledge management as soon as data have been defined or accumulated. A good XML schema support is a way to increase the competitiveness of an organization by ensuring structured data quality and simplifying the Knowledge Management process.