Robust Cardinality: a novel approach for cardinality prediction in SQL queries

Background Database Management Systems (DBMSs) use declarative language to execute queries to stored data. The DBMS defines how data will be processed and ultimately retrieved. Therefore, it must choose the best option from the different possibilities based on an estimation process. The optimization process uses estimated cardinalities to make optimization decisions, such as choosing predicate order. Methods In this paper, we propose Robust Cardinality, an approach to calculate cardinality estimates of query operations to guide the execution engine of the DBMSs to choose the best possible form or at least avoid the worst one. By using machine learning, instead of the current histogram heuristics, it is possible to improve these estimates; hence, leading to more efficient query execution. Results We perform experimental tests using PostgreSQL, comparing both estimators and a modern technique proposed in the literature. With Robust Cardinality, a lower estimation error of a batch of queries was obtained and PostgreSQL executed these queries more efficiently than when using the default estimator. We observed a 3% reduction in execution time after reducing 4 times the query estimation error. Conclusions From the results, it is possible to conclude that this new approach results in improvements in query processing in DBMSs, especially in the generation of cardinality estimates.

Knowledge Graph OLAP

A knowledge graph (KG) represents real-world entities and their relationships. The represented knowledge is often context-dependent, leading to the construction of contextualized KGs. The multidimensional and hierarchical nature of context invites comparison with the OLAP cube model from multidimensional data analysis. Traditional systems for online analytical processing (OLAP) employ multidimensional models to represent numeric values for further analysis using dedicated query operations. In this paper, along with an adaptation of the OLAP cube model for KGs, we introduce an adaptation of the traditional OLAP query operations for the purposes of performing analysis over KGs. In particular, we decompose the roll-up operation from traditional OLAP into a merge and an abstraction operation. The merge operation corresponds to the selection of knowledge from different contexts whereas abstraction replaces entities with more general entities. The result of such a query is a more abstract, high-level view – a management summary – of the knowledge.

Efficient Boolean Keywords Search over Encrypted Cloud Data in Public Key Setting

Searchable public key encryption- (SPE-) supporting keyword search plays an important role in cloud computing for data confidentiality. The current SPE scheme mainly supports conjunctive or disjunctive keywords search which belongs to very basic query operations. In this paper, we propose an efficient and secure SPE scheme that supports Boolean keywords search, which is more advanced than the conjunctive and disjunctive keywords search. We first develop a keyword conversion method, which can change the index and Boolean keywords query into a group of vectors. Then, through applying a technique so-called dual pairing vector space to encrypt the obtained vectors, we propose a concrete scheme proven to be secure under chosen keyword attack. Finally, we put forward a detailed theoretical and experimental analysis to demonstrate the efficiency of our scheme.

Modeling simulation and virtual platform construction of crossbow making technology

In this paper, a set of virtual simulation platform for crossbow making is established. First, build a virtual model of Lisu crossbow, optimize the model, accelerate the organization, model generation, modification and editing of the database; then complete the database establishment of the corresponding model, nest the comment entity class and filemeta entity class into the file entity class, store them in a nested data set, and complete the data storage, management and query operations; finally, combined with Kalman filtering, arithmetic average filtering and KNN algorithm are used to build the quest 3D virtual platform to improve the 3D interaction effect with data gloves and complete the construction of the 3D platform.

The Algebraic Operations and Their Implementation Based on a Two-Layer Cloud Data Model

The existing cloud data models cannot meet the management requirements of structured data very well including a great deal of relational data, therefore a two-layer cloud data model is proposed. The composite object is defined to model the nested data in the representation layer, while a 4-tuple is defined to model the non-nested data in the storage layer. Referring the relational algebra, the concept of SNO (Simple Nested Object) is defined as basic operational unit of the algebraic operations; the formal definitions of the algebraic operations consisting of the set operations and the query operations on the representation layer are proposed. The algorithm of extracting all SNOs from a CAO (Component-Attribute-Object) set of a composite object is proposed firstly as the foundation, and then as the idea; the pseudo code implementation of algorithms of the algebraic operations on the storage layer are proposed. Logic proof and example proof indicate that the definition and the algorithms of the algebraic operations are correct.

A Graph Extension of the Positional Burrows-Wheeler Transform and its Applications

1AbstractWe present a generalization of the Positional Burrows-Wheeler Transform (PBWT) to genome graphs, which we call the gPBWT. A genome graph is a collapsed representation of a set of genomes described as a graph. In a genome graph, a haplotype corresponds to a restricted form of walk. The gPBWT is a compressible representation of a set of these graph-encoded haplotypes that allows for efficient subhaplotype match queries. We give efficient algorithms for gPBWT construction and query operations. We describe our implementation, showing the compression and search of 1000 Genomes data. As a demonstration, we use the gPBWT to quickly count the number of haplotypes consistent with random walks in a genome graph, and with the paths taken by mapped reads; results suggest that haplotype consistency information can be practically incorporated into graph-based read mappers.

The Design of Query Processing in Data Stream Management System

data stream management system is used to manage and query coming large, continuous, fast and flexible data stream. The system is based on the flow of data extraction, transformation, combination, which is the main content and task query execution. This paper mainly discusses the design and implementation of query execution module and query execution is composed of two parts which include query operations, query execution and scheduling.

WD-PWS: THE FIRST SEMANTICS FOR QUERYING OVER PROBABILISTIC DATA STREAMS WITH CONTINUOUS DISTRIBUTIONS

Many emerging applications need continuous querying over uncertain event streams, mostly for online monitoring. These streaming uncertain events may come from radars, sensors, or even software hooks. The uncertainty is usually due to measurement errors, inherent ambiguities and privacy preserving reasons. To cover new requirements, we have designed and implemented a new system called Probabilistic Data Stream Management System (PDSMS) in Ref. 1. PDSMS is a data processing engine which runs continuous queries over probabilistic streams. However, lack of a semantics for probabilistic databases which supports continuous distributions prevented us from having a strong foundation for our query operators. It also precludes us from proving consistency and correctness of query operations especially after optimization and adaption. In fact, in the probabilistic database literature, there is no semantics available which covers continuous distributions. This limitation is very restrictive as in real-world, uncertainty is usually modeled by continuous distributions. In this paper, after presenting a basic probabilistic data model for PDSMS, we focus on querying and formally present the first semantics for probabilistic query operations which supports continuous distributions as well as discrete ones. Using this new semantics, we define our query operators (e.g. select, project, and join) formally without ambiguity and compatible with operators in relational algebra. Thus, we can leverage many transformation rules in relational algebra as well. This new semantics allows us to have different strictness levels and consistency between operators. We also proved many strictness theorems about different alternatives for query operators.

A Horizontal Segmentation Based Ontology Parallel Query Approach

The performance of ontology query has become one of the bottlenecks of the large-scale bulk applications. Firstly OWL ontology files stored into the database in the form of triple table using Oracle 11g semantic technology. And then we designed and implemented the ontology parting method based on horizontal segmentation. Thirdly, several typical ontology query operations were achieved based on the multi-threading technology. Experimental results show that the parallel query methods described herein significantly improve query performance.

Tool Assisted Analysis of Open Source Projects

Open Source Software (OSS) is currently a widely adopted approach to developing and distributing software. OSS code adoption requires an understanding of the structure of the code base. For a deeper understanding of the maintenance, bug fixing and development activities, the structure of the developer community also needs to be understood, especially the relations between the code and community structures. This, in turn, is essential for the development and maintenance of software containing OSS code. This paper proposes a method and support tool for exploring the relations of the code base and community structures of OSS projects. The method and proposed tool, Binoculars, rely on generic and reusable query operations, formal definitions of which are given in the paper. The authors demonstrate the applicability of Binoculars with two examples. The authors analyze a well-known and active open source project, FFMpeg, and the open source version of the IaaS cloud computing project Eucalyptus.