Optimizing One-time and Continuous Subgraph Queries using Worst-case Optimal Joins

We study the problem of optimizing one-time and continuous subgraph queries using the new worst-case optimal join plans. Worst-case optimal plans evaluate queries by matching one query vertex at a time using multiway intersections. The core problem in optimizing worst-case optimal plans is to pick an ordering of the query vertices to match. We make two main contributions: 1. A cost-based dynamic programming optimizer for one-time queries that (i) picks efficient query vertex orderings for worst-case optimal plans and (ii) generates hybrid plans that mix traditional binary joins with worst-case optimal style multiway intersections. In addition to our optimizer, we describe an adaptive technique that changes the query vertex orderings of the worst-case optimal subplans during query execution for more efficient query evaluation. The plan space of our one-time optimizer contains plans that are not in the plan spaces based on tree decompositions from prior work. 2. A cost-based greedy optimizer for continuous queries that builds on the delta subgraph query framework. Given a set of continuous queries, our optimizer decomposes these queries into multiple delta subgraph queries, picks a plan for each delta query, and generates a single combined plan that evaluates all of the queries. Our combined plans share computations across operators of the plans for the delta queries if the operators perform the same intersections. To increase the amount of computation shared, we describe an additional optimization that shares partial intersections across operators. Our optimizers use a new cost metric for worst-case optimal plans called intersection-cost . When generating hybrid plans, our dynamic programming optimizer for one-time queries combines intersection-cost with the cost of binary joins. We demonstrate the effectiveness of our plans, adaptive technique, and partial intersection sharing optimization through extensive experiments. Our optimizers are integrated into GraphflowDB.

Query Rewriting for Incremental Continuous Query Evaluation in HIFUN

HIFUN is a high-level query language for expressing analytic queries of big datasets, offering a clear separation between the conceptual layer, where analytic queries are defined independently of the nature and location of data, and the physical layer, where queries are evaluated. In this paper, we present a methodology based on the HIFUN language, and the corresponding algorithms for the incremental evaluation of continuous queries. In essence, our approach is able to process the most recent data batch by exploiting already computed information, without requiring the evaluation of the query over the complete dataset. We present the generic algorithm which we translated to both SQL and MapReduce using SPARK; it implements various query rewriting methods. We demonstrate the effectiveness of our approach in temrs of query answering efficiency. Finally, we show that by exploiting the formal query rewriting methods of HIFUN, we can further reduce the computational cost, adding another layer of query optimization to our implementation.

Support for Adaptive and Distributed Deployment of CEP Continuous Queries for the IoMT

The current dissemination of IoT increases the deployment of stream processing solutions for monitoring and controlling elements of the real-world. One of those solutions is Complex Event Processing (CEP), and to handle the high volume, velocity and volatility of data streams from IoT sensors the CEP pipeline should be distributed, preferably having CEP operators both in the cloud/cluster and in edge devices. In this paper, we present a model for a distributed CEP platform and an implementation of this model called Global CEP Manager (GCM). GCM is a service of the ContextNet middleware that supports the deployment and dynamic rearrangement of CEP queries to CEP engines executing in the cloud and in M-Hubs, that are ContextNet’s mobile edge devices.

On-board Clutch Slippage Detection and Diagnosis in Heavy Duty Machine

In order to reduce unnecessary stops and expensive downtime originating from clutch failure of construction equipment machines; adequate real time sensor data measured on the machine in combination with feature extraction and classification methods may be utilized.This paper presents a framework with feature extraction methods and an anomaly detection module combined with Case-Based Reasoning (CBR) for on-board clutch slippage detection and diagnosis in heavy duty equipment. The feature extraction methods used are Moving Average Square Value Filtering (MASVF) and a measure of the fourth order statistical properties of the signals implemented as continuous queries over data streams. The anomaly detection module has two components, the Gaussian Mixture Model (GMM) and the Logistics Regression classifier. CBR is a learning approach that classifies faults by creating a new solution for a new fault case from the solution of the previous fault cases. Through use of a data stream management system and continuous queries (CQs), the anomaly detection module continuously waits for a clutch slippage event detected by the feature extraction methods, the query returns a set of features, which activates the anomaly detection module. The first component of the anomaly detection module trains a GMM to extracted features while the second component uses a Logistic Regression classifier for classifying normal and anomalous data. When an anomaly is detected, the Case-Based diagnosis module is activated for fault severity estimation.

Analysis of hybrid approach for Answering holistic queries in Wireless Sensor

Sensor networks are greatly utilized in unique areas like the transportation methods, wellbeing monitoring, soil monitoring, habitat monitoring and so forth. Clients pose queries to sensors and acquire sensing information. In view of the low quality detecting devices, sensor information is regularly boisterous. Continuous queries are more commonly employed to increase the reliability of query result. On this work we revolve round on regular holistic queries like Median. Present methodologies are for essentially the most phase supposed for non-holistic queries like Average. As a result of the no decomposable property of sensors, answering holistic queries are given much less value. We initially recommend two ways elegant on the information correlation be tween’s two rounds, with one for finding the detailed solutions and the other one for opting for the estimated outcome. We at that factor consolidate the two proposed plans into a hybrid methodology, which is flexible to the information evolving velocity. The effects show that our methodology has reduced the traffic rate even as keeping equal accuracy.

Hypothetical Answers to Continuous Queries over Data Streams

Continuous queries over data streams often delay answers until some relevant input arrives through the data stream. These delays may turn answers, when they arrive, obsolete to users who sometimes have to make decisions with no help whatsoever. Therefore, it can be useful to provide hypothetical answers – “given the current information, it is possible that X will become true at time t” – instead of no information at all. In this paper we present a semantics for queries and corresponding answers that covers such hypothetical answers, together with an online algorithm for updating the set of facts that are consistent with the currently available information.

Processing Data Streams

A data stream is a real-time continuous sequence that may be comprised of data or events. Data stream processing is different from static data processing which resides in a database. The data stream data is seen only once. It is too voluminous to store statically. A small portion of data called a window is considered at a time for querying, computing aggregates, etc. In this chapter, the authors explain the different types of window movement over incoming data. A query on a stream is repeatedly executed on the new data created by the movement of the window. SQL extensions to handle continuous queries is addressed in this chapter. Streams that contain transactional data as well as those that contain events are considered.