Reasoning on the Efficiency of Distributed Complex Event Processing

Complex event processing (CEP) evaluates queries over streams of event data to detect situations of interest. If the event data are produced by geographically distributed sources, CEP may exploit in-network processing that distributes the evaluation of a query among the nodes of a network. To this end, a query is modularized and individual query operators are assigned to nodes, especially those that act as data sources. Existing solutions for such operator placement, however, are limited in that they assume all query results to be gathered at one designated node, commonly referred to as a sink. Hence, existing techniques postulate a hierarchical structure of the network that generates and processes the event data. This largely neglects the optimisation potential that stems from truly decentralised query evaluation with potentially many sinks. To address this gap, in this paper, we propose Multi-Sink Evaluation (MuSE) graphs as a formal computational model to evaluate common CEP queries in a decentralised manner. We further prove the completeness of query evaluation under this model. Striving for distributed CEP that can scale to large volumes of high-frequency event streams, we show how to reason on the network costs induced by distributed query evaluation and prune inefficient query execution plans. As such, our work lays the foundation for distributed CEP that is both, sound and efficient.

Download Full-text

Network-wide complex event processing over geographically distributed data sources

Information Systems ◽

10.1016/j.is.2019.101442 ◽

2020 ◽

Vol 88 ◽

pp. 101442

Author(s):

Ioannis Flouris ◽

Nikos Giatrakos ◽

Antonios Deligiannakis ◽

Minos Garofalakis

Keyword(s):

Complex Event Processing ◽

Data Sources ◽

Event Processing ◽

Distributed Data ◽

Geographically Distributed

Download Full-text

Event data warehousing for Complex Event Processing

2010 Fourth International Conference on Research Challenges in Information Science (RCIS) ◽

10.1109/rcis.2010.5507380 ◽

2010 ◽

Cited By ~ 13

Author(s):

Heinz Roth ◽

Josef Schiefer ◽

Hannes Obweger ◽

Szabolcs Rozsnyai

Keyword(s):

Data Warehousing ◽

Complex Event Processing ◽

Event Processing ◽

Event Data

Download Full-text

Application of Complex Event Processing Techniques to Big Data Related to Healthcare

Advances in Business Information Systems and Analytics - Enterprise Big Data Engineering, Analytics, and Management ◽

10.4018/978-1-5225-0293-7.ch012 ◽

2016 ◽

pp. 201-220

Author(s):

Fehmida Mohamedali ◽

Samia Oussena

Keyword(s):

Health Care ◽

Big Data ◽

Query Language ◽

Care Delivery ◽

Health Sector ◽

Healthcare Services ◽

Health Care Delivery System ◽

Complex Event Processing ◽

Data Sources ◽

Event Processing

Healthcare is a growth area for event processing applications. Computers and information systems have been used for collecting patient data in health care for over fifty years. However, progress towards a unified health care delivery system in the UK has been slow. Big Data, the Internet of Things (IoT) and Complex Event Processing (CEP) have the potential not only to deal with treatment areas of healthcare domain but also to redefine healthcare services. This study is intended to provide a broad overview of where in the health sector, the application of CEP is most used, the data sources that contribute to it and the types of event processing languages and techniques implemented. By systematic review of existing literature on the application of CEP techniques in Healthcare, a number of use cases have been identified to provide a detailed analysis of the most common used case(s), common data sources in use and highlight CEP query language types and techniques that have been considered.

Download Full-text

Model Driven Development Applied to Complex Event Processing for Near Real-Time Open Data

Sensors ◽

10.3390/s18124125 ◽

2018 ◽

Vol 18 (12) ◽

pp. 4125 ◽

Cited By ~ 4

Author(s):

Pedro Clemente ◽

Adolfo Lozano-Tello

Keyword(s):

Air Quality ◽

Real Time ◽

Open Data ◽

Complex Event Processing ◽

Data Sources ◽

Event Processing ◽

Model Driven Development ◽

Model Driven ◽

Concrete Syntax ◽

Use Of The Internet

Nowadays, data are being produced like never before because the use of the Internet of Things, social networks, and communication in general are increasing exponentially. Many of these data, especially those from public administrations, are freely offered using the open data concept where data are published to improve their reutilisation and transparency. Initially, the data involved information that is not updated continuously such as budgets, tourist information, office information, pharmacy information, etc. This kind of information does not change during large periods of time, such as days, weeks or months. However, when open data are produced near to real-time such as air quality sensors or people counters, suitable methodologies and tools are lacking to identify, consume, and analyse them. This work presents a methodology to tackle the analysis of open data sources using Model-Driven Development (MDD) and Complex Event Processing (CEP), which help users to raise the abstraction level utilised to manage and analyse open data sources. That means that users can manage heterogeneous and complex technology by using domain concepts defined by a model that could be used to generate specific code. Thus, this methodology is supported by a domain-specific language (DSL) called OpenData2CEP, which includes a metamodel, a graphical concrete syntax, and a model-to-text transformation to specific platforms, such as complex event processing engines. Finally, the methodology and the DSL have been applied to two near real-time contexts: the analysis of air quality for citizens’ proposals and the analysis of earthquake data.

Download Full-text

Streaming MASSIF: Cascading Reasoning for Efficient Processing of IoT Data Streams

Sensors ◽

10.3390/s18113832 ◽

2018 ◽

Vol 18 (11) ◽

pp. 3832 ◽

Cited By ~ 5

Author(s):

Pieter Bonte ◽

Riccardo Tommasini ◽

Emanuele Della Valle ◽

Filip De Turck ◽

Femke Ongenae

Keyword(s):

Data Streams ◽

High Velocity ◽

High Frequency ◽

Complex Event Processing ◽

Event Processing ◽

Multiple Sensors ◽

Time Dependencies ◽

Efficient Processing ◽

High Level ◽

The Internet Of Things

In the Internet of Things (IoT), multiple sensors and devices are generating heterogeneous streams of data. To perform meaningful analysis over multiple of these streams, stream processing needs to support expressive reasoning capabilities to infer implicit facts and temporal reasoning to capture temporal dependencies. However, current approaches cannot perform the required reasoning expressivity while detecting time dependencies over high frequency data streams. There is still a mismatch between the complexity of processing and the rate data is produced in volatile domains. Therefore, we introduce Streaming MASSIF, a Cascading Reasoning approach performing expressive reasoning and complex event processing over high velocity streams. Cascading Reasoning is a vision that solves the problem of expressive reasoning over high frequency streams by introducing a hierarchical approach consisting of multiple layers. Each layer minimizes the processed data and increases the complexity of the data processing. Cascading Reasoning is a vision that has not been fully realized. Streaming MASSIF is a layered approach allowing IoT service to subscribe to high-level and temporal dependent concepts in volatile data streams. We show that Streaming MASSIF is able to handle high velocity streams up to hundreds of events per second, in combination with expressive reasoning and complex event processing. Streaming MASSIF realizes the Cascading Reasoning vision and is able to combine high expressive reasoning with high throughput of processing. Furthermore, we formalize semantically how the different layers in our Cascading Reasoning Approach collaborate.

Download Full-text