Real-time big data stream analytics and complex event detection

A Big Data Stream Computing (BDSC) Platform handles real-time data from various applications such as risk management, marketing management and business intelligence. Now a days Internet of Things (IoT) deployment is increasing massively in all the areas. These IoTs engender real-time data for analysis. Existing BDSC is inefficient to handle Real-data stream from IoTs because the data stream from IoTs is unstructured and has inconstant velocity. So, it is challenging to handle such real-time data stream. This work proposes a framework that handles real-time data stream through device control techniques to improve the performance. The frame work includes three layers. First layer deals with Big Data platforms that handles real data streams based on area of importance. Second layer is performance layer which deals with performance issues such as low response time, and energy efficiency. The third layer is meant for Applying developed method on existing BDSC platform. The experimental results have been shown a performance improvement 20%-30% for real time data stream from IoT application.

Download Full-text

Energy-Saving QoS Resource Management of Virtualized Networked Data Centers for Big Data Stream Computing

Big Data ◽

10.4018/978-1-4666-9840-6.ch040 ◽

2016 ◽

pp. 848-886

Author(s):

Nicola Cordeschi ◽

Mohammad Shojafar ◽

Danilo Amendola ◽

Enzo Baccarelli

Keyword(s):

Big Data ◽

Resource Management ◽

Real Time ◽

Data Stream ◽

Data Centers ◽

Software As A Service ◽

Stream Computing ◽

Management Optimization ◽

Hard Real Time ◽

Optimal Scheduler

In this chapter, the authors develop the scheduler which optimizes the energy-vs.-performance trade-off in Software-as-a-Service (SaaS) Virtualized Networked Data Centers (VNetDCs) that support real-time Big Data Stream Computing (BDSC) services. The objective is to minimize the communication-plus-computing energy which is wasted by processing streams of Big Data under hard real-time constrains on the per-job computing-plus-communication delays. In order to deal with the inherently nonconvex nature of the resulting resource management optimization problem, the authors develop a solving approach that leads to the lossless decomposition of the afforded problem into the cascade of two simpler sub-problems. The resulting optimal scheduler is amenable of scalable and distributed adaptive implementation. The performance of a Xen-based prototype of the scheduler is tested under several Big Data workload traces and compared with the corresponding ones of some state-of-the-art static and sequential schedulers.

Download Full-text

Conceptualization, Operationalization, and Validation of the Digital Data Stream Readiness Index

Journal of Global Information Management ◽

10.4018/jgim.2018100106 ◽

2018 ◽

Vol 26 (4) ◽

pp. 92-112 ◽

Cited By ~ 2

Author(s):

Elisabetta Raguseo ◽

Federico Pigni ◽

Gabriele Piccoli

Keyword(s):

Big Data ◽

Real Time ◽

Value Creation ◽

Data Stream ◽

Organizational Readiness ◽

Digital Data ◽

Measurement Instruments ◽

Scholarly Research ◽

Four Dimensions ◽

Significant Attention

This article describes how in their search for value creation, companies are investing considerable resources in so-called “Big Data” initiatives. A peculiar aspect of these initiatives is the increasing availability of real-time streams of data. Successfully leveraging these streams to extract value is emerging as a critical competence for the modern firm. Despite the significant attention received, scholarly research on Digital Data Stream (DDS) remains insufficient. More importantly, there are no specialized definitions and measurement instruments that can move the field forward by initiating a cumulative research tradition. This article can provide clarification on key definitions, differentiating DDS from Big Data. Drawing on the organizational readiness concept, the DDS readiness index develops as a measure of organizational readiness to exploit real-time digital data. This article will conceptualize, define, operationalize and validate the index. By identifying the four dimensions of mindset, skillset, dataset and toolset as the elements of the DDS readiness index and discussing its managerial and research implications

Download Full-text