A Machine Learning-Based Elastic Strategy for Operator Parallelism in a Big Data Stream Computing System

Author(s):  
Wei Li ◽  
Dawei Sun ◽  
Shang Gao ◽  
Rajkumar Buyya
2015 ◽  
Vol 319 ◽  
pp. 92-112 ◽  
Author(s):  
Dawei Sun ◽  
Guangyan Zhang ◽  
Songlin Yang ◽  
Weimin Zheng ◽  
Samee U. Khan ◽  
...  

Author(s):  
Rizwan Patan ◽  
Rajasekhara Babu M ◽  
Suresh Kallam

A Big Data Stream Computing (BDSC) Platform handles real-time data from various applications such as risk management, marketing management and business intelligence. Now a days Internet of Things (IoT) deployment is increasing massively in all the areas. These IoTs engender real-time data for analysis. Existing BDSC is inefficient to handle Real-data stream from IoTs because the data stream from IoTs is unstructured and has inconstant velocity. So, it is challenging to handle such real-time data stream. This work proposes a framework that handles real-time data stream through device control techniques to improve the performance. The frame work includes three layers. First layer deals with Big Data platforms that handles real data streams based on area of importance. Second layer is performance layer which deals with performance issues such as low response time, and energy efficiency. The third layer is meant for Applying developed method on existing BDSC platform. The experimental results have been shown a performance improvement 20%-30% for real time data stream from IoT application.


Big Data ◽  
2016 ◽  
pp. 848-886
Author(s):  
Nicola Cordeschi ◽  
Mohammad Shojafar ◽  
Danilo Amendola ◽  
Enzo Baccarelli

In this chapter, the authors develop the scheduler which optimizes the energy-vs.-performance trade-off in Software-as-a-Service (SaaS) Virtualized Networked Data Centers (VNetDCs) that support real-time Big Data Stream Computing (BDSC) services. The objective is to minimize the communication-plus-computing energy which is wasted by processing streams of Big Data under hard real-time constrains on the per-job computing-plus-communication delays. In order to deal with the inherently nonconvex nature of the resulting resource management optimization problem, the authors develop a solving approach that leads to the lossless decomposition of the afforded problem into the cascade of two simpler sub-problems. The resulting optimal scheduler is amenable of scalable and distributed adaptive implementation. The performance of a Xen-based prototype of the scheduler is tested under several Big Data workload traces and compared with the corresponding ones of some state-of-the-art static and sequential schedulers.


2017 ◽  
Vol 89 ◽  
pp. 4-23 ◽  
Author(s):  
Dawei Sun ◽  
Guangyan Zhang ◽  
Chengwen Wu ◽  
Keqin Li ◽  
Weimin Zheng

2016 ◽  
Vol 78 (10) ◽  
Author(s):  
Rizwan Patan ◽  
Rajasekhara Babu M.

It is necessary to model an energy efficient and stream optimization towards achieve high energy efficiency for Streaming data without degrading response time in big data stream computing. This paper proposes an Energy Efficient Traffic aware resource scheduling and Re-Streaming Stream Structure to replace a default scheduling strategy of storm is entitled as re-storm. The model described in three parts; First, a mathematical relation among energy consumption, low response time and high traffic streams. Second, various approaches provided for reducing an energy without affecting response time and which provides high performance in overall stream computing in big data. Third, re-storm deployed energy efficient traffic aware scheduling on the storm platform. It allocates worker nodes online by using hot-swapping technique with task utilizing by energy consolidation through graph partitioning. Moreover, re-storm is achieved high energy efficiency, low response time in all types of data arriving speeds.it is suitable for allocation of worker nodes in a storm topology. Experiment results have been demonstrated the comparing existing strategies which are dealing with energy issues without affecting or reducing response time for a different data stream speed levels. Finally, it shows that the re-storm platform achieved high energy efficiency and low response time when compared to all existing approaches.


Sign in / Sign up

Export Citation Format

Share Document