Performance Improvement IoT Applications Through Multimedia Analytics Using Big Data Stream Computing Platforms

2018 ◽  
pp. 200-221
Author(s):  
Rizwan Patan ◽  
Rajasekhara Babu M ◽  
Suresh Kallam
Keyword(s):  
Big Data ◽  
Real Time ◽  
Performance Improvement ◽  
Data Stream ◽  
Real Data ◽  
Stream Computing ◽  
Time Data ◽  
Real Time Data ◽  
Computing Platforms ◽  
Time And Energy

A Big Data Stream Computing (BDSC) Platform handles real-time data from various applications such as risk management, marketing management and business intelligence. Now a days Internet of Things (IoT) deployment is increasing massively in all the areas. These IoTs engender real-time data for analysis. Existing BDSC is inefficient to handle Real-data stream from IoTs because the data stream from IoTs is unstructured and has inconstant velocity. So, it is challenging to handle such real-time data stream. This work proposes a framework that handles real-time data stream through device control techniques to improve the performance. The frame work includes three layers. First layer deals with Big Data platforms that handles real data streams based on area of importance. Second layer is performance layer which deals with performance issues such as low response time, and energy efficiency. The third layer is meant for Applying developed method on existing BDSC platform. The experimental results have been shown a performance improvement 20%-30% for real time data stream from IoT application.

scholarly journals Re-Stream: Real-time and energy-efficient resource scheduling in big data stream computing environments

2015 ◽  
Vol 319 ◽  
pp. 92-112 ◽  
Author(s):  
Dawei Sun ◽  
Guangyan Zhang ◽  
Songlin Yang ◽  
Weimin Zheng ◽  
Samee U. Khan ◽  
...  
Keyword(s):  
Big Data ◽  
Real Time ◽  
Energy Efficient ◽  
Data Stream ◽  
Resource Scheduling ◽  
Stream Computing ◽  
Efficient Resource ◽  
Computing Environments ◽  
Time And Energy

Big Data Management in the Context of Real-Time Data Warehousing

2013 ◽  
pp. 150-176
Author(s):  
M. Asif Naeem ◽  
Gillian Dobbie ◽  
Gerald Weber
Keyword(s):  
Big Data ◽  
Data Integration ◽  
Real Time ◽  
Real Life ◽  
Skewed Distribution ◽  
Stream Data ◽  
Time Data ◽  
Master Data ◽  
Real Time Data ◽  
Resource Aware

In order to make timely and effective decisions, businesses need the latest information from big data warehouse repositories. To keep these repositories up to date, real-time data integration is required. An important phase in real-time data integration is data transformation where a stream of updates, which is huge in volume and infinite, is joined with large disk-based master data. Stream processing is an important concept in Big Data, since large volumes of data are often best processed immediately. A well-known algorithm called Mesh Join (MESHJOIN) was proposed to process stream data with disk-based master data, which uses limited memory. MESHJOIN is a candidate for a resource-aware system setup. The problem that the authors consider in this chapter is that MESHJOIN is not very selective. In particular, the performance of the algorithm is always inversely proportional to the size of the master data table. As a consequence, the resource consumption is in some scenarios suboptimal. They present an algorithm called Cache Join (CACHEJOIN), which performs asymptotically at least as well as MESHJOIN but performs better in realistic scenarios, particularly if parts of the master data are used with different frequencies. In order to quantify the performance differences, the authors compare both algorithms with a synthetic dataset of a known skewed distribution as well as TPC-H and real-life datasets.

Energy-Saving QoS Resource Management of Virtualized Networked Data Centers for Big Data Stream Computing

Big Data ◽  
2016 ◽  
pp. 848-886
Author(s):  
Nicola Cordeschi ◽  
Mohammad Shojafar ◽  
Danilo Amendola ◽  
Enzo Baccarelli
Keyword(s):  
Big Data ◽  
Resource Management ◽  
Real Time ◽  
Data Stream ◽  
Data Centers ◽  
Software As A Service ◽  
Stream Computing ◽  
Management Optimization ◽  
Hard Real Time ◽  
Optimal Scheduler

In this chapter, the authors develop the scheduler which optimizes the energy-vs.-performance trade-off in Software-as-a-Service (SaaS) Virtualized Networked Data Centers (VNetDCs) that support real-time Big Data Stream Computing (BDSC) services. The objective is to minimize the communication-plus-computing energy which is wasted by processing streams of Big Data under hard real-time constrains on the per-job computing-plus-communication delays. In order to deal with the inherently nonconvex nature of the resulting resource management optimization problem, the authors develop a solving approach that leads to the lossless decomposition of the afforded problem into the cascade of two simpler sub-problems. The resulting optimal scheduler is amenable of scalable and distributed adaptive implementation. The performance of a Xen-based prototype of the scheduler is tested under several Big Data workload traces and compared with the corresponding ones of some state-of-the-art static and sequential schedulers.

Trends and Technologies in Big Data Processing

2020 ◽  
pp. 17-42
Author(s):  
Amitava Choudhury ◽  
Kalpana Rangra
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Internet Of Things ◽  
Data Processing ◽  
Real Time ◽  
Computing Technology ◽  
Time Data ◽  
Big Data Processing ◽  
Real Time Data ◽  
Real Time Data Processing

Data type and amount in human society is growing at an amazing speed, which is caused by emerging new services such as cloud computing, internet of things, and location-based services. The era of big data has arrived. As data has been a fundamental resource, how to manage and utilize big data better has attracted much attention. Especially with the development of the internet of things, how to process a large amount of real-time data has become a great challenge in research and applications. Recently, cloud computing technology has attracted much attention to high performance, but how to use cloud computing technology for large-scale real-time data processing has not been studied. In this chapter, various big data processing techniques are discussed.

Sign in / Sign up

Export Citation Format

Share Document