scholarly journals The Improvement Plan for Fire Response Time using Big Data

2015 ◽  
Vol 8 (23) ◽  
Author(s):  
Byung Sung Kim ◽  
Do Yeon Kim ◽  
Keun Won Kim ◽  
Seong Taek Park
Keyword(s):  
Big Data ◽  
Response Time ◽  
Fire Response ◽  
Improvement Plan

scholarly journals Support Vector Regression based Mapreduce Throttled Load Balancer for Data Centers

2019 ◽  
Vol 9 (1) ◽  
pp. 4162-4171
Keyword(s):  
Big Data ◽  
Load Balancing ◽  
Response Time ◽  
Support Vector Regression ◽  
Virtual Machine ◽  
Data Centers ◽  
Migration Time ◽  
Support Vector ◽  
Load Balancer ◽  
Cloud User

The process of analyzing big data and other valuable information is a significant process in the cloud. Since big data processing utilizes a large number of resources for completing certain tasks. Therefore, the incoming tasks are allocated with better utilization of resources to minimize the workload across the server in the cloud. The conventional load balancing technique failed to balance the load effectively among data centers and dynamic QoS requirements of big data application. In order to improve the load balancing with maximum throughput and minimum makespan, Support Vector Regression based MapReduce Throttled Load Balancing (SVR-MTLB) technique is introduced. Initially, a large number of cloud user requests (data/file) are sent to the cloud server from different locations. After collecting the cloud user request, the SVR-MTLB technique balances the workload of the virtual machine with the help of support vector regression. The load balancer uses the index table for maintaining the virtual machines. Then, map function performs the regression analysis using optimal hyperplane and provides three resource status of the virtual machine namely overloaded, less loaded and balanced load. After finding the less loaded VM, the load balancer sends the ID of the virtual machine to the data center controller. The controller performs migration of the task from an overloaded VM to a less loaded VM at run time. This in turn assists to minimize the response time. Experimental evaluation is carried out on the factors such as throughput, makespan, migration time and response time with respect to a number of tasks. The experimental results reported that the proposed SVR-MTLB technique obtains high throughput with minimum response time, makespan as well as migration time than the state -of -the -art methods.

scholarly journals Enhancing Big Data Analysis by using Map-reduce Technique

2018 ◽  
Vol 7 (1) ◽  
pp. 113-116
Author(s):  
Alaa Hussein Al-Hamami ◽  
Ali Adel Flayyih
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Response Time ◽  
Data Analytics ◽  
Big Data Analysis ◽  
Map Reduce ◽  
Fast Method ◽  
Traditional Methods ◽  
Electroencephalogram Eeg

Database is defined as a set of data that is organized and distributed in a manner that permits the user to access the data being stored in an easy and more convenient manner. However, in the era of big-data the traditional methods of data analytics may not be able to manage and process the large amount of data. In order to develop an efficient way of handling big-data, this work enhances the use of Map-Reduce technique to handle big-data distributed on the cloud. This approach was evaluated using Hadoop server and applied on Electroencephalogram (EEG) Big-data as a case study. The proposed approach showed clear enhancement on managing and processing the EEG Big-data with average of 50% reduction on response time. The obtained results provide EEG researchers and specialist with an easy and fast method of handling the EEG big data.

Research on dynamic load balancing of data flow under big data platform

2021 ◽  
pp. 2150014
Author(s):  
Junlin Sun ◽  
Yi Zhang
Keyword(s):  
Big Data ◽  
Load Balancing ◽  
Response Time ◽  
Dynamic Load ◽  
Loss Rate ◽  
Data Flow ◽  
Experimental Results ◽  
Dynamic Load Balancing ◽  
Data Platform ◽  
Flow Loss

In the big data platform, because of the large amount of data, the problem of load imbalance is prominent. Most of the current load balancing methods have problems such as high data flow loss rate and long response time; therefore, more effective load balancing method is urgently needed. Taking HBase as the research subject, the study analyzed the dynamic load balancing method of data flow. First, the HBase platform was introduced briefly, and then the dynamic load-balancing algorithm was designed. The data flow was divided into blocks, and then the load of nodes was predicted based on the grey prediction GM(1,1) model. Finally, the load was migrated through the dynamic adjustable method to achieve load balancing. The experimental results showed that the accuracy of the method for load prediction was high, the average error percentage was 0.93%, and the average response time was short; under 3000 tasks, the response time of the method designed in this study was 14.17% shorter than that of the method combining TV white space (TVWS) and long-term evolution (LTE); the average flow of nodes with the largest load was also smaller, and the data flow loss rate was basically 0%. The experimental results show the effectiveness of the proposed method, which can be further promoted and applied in practice.

scholarly journals Migrating From Data Mining to Big Data Mining

2018 ◽  
Vol 7 (3.4) ◽  
pp. 13
Author(s):  
Gourav Bathla ◽  
Himanshu Aggarwal ◽  
Rinkle Rani
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Response Time ◽  
Large Scale ◽  
Naive Bayes ◽  
Naïve Bayes ◽  
Data Mining Algorithm ◽  
Big Data Mining ◽  
Data Mining Algorithms ◽  
Mining Algorithms

Data mining is one of the most researched fields in computer science. Several researches have been carried out to extract and analyse important information from raw data. Traditional data mining algorithms like classification, clustering and statistical analysis can process small scale of data with great efficiency and accuracy. Social networking interactions, business transactions and other communications result in Big data. It is large scale of data which is not in competency for traditional data mining techniques. It is observed that traditional data mining algorithms are not capable for storage and processing of large scale of data. If some algorithms are capable, then response time is very high. Big data have hidden information, if that is analysed in intelligent manner can be highly beneficial for business organizations. In this paper, we have analysed the advancement from traditional data mining algorithms to Big data mining algorithms. Applications of traditional data mining algorithms can be straight forward incorporated in Big data mining algorithm. Several studies have analysed traditional data mining with Big data mining, but very few have analysed most important algortihsm within one research work, which is the core motive of our paper. Readers can easily observe the difference between these algorthithms with  pros and cons. Mathemtics concepts are applied in data mining algorithms. Means and Euclidean distance calculation in Kmeans, Vectors application and margin in SVM and Bayes therorem, conditional probability in Naïve Bayes algorithm are real examples.  Classification and clustering are the most important applications of data mining. In this paper, Kmeans, SVM and Naïve Bayes algorithms are analysed in detail to observe the accuracy and response time both on concept and empirical perspective. Hadoop, Mapreduce etc. Big data technologies are used for implementing Big data mining algorithms. Performace evaluation metrics like speedup, scaleup and response time are used to compare traditional mining with Big data mining.  

RE-STORM: REAL-TIME ENERGY EFFICIENT DATA ANALYSIS ADAPTING STORM PLATFORM

2016 ◽  
Vol 78 (10) ◽  
Author(s):  
Rizwan Patan ◽  
Rajasekhara Babu M.
Keyword(s):  
Energy Efficiency ◽  
Big Data ◽  
Response Time ◽  
Energy Efficient ◽  
Data Stream ◽  
High Performance ◽  
High Energy ◽  
Streaming Data ◽  
Stream Computing ◽  
High Energy Efficiency

It is necessary to model an energy efficient and stream optimization towards achieve high energy efficiency for Streaming data without degrading response time in big data stream computing. This paper proposes an Energy Efficient Traffic aware resource scheduling and Re-Streaming Stream Structure to replace a default scheduling strategy of storm is entitled as re-storm. The model described in three parts; First, a mathematical relation among energy consumption, low response time and high traffic streams. Second, various approaches provided for reducing an energy without affecting response time and which provides high performance in overall stream computing in big data. Third, re-storm deployed energy efficient traffic aware scheduling on the storm platform. It allocates worker nodes online by using hot-swapping technique with task utilizing by energy consolidation through graph partitioning. Moreover, re-storm is achieved high energy efficiency, low response time in all types of data arriving speeds.it is suitable for allocation of worker nodes in a storm topology. Experiment results have been demonstrated the comparing existing strategies which are dealing with energy issues without affecting or reducing response time for a different data stream speed levels. Finally, it shows that the re-storm platform achieved high energy efficiency and low response time when compared to all existing approaches.

On the Dynamic Shifting of the MapReduce Timeout

2016 ◽  
pp. 1-22
Author(s):  
Bunjamin Memishi ◽  
Shadi Ibrahim ◽  
Maria S. Perez ◽  
Gabriel Antoniu
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Response Time ◽  
Case Studies ◽  
Large Scale ◽  
Failure Detection ◽  
Mapreduce Framework ◽  
Big Data Applications ◽  
Design Ideas

MapReduce has become a relevant framework for Big Data processing in the cloud. At large-scale clouds, failures do occur and may incur unwanted performance degradation to Big Data applications. As the reliability of MapReduce depends on how well they detect and handle failures, this book chapter investigates the problem of failure detection in the MapReduce framework. The case studies of this contribution reveal that the current static timeout value is not adequate and demonstrate significant variations in the application's response time with different timeout values. While arguing that comparatively little attention has been devoted to the failure detection in the framework, the chapter presents design ideas for a new adaptive timeout.

Performance Improvement of Data Analysis of IoT Applications Using Re-Storm in Big Data Stream Computing Platform

2016 ◽  
Vol 22 ◽  
pp. 141-151 ◽  
Author(s):  
Patan Rizwan ◽  
M. Rajasekhara Babu
Keyword(s):  
Energy Efficiency ◽  
Big Data ◽  
Response Time ◽  
Data Stream ◽  
Dynamic Scheduling ◽  
High Energy ◽  
Stream Computing ◽  
Time Data ◽  
Iot Applications ◽  
High Energy Efficiency

Big Data and Internet of Things (IoT) are Two Popular Technical Terms in Current IT Industry. the Analysis of Iot Data Consumes more Energy since it is Huge in Size. this Paper Proposes a Methodology re-Storm that Addresses Energy Issues and Response Time of Iot Applications Data. it Uses Big Data Stream Computing for re-Storm against Existing Method Storm. the Storm Failed to Address Dynamic Scheduling but re-Storm Deals with Energy-Efficient Traffic Aware Resource Scheduling. this Paper Presents a Model that Different Traffic Arriving Rate of Streams re-Storm at Multiple Traffic Levels for High Energy Efficiency, Low Response Time. it Deals at Three Levels, Firstly, a Mathematical Model for High Energy Efficiency, Low Response Time. Secondly, Allocation of Resources Bearing in Mind DVFS (Dynamic Voltage and Frequency Scaling) Methods and Existing Effective Optimal Consolidation Methods. Thirdly, Online Task Allocation Using Hot Swapping Technique, Streaming Graph Optimizing. Finally, the Experimental Results Show that re-Storm has been Improved the Performance 30-40% against Storm for Real Time Data of Iot Applications.

Sign in / Sign up

Export Citation Format

Share Document