Load Balancing Approach for a MapReduce Job Running on a Heterogeneous Hadoop Cluster

2020 ◽  
pp. 289-298
Author(s):  
Kamalakant Laxman Bawankule ◽  
Rupesh Kumar Dewang ◽  
Anil Kumar Singh
Keyword(s):  
Load Balancing ◽  
Hadoop Cluster

Efficient Dynamic Resource Allocation in Hadoop Multiclusters for Load- Balancing Problem

2020 ◽  
Vol 13 (4) ◽  
pp. 686-693
Author(s):  
Karthikeyan S. ◽  
Hari Seetha ◽  
Manimegalai R.
Keyword(s):  
Resource Allocation ◽  
Load Balancing ◽  
Performance Test ◽  
Data Locality ◽  
Speculative Execution ◽  
Map Reduce ◽  
Slot Allocation ◽  
Allocation Policy ◽  
Hadoop Cluster ◽  
The Times

Background: ‘Map-Reduce’ is the framework and its processing of data by rationalizing the distributed servers. also its running the various tasks in parallel way. The most important problem in map reduce environment is Resource Allocation in distributed environments and data locality to its corresponding slave nodes. If the applications are not scheduled properly then it leads to load unbalancing problems in the cloud environments. Objective: This Research suggests a new dynamic way of allocating the resources in hadoop multi cluster environment in order to effectively monitor the nodes for faster computation, load balancing and data locality issues. The dynamic slot allocation is explained theoretically in order to address the problems of pre configuration, speculative execution, delay in scheduling and pre slot allocation in hadoop environments. Experiment is done with Hadoop cluster which increases the efficiency of the nodes and solves the load balancing problem. Methods: The Current design of Map Reduce Hadoop systems are affected by underutilization of slots. The reason is the number of maps and reducer is allotted is smaller than the available number of maps and reducers. In Hadoop most of the times its noticed that dynamic slot allocation policy, the mapper or reducers are idle. So we can use those unused map tasks to overloaded reducer tasks in-order to increase the efficiency of MR jobs and vice versa. Results: The real time experiment was implemented with Multinode Hadoop cluster map reduce jobs of file size 1giga bytes to 5 giga bytes and various performance test has been taken. Conclusion: This paper focused on Hadoop map reduce resource allocation management techniques for multi cluster environments. It proposes a novel dynamic slot allocation policy to improve the performance of yarn scheduler and eliminates the load balancing problem. This work proves that dynamic slot allocation is outperforms more than yarn framework. In future it considered to concentrate on CPU bandwidth and processing time.

Task Scheduling Algorithm based on Resources Segregation in Cloud Environment

2018 ◽  
pp. 96-101
Author(s):  
Shailendra Raghuvanshi ◽  
Priyanka Dubey
Keyword(s):  
Load Balancing ◽  
Task Scheduling ◽  
Waiting Time ◽  
Virtual Machines ◽  
Scheduling Algorithm ◽  
Cloud Environment ◽  
Task Scheduling Algorithm ◽  
Machine Utilization ◽  
Independent Tasks ◽  
Optimal Machine

Load balancing of non-preemptive independent tasks on virtual machines (VMs) is an important aspect of task scheduling in clouds. Whenever certain VMs are overloaded and remaining VMs are under loaded with tasks for processing, the load has to be balanced to achieve optimal machine utilization. In this paper, we propose an algorithm named honey bee behavior inspired load balancing, which aims to achieve well balanced load across virtual machines for maximizing the throughput. The proposed algorithm also balances the priorities of tasks on the machines in such a way that the amount of waiting time of the tasks in the queue is minimal. We have compared the proposed algorithm with existing load balancing and scheduling algorithms. The experimental results show that the algorithm is effective when compared with existing algorithms. Our approach illustrates that there is a significant improvement in average execution time and reduction in waiting time of tasks on queue using workflowsim simulator in JAVA.

Development of Load Balancing Systems in a Parallel MRP System

2003 ◽  
Vol 123 (10) ◽  
pp. 1847-1857
Author(s):  
Takahiro Tsukishima ◽  
Masahiro Sato ◽  
Hisashi Onari
Keyword(s):  
Load Balancing

A Load Balancing Method for M2M Proxy Server to Realize Multi-service Connectivity and its Implementation

2014 ◽  
Vol 134 (8) ◽  
pp. 1104-1113
Author(s):  
Shinji Kitagami ◽  
Yosuke Kaneko ◽  
Hidetoshi Kambe ◽  
Shigeki Nankaku ◽  
Takuo Suganuma
Keyword(s):  
Load Balancing ◽  
Proxy Server
Sign in / Sign up

Export Citation Format

Share Document