Efficiently Scheduling Hadoop Cluster in Cloud Environment

2014 ◽  
pp. 95-102
Author(s):  
D. Raghuram ◽  
N. Gayathri ◽  
G. Sudha Sadasivam
Keyword(s):  
Cloud Environment ◽  
Hadoop Cluster

scholarly journals A Novel Hadoop Security Model for Addressing Malicious Collusive Workers

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Amr M. Sauber ◽  
Ahmed Awad ◽  
Amr F. Shawish ◽  
Passent M. El-Kafrawy
Keyword(s):  
Security Model ◽  
Cloud Environment ◽  
Master Node ◽  
Computing Model ◽  
System A ◽  
Proposed Model ◽  
Data Production ◽  
Hadoop Cluster ◽  
High Level ◽  
Novel Model

With the daily increase of data production and collection, Hadoop is a platform for processing big data on a distributed system. A master node globally manages running jobs, whereas worker nodes process partitions of the data locally. Hadoop uses MapReduce as an effective computing model. However, Hadoop experiences a high level of security vulnerability over hybrid and public clouds. Specially, several workers can fake results without actually processing their portions of the data. Several redundancy-based approaches have been proposed to counteract this risk. A replication mechanism is used to duplicate all or some of the tasks over multiple workers (nodes). A drawback of such approaches is that they generate a high overhead over the cluster. Additionally, malicious workers can behave well for a long period of time and attack later. This paper presents a novel model to enhance the security of the cloud environment against untrusted workers. A new component called malicious workers’ trap (MWT) is developed to run on the master node to detect malicious (noncollusive and collusive) workers as they convert and attack the system. An implementation to test the proposed model and to analyze the performance of the system shows that the proposed model can accurately detect malicious workers with minor processing overhead compared to vanilla MapReduce and Verifiable MapReduce (V-MR) model [1]. In addition, MWT maintains a balance between the security and usability of the Hadoop cluster.

Task Scheduling Algorithm based on Resources Segregation in Cloud Environment

2018 ◽  
pp. 96-101
Author(s):  
Shailendra Raghuvanshi ◽  
Priyanka Dubey
Keyword(s):  
Load Balancing ◽  
Task Scheduling ◽  
Waiting Time ◽  
Virtual Machines ◽  
Scheduling Algorithm ◽  
Cloud Environment ◽  
Task Scheduling Algorithm ◽  
Machine Utilization ◽  
Independent Tasks ◽  
Optimal Machine

Load balancing of non-preemptive independent tasks on virtual machines (VMs) is an important aspect of task scheduling in clouds. Whenever certain VMs are overloaded and remaining VMs are under loaded with tasks for processing, the load has to be balanced to achieve optimal machine utilization. In this paper, we propose an algorithm named honey bee behavior inspired load balancing, which aims to achieve well balanced load across virtual machines for maximizing the throughput. The proposed algorithm also balances the priorities of tasks on the machines in such a way that the amount of waiting time of the tasks in the queue is minimal. We have compared the proposed algorithm with existing load balancing and scheduling algorithms. The experimental results show that the algorithm is effective when compared with existing algorithms. Our approach illustrates that there is a significant improvement in average execution time and reduction in waiting time of tasks on queue using workflowsim simulator in JAVA.

A Review on Load Balancing Model Using Best Partition Technique

2017 ◽  
Vol 7 (8) ◽  
pp. 284
Author(s):  
M. Chaitanya ◽  
K. Durga Charan
Keyword(s):  
Cloud Computing ◽  
Fault Tolerance ◽  
Load Balancing ◽  
Load Balance ◽  
Large Impact ◽  
Cloud Environment ◽  
Public Cloud ◽  
The Public ◽  
Partition Technique ◽  
Textual Content

Load balancing makes cloud computing greater knowledgeable and could increase client pleasure. At reward cloud computing is among the all most systems which offer garage of expertise in very lowers charge and available all the time over the net. However, it has extra vital hassle like security, load administration and fault tolerance. Load balancing inside the cloud computing surroundings has a large impact at the presentation. The set of regulations relates the sport idea to the load balancing manner to amplify the abilties in the public cloud environment. This textual content pronounces an extended load balance mannequin for the majority cloud concentrated on the cloud segregating proposal with a swap mechanism to select specific strategies for great occasions.

Workflow Scheduling for Scientific Application in Homogeneous Cloud Environment

2017 ◽  
Vol 7 (7) ◽  
pp. 137
Author(s):  
. Monika ◽  
Pardeep Kumar ◽  
Sanjay Tyagi
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Large Data ◽  
Workflow Scheduling ◽  
Cloud Environment ◽  
Computing Environment ◽  
Scientific Application ◽  
Cloud Computing Environment ◽  
Flow Of Information

In Cloud computing environment QoS i.e. Quality-of-Service and cost is the key element that to be take care of. As, today in the era of big data, the data must be handled properly while satisfying the request. In such case, while handling request of large data or for scientific applications request, flow of information must be sustained. In this paper, a brief introduction of workflow scheduling is given and also a detailed survey of various scheduling algorithms is performed using various parameter.

Enhanced Data Security from Single Clouds to Multiclouds

2018 ◽  
Vol 6 (9) ◽  
pp. 66
Author(s):  
Umesh Banodha ◽  
Praveen Kumar Kataria
Keyword(s):  
Data Storage ◽  
Network Connectivity ◽  
Data Replication ◽  
Recovery Process ◽  
Cloud Environment ◽  
Storage Security ◽  
Cloud Data ◽  
Replication Strategy ◽  
Cloud Data Storage ◽  
Data Files

Cloud is an emerging technology that stores the necessary data and electronic form of data is produced in gigantic quantity. It is vital to maintain the efficacy of this data the need of data recovery services is highly essential. Cloud computing is anticipated as the vital foundation for the creation of IT enterprise and it is an impeccable solution to move databases and application software to big data centers where managing data and services is not completely reliable. Our focus will be on the cloud data storage security which is a vital feature when it comes to giving quality service. It should also be noted that cloud environment comprises of extremely dynamic and heterogeneous environment and because of high scale physical data and resources, the failure of data centre nodes is completely normal.Therefore, cloud environment needs effective adaptive management of data replication to handle the indispensable characteristic of the cloud environment. Disaster recovery using cloud resources is an attractive approach and data replication strategy which attentively helps to choose the data files for replication and the strategy proposed tells dynamically about the number of replicas and effective data nodes for replication. Thus, the objective of future algorithm is useful to help users together the information from a remote location where network connectivity is absent and secondly to recover files in case it gets deleted or wrecked because of any reason. Even, time oriented problems are getting resolved so in less time recovery process is executed.

Anomaly Detection and Categorization in Cloud Environment using Deep Learning Techniques

2019 ◽  
Vol 7 (5) ◽  
pp. 211-214
Author(s):  
Nidhi Thakkar ◽  
Miren Karamta ◽  
Seema Joshi ◽  
M. B. Potdar
Keyword(s):  
Deep Learning ◽  
Anomaly Detection ◽  
Cloud Environment ◽  
Learning Techniques

Load balancing in Fog-Cloud Environment

2019 ◽  
Vol 7 (6) ◽  
pp. 71-77
Author(s):  
T. A. Bhat ◽  
J. S. Saini
Keyword(s):  
Load Balancing ◽  
Cloud Environment

Cost Effective PSO Model for MapReduce in Cloud Environment

2018 ◽  
Vol 6 (4) ◽  
pp. 497-501 ◽  
Author(s):  
Vidhyasagar B S ◽  
◽  
◽  
◽  
◽  
...  
Keyword(s):  
Cost Effective ◽  
Cloud Environment
Sign in / Sign up

Export Citation Format

Share Document