scholarly journals Optimized memory model for hadoop map reduce framework

Author(s):  
Archana Bhaskar ◽  
Rajeev Ranjan

Map Reduce is the preferred computing framework used in large data analysis and processing applications. Hadoop is a widely used Map Reduce framework across different community due to its open source nature. Cloud service provider such as Microsoft azure HDInsight offers resources to its customer and only pays for their use. However, the critical challenges of cloud service provider is to meet user task Service level agreement (SLA) requirement (task deadline). Currently, the onus is on client to compute the amount of resource required to run a job on cloud. This work present a novel memory optimization model for Hadoop Map Reduce framework namely MOHMR (Optimized Hadoop Map Reduce) to process data in real-time and utilize system resource efficiently. The MOHMR present accurate model to compute job memory optimization and also present a model to provision the amount of cloud resource required to meet task deadline. The MOHMR first build a profile for each job and computes memory optimization time of job using greedy approach. Experiment are conducted on Microsoft Azure HDInsight cloud platform considering different application such as text computing and bioinformatics application to evaluate performance of MOHMR of over existing model shows significant performance improvement in terms of computation time. Experiment are conducted on Microsoft Azure HDInsight cloud. Overall, good correlation is reported between practical memory optimization values and theoretical memory optimization values.

Author(s):  
D C Vinutha ◽  
G.T Raju

MapReduce is the preferred computing framework used in large data analysis and processing applications. Hadoop is a widely used MapReduce framework across different community due to its open source nature. Cloud service provider such as Microsoft azure HDInsight offers resources to its customer and only pays for their use. However, the critical challenges of cloud service provider is to meet user task Service level agreement (SLA) requirement (task deadline). Currently, the onus is on client to compute the amount of resource required to run a job on cloud. This work present a novel makespan model for Hadoop MapReduce framework namely OHMR (Optimized Hadoop MapReduce) to process data in real-time and utilize system resource efficiently. The OHMR present accurate model to compute job makespan time and also present a model to provision the amount of cloud resource required to meet task deadline. The OHMR first build a profile for each job and computes makespan time of job using greedy approach. Furthermore, to provision amount of resource required to meet task deadline Lagrange Multipliers technique is applied. Experiment are conducted on Microsoft Azure HDInsight cloud platform considering different application such as text computing and bioinformatics application to evaluate performance of OHMR of over existing model shows significant performance improvement in terms of computation time. Experiment are conducted on Microsoft Azure HDInsight cloud. Overall good correlation is reported between practical makespan values and theoretical makespan values.


2020 ◽  
Vol 8 (5) ◽  
pp. 4124-4232

Picking up public cloud service providers is now becoming a harder task in an enterprise organization. This paper will help in reducing more hesitation to choose a public cloud service provider. This paper is highlighting computation, storage, and infrastructure is important to service features that have an impact when choosing cloud service providers. Compare these three (AWS, Microsoft Azure, GCP) CSPs concerning service, price, advantages, and highlight significant service features. Studies discuss the primary reason to choose a CSP that normally enhance features, familiarity with the brand, suitable for organization and security parameters considered when choosing CSP. Amazon Web Services proved its leadership by maintaining about 33% share in the market throughout for several quarters irrespective of the market size increased by a factor of 3. Microsoft has shown prominent performance in SaaS. Since 2008, after introducing PaaS in the form of Google App Engine, Google is continuously enhancing its cloud computing services of Google Cloud Platform.


2018 ◽  
Vol 6 (5) ◽  
pp. 340-345
Author(s):  
Rajat Pugaliya ◽  
Madhu B R

Cloud Computing is an emerging field in the IT industry. Cloud computing provides computing services over the Internet. Cloud Computing demand increasing drastically, which has enforced cloud service provider to ensure proper resource utilization with less cost and less energy consumption. In recent time various consolidation problems found in cloud computing like the task, VM, and server consolidation. These consolidation problems become challenging for resource utilization in cloud computing. We found in the literature review that there is a high level of coupling in resource utilization, cost, and energy consumption. The main challenge for cloud service provider is to maximize the resource utilization, reduce the cost and minimize the energy consumption. The dynamic task consolidation of virtual machines can be a way to solve the problem. This paper presents the comparative study of various task consolidation algorithms.


Cloud service provider in cloud environment will provide or provision resource based on demand from the user. The cloud service provider (CSP) will provide resources as and when required or demanded by the user for execution of the job on the cloud environment. The CSP will perform this in a static and dynamic manner. The CSP should also consider various other factors in order to provide the resources to the user, the prime among that will be the Service Level Agreement (SLA), which is normally signed by the user and cloud service provider during the inception phase of service. There are many algorithm which are used in order to allocate resources to the user in cloud environment. The algorithm which is proposed will be used to reduce the amount of energy utilized in performing various job execution in cloud environment. Here the energy utilized for execution of various jobs are taken into account by increasing the number of virtual machines that are used on a single physical host system. There is no thumb rule to calculate the number of virtual machines to be executed on a single host. The same can be derived by calculating the amount of space, speed required along with the time to execute the job on a virtual machine. Based up on this we can derive the number of Virtual machine on a single host system. There can be 10 virtual machines on a single system or even 20 number of virtual machines on single physical system. But if the same is calculated by the equation then the result will be exactly matching with the threshold capacity of the physical system[1]. If more number of physical systems are used to execute fewer virtual machines on each then the amount of energy consumed will be very high. So in order to reduce the energy consumption , the algorithm can be used will not only will help to calculate the number of virtual machines on single physical system , but also will help to reduce the energy as less number of physical systems will be in need[2].


2021 ◽  
Vol 17 (4) ◽  
pp. 75-88
Author(s):  
Padmaja Kadiri ◽  
Seshadri Ravala

Security threats are unforeseen attacks to the services provided by the cloud service provider. Depending on the type of attack, the cloud service and its associated features will be unavailable. The mitigation time is an integral part of attack recovery. This research paper explores the different parameters that will aid in predicting the mitigation time after an attack on cloud services. Further, the paper presents machine learning models that can predict the mitigation time. The paper presents the kernel-based machine learning models that can predict the average mitigation time during security attacks. The analysis of the results shows that the kernel-based models show 87% accuracy in predicting the mitigation time. Furthermore, the paper explores the performance of the kernel-based machine learning models based on the regression-based predictive models. The regression model is used as a benchmark model to analyze the performance of the machine learning-based predictive models in the prediction of mitigation time in the wake of an attack.


Author(s):  
Alexander Herzfeldt ◽  
Sebastian Floerecke ◽  
Christoph Ertl ◽  
Helmut Krcmar

With the increasing maturity of cloud technologies and the growing demand from customers, the cloud computing ecosystem has been expanding continuously with both incumbents and new entrants, whereby it has become more distributed and less transparent. For cloud service providers previously focusing on growth strategies, it is now necessary to shift the attention to providing service efficiently, as well as profitably. Based on 14 explorative interviews with cloud service experts, the relationship between cloud service provider profitability and value facilitation, which stands for the capability to build up resources in advance of future customer engagements, is investigated. The results indicate a positive relationship between cloud service profitability and value facilitation and deliver valuable insights for both researchers and practitioners. In particular, guidelines on how to design profitable cloud service offerings are discussed.


2022 ◽  
pp. 205-224
Author(s):  
Dhiviya Ram

One of the most unique forms of contracting is apparent in cloud computing. Cloud computing, unlike other conventional methods, has adopted a different approach in the formation of binding contract that will be used for the governance of the cloud. This method is namely the clickwrap agreement. Click wrap agreement follows a take it or leave it basis in which the end users are provided with limited to no option in terms of having a say on the contract that binds them during the use of cloud services. The terms found in the contract are often cloud service provider friendly and will be less favourable to the end user. In this article, the authors examine the terms that are often found in the cloud computing agreement as well as study the benefit that is entailed in adopting this contracting method. This chapter has undertaken a qualitative study that comprises interviews of cloud service providers in Malaysia. Hence, this study is a novel approach that also provides insight in terms of the cloud service provider perspective regarding the click wrap agreement.


Sign in / Sign up

Export Citation Format

Share Document