scholarly journals Lightweight dynamic integration of opportunistic resources

2020 ◽  
Vol 245 ◽  
pp. 07040
Author(s):  
Max Fischer ◽  
Eileen Kuehn ◽  
Manuel Giffels ◽  
Matthias Jochen Schnepf ◽  
Andreas Petzold ◽  
...  
Keyword(s):  
Virtual Machines ◽  
Resource Provisioning ◽  
Third Party ◽  
New Approach ◽  
Dynamic Resource Provisioning ◽  
Key Concepts ◽  
Resource Requirements ◽  
Computing Grid ◽  
Dynamic Resource ◽  
Heterogeneous Resources

To satisfy future computing demands of the Worldwide LHC Computing Grid (WLCG), opportunistic usage of third-party resources is a promising approach. While the means to make such resources compatible with WLCG requirements are largely satisfied by virtual machines and containers technologies, strategies to acquire and disband many resources from many providers are still a focus of current research. Existing meta-schedulers that manage resources in the WLCG are hitting the limits of their design when tasked to manage heterogeneous resources from many diverse resource providers. To provide opportunistic resources to the WLCG as part of a regular WLCG site, we propose a new meta-scheduling approach suitable for opportunistic, heterogeneous resource provisioning. Instead of anticipating future resource requirements, our approach observes resource usage and promotes well-used resources. Following this approach, we have developed an inherently robust meta-scheduler, COBalD, for managing diverse, heterogeneous resources given unpredictable resource requirements. This paper explains the key concepts of our approach, and discusses the benefits and limitations of our new approach to dynamic resource provisioning compared to previous approaches.

A Value Based Dynamic Resource Provisioning Model in Cloud

2013 ◽  
Vol 3 (2) ◽  
pp. 35-46 ◽  
Author(s):  
Sandeep K. Sood
Keyword(s):  
Resource Utilization ◽  
Virtual Machines ◽  
Fault Tolerant ◽  
Resource Provisioning ◽  
Resource Usage ◽  
Resource Manager ◽  
Usage Rate ◽  
Computing Paradigm ◽  
A Value ◽  
Dynamic Resource Provisioning

Cloud computing has become an innovative computing paradigm, which aims at providing reliable, customized, Quality of Service (QoS) and guaranteed computing infrastructures for users. Efficient resource provisioning is required in cloud for effective resource utilization. For resource provisioning, cloud provides virtualized computing resources that are dynamically scalable. This property of cloud differentiates it from the traditional computing paradigm. But the initialization of a new virtual instance causes a several minutes delay in the hardware resource allocation. Furthermore, cloud provides a fault tolerant service to its clients using the virtualization. But, in order to attain higher resource utilization over this technology, a technique or a strategy is needed using which virtual machines can be deployed over physical machines by predicting its need in advance so that the delay can be avoided. To address these issues, a value based prediction model in this paper is proposed for resource provisioning in which a resource manager is used for dynamically allocating or releasing a virtual machine depending upon the resource usage rate. In order to know the recent resource usage rate, the resource manager uses sliding window to analyze the resource usage rate and to predict the system behavior in advance. By predicting the resource requirements in advance, a lot of processing time can be saved. Earlier, a server has to perform all the calculations regarding the resource usage that in turn wastes a lot of processing power thus decreasing its overall capacity to handle the incoming request. The main feature of the proposed model is that a lot of load is being shifted from the individual server to the resource manager as it performs all the calculations and therefore the server is free to handle the incoming requests to its full capacity.

scholarly journals Experience with dynamic resource provisioning of the CMS online cluster using a cloud overlay

2019 ◽  
Vol 214 ◽  
pp. 07017
Author(s):  
Jean-Marc Andre ◽  
Ulf Behrens ◽  
James Branson ◽  
Philipp Brummer ◽  
Olivier Chaze ◽  
...  
Keyword(s):  
Virtual Machines ◽  
Hadron Collider ◽  
A Priori ◽  
Resource Provisioning ◽  
Static Mode ◽  
Level Trigger ◽  
Network Connection ◽  
Use Of Resources ◽  
Dynamic Resource Provisioning ◽  
High Level

The primary goal of the online cluster of the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider (LHC) is to build event data from the detector and to select interesting collisions in the High Level Trigger (HLT) farm for offline storage. With more than 1500 nodes and a capacity of about 850 kHEPSpecInt06, the HLT machines represent similar computing capacity of all the CMS Tier1 Grid sites together. Moreover, it is currently connected to the CERN IT datacenter via a dedicated 160 Gbps network connection and hence can access the remote EOS based storage with a high bandwidth. In the last few years, a cloud overlay based on OpenStack has been commissioned to use these resources for the WLCG when they are not needed for data taking. This online cloud facility was designed for parasitic use of the HLT, which must never interfere with its primary function as part of the DAQ system. It also allows to abstract from the different types of machines and their underlying segmented networks. During the LHC technical stop periods, the HLT cloud is set to its static mode of operation where it acts like other grid facilities. The online cloud was also extended to make dynamic use of resources during periods between LHC fills. These periods are a-priori unscheduled and of undetermined length, typically of several hours, once or more a day. For that, it dynamically follows LHC beam states and hibernates Virtual Machines (VM) accordingly. Finally, this work presents the design and implementation of a mechanism to dynamically ramp up VMs when the DAQ load on the HLT reduces towards the end of the fill.

Sign in / Sign up

Export Citation Format

Share Document