Performance-Oriented VM Provisioning in Clouds

Managing applications on the cloud requires extensive decision making on the part of the Application Provider (AP). When an application faces changing workload, the services of the application are either scaled up or down in response. The services run on Virtual Machines (VM) or container instances. APs decide on how the application scales through VM provisioning and the placement of the services on the VMs. Various drivers guide this decision making. Application performance and cost are two such drivers. This thesis answers the question of how APs can meet the performance constraints of their applications while minimizing the cost of the running VMs. Two versions of the problem are presented. The first version expects to meet mean response time constraints given a deployment configuration through the replication of VMs and addition of virtual processors. The presented solution is based on layered bottlenecks. A case study shows the solution meets response time constraints and uses fewer resources in comparison to a simple utilization based approach. The second version adds the minimization of cost as an objective, where VM-types having different cost rates are used. This problem does not require a deployment configuration and provides a complete solution, where resources can be added and removed. A novel solution based on the layered bottleneck strength value with genetic algorithm has been presented. For the case study, a decision maker is implemented for a web application. The proposed solution is compared with three algorithms, all of which run within the decision maker. The results from the case study show that the proposed solution provides shorter runtime than the exhaustive search, and is able to meet response time constraints with near optimal minimization of cost. The solution also results in better cost than a plain genetic algorithm and random search, at the expense of slightly longer runtime.

Performance-Oriented VM Provisioning in Clouds

Managing applications on the cloud requires extensive decision making on the part of the Application Provider (AP). When an application faces changing workload, the services of the application are either scaled up or down in response. The services run on Virtual Machines (VM) or container instances. APs decide on how the application scales through VM provisioning and the placement of the services on the VMs. Various drivers guide this decision making. Application performance and cost are two such drivers. This thesis answers the question of how APs can meet the performance constraints of their applications while minimizing the cost of the running VMs. Two versions of the problem are presented. The first version expects to meet mean response time constraints given a deployment configuration through the replication of VMs and addition of virtual processors. The presented solution is based on layered bottlenecks. A case study shows the solution meets response time constraints and uses fewer resources in comparison to a simple utilization based approach. The second version adds the minimization of cost as an objective, where VM-types having different cost rates are used. This problem does not require a deployment configuration and provides a complete solution, where resources can be added and removed. A novel solution based on the layered bottleneck strength value with genetic algorithm has been presented. For the case study, a decision maker is implemented for a web application. The proposed solution is compared with three algorithms, all of which run within the decision maker. The results from the case study show that the proposed solution provides shorter runtime than the exhaustive search, and is able to meet response time constraints with near optimal minimization of cost. The solution also results in better cost than a plain genetic algorithm and random search, at the expense of slightly longer runtime.

Intelligent Virtual Machine Provisioning in Cloud Computing

Virtual machine (VM) provisioning is a common and critical problem in cloud computing. In industrial cloud platforms, there are a huge number of VMs provisioned per day. Due to the complexity and resource constraints, it needs to be carefully optimized to make cloud platforms effectively utilize the resources. Moreover, in practice, provisioning a VM from scratch requires fairly long time, which would degrade the customer experience. Hence, it is advisable to provision VMs ahead for upcoming demands. In this work, we formulate the practical scenario as the predictive VM provisioning (PreVMP) problem, where upcoming demands are unknown and need to be predicted in advance, and then the VM provisioning plan is optimized based on the predicted demands. Further, we propose Uncertainty-Aware Heuristic Search (UAHS) for solving the PreVMP problem. UAHS first models the prediction uncertainty, and then utilizes the prediction uncertainty in optimization. Moreover, UAHS leverages Bayesian optimization to interact prediction and optimization to improve its practical performance. Extensive experiments show that UAHS performs much better than state-of-the-art competitors on two public datasets and an industrial dataset. UAHS has been successfully applied in Microsoft Azure and brought practical benefits in real-world applications.

High-Throughput Cloud Computing with the Cloudscheduler VM Provisioning Service

We describe a high-throughput computing system for running jobs on public and private computing clouds using the HTCondor job scheduler and the cloudscheduler VM provisioning service. The distributed cloud computing system is designed to simultaneously use dedicated and opportunistic cloud resources at local and remote locations. It has been used for large-scale production particle physics workloads for many years using thousands of cores on three continents. A decade after its initial design and implementation, cloudscheduler has been modernized to take advantage of new software designs, improved operating system capabilities and support packages. The updated cloudscheduler is more resilient and scalable, with expanded capabilities. We present an overview of the original design and then describe the new version of the distributed compute cloud system. We conclude with a review of the current status and future plans.

Cloudscheduler: a VM provisioning system for a distributed compute cloud

We describe a high-throughput computing system for running jobs on public and private computing clouds using the HTCondor job scheduler and the cloudscheduler VM provisioning service. The distributed cloud computing system is designed to simultaneously use dedicated and opportunistic cloud resources at local and remote locations. It has been used for large scale production particle physics workloads for many years using thousands of cores on three continents. Cloudscheduler has been modernized to take advantage of new software designs, improved operating system capabilities and support packages. The result is a more resilient and scalable system, with expanded capabilities.