scholarly journals Intra- and Inter-Server Smart Task Scheduling for Profit and Energy Optimization of HPC Data Centers

2020 ◽  
Vol 10 (4) ◽  
pp. 32
Author(s):  
Sayed Ashraf Mamun ◽  
Alexander Gilday ◽  
Amit Kumar Singh ◽  
Amlan Ganguly ◽  
Geoff V. Merrett ◽  
...  
Keyword(s):  
Energy Consumption ◽  
Power Consumption ◽  
Data Center ◽  
High Performance ◽  
Data Centers ◽  
Virtual Machines ◽  
For Profit ◽  
Dynamic Voltage ◽  
High Power Consumption ◽  
Performance Computing

Servers in a data center are underutilized due to over-provisioning, which contributes heavily toward the high-power consumption of the data centers. Recent research in optimizing the energy consumption of High Performance Computing (HPC) data centers mostly focuses on consolidation of Virtual Machines (VMs) and using dynamic voltage and frequency scaling (DVFS). These approaches are inherently hardware-based, are frequently unique to individual systems, and often use simulation due to lack of access to HPC data centers. Other approaches require profiling information on the jobs in the HPC system to be available before run-time. In this paper, we propose a reinforcement learning based approach, which jointly optimizes profit and energy in the allocation of jobs to available resources, without the need for such prior information. The approach is implemented in a software scheduler used to allocate real applications from the Princeton Application Repository for Shared-Memory Computers (PARSEC) benchmark suite to a number of hardware nodes realized with Odroid-XU3 boards. Experiments show that the proposed approach increases the profit earned by 40% while simultaneously reducing energy consumption by 20% when compared to a heuristic-based approach. We also present a network-aware server consolidation algorithm called Bandwidth-Constrained Consolidation (BCC), for HPC data centers which can address the under-utilization problem of the servers. Our experiments show that the BCC consolidation technique can reduce the power consumption of a data center by up-to 37%.

scholarly journals Power consumption prediction in cloud data center using machine learning

2020 ◽  
Vol 10 (2) ◽  
pp. 1524 ◽  
Author(s):  
Deepika T. ◽  
Prakash P.
Keyword(s):  
Machine Learning ◽  
Cloud Computing ◽  
Power Consumption ◽  
Data Center ◽  
Data Centers ◽  
Virtual Machines ◽  
Service Providers ◽  
Business World ◽  
Cloud Data ◽  
Cloud Data Centers

The flourishing development of the cloud computing paradigm provides several services in the industrial business world. Power consumption by cloud data centers is one of the crucial issues for service providers in the domain of cloud computing. Pursuant to the rapid technology enhancements in cloud environments and data centers augmentations, power utilization in data centers is expected to grow unabated. A diverse set of numerous connected devices, engaged with the ubiquitous cloud, results in unprecedented power utilization by the data centers, accompanied by increased carbon footprints. Nearly a million physical machines (PM) are running all over the data centers, along with (5 – 6) million virtual machines (VM). In the next five years, the power needs of this domain are expected to spiral up to 5% of global power production. The virtual machine power consumption reduction impacts the diminishing of the PM’s power, however further changing in power consumption of data center year by year, to aid the cloud vendors using prediction methods. The sudden fluctuation in power utilization will cause power outage in the cloud data centers. This paper aims to forecast the VM power consumption with the help of regressive predictive analysis, one of the Machine Learning (ML) techniques. The potency of this approach to make better predictions of future value, using Multi-layer Perceptron (MLP) regressor which provides 91% of accuracy during the prediction process.

Data Center Enhancement by Server Resources Utilization

2021 ◽  
Vol 39 (1B) ◽  
pp. 203-208
Author(s):  
Haider A. Ghanem ◽  
Rana F. Ghani ◽  
Maha J. Abbas
Keyword(s):  
Cloud Computing ◽  
Power Consumption ◽  
Resource Utilization ◽  
Virtual Machine ◽  
Data Center ◽  
Data Centers ◽  
Virtual Machines ◽  
The Internet ◽  
Memory Resources

Data centers are the main nerve of the Internet because of its hosting, storage, cloud computing and other services. All these services require a lot of work and resources, such as energy and cooling. The main problem is how to improve the work of data centers through increased resource utilization by using virtual host simulations and exploiting all server resources. In this paper, we have considered memory resources, where Virtual machines were distributed to hosts after comparing the virtual machines with the host from where the memory and putting the virtual machine on the appropriate host, this will reduce the host machines in the data centers and this will improve the performance of the data centers, in terms of power consumption and the number of servers used and cost.

scholarly journals Accurate Energy and Performance Prediction for Frequency-Scaled GPU Kernels

Computation ◽  
2020 ◽  
Vol 8 (2) ◽  
pp. 37
Author(s):  
Kaijie Fan ◽  
Biagio Cosenza ◽  
Ben Juurlink
Keyword(s):  
Energy Consumption ◽  
Performance Prediction ◽  
High Performance ◽  
Pareto Set ◽  
Large Set ◽  
Balance Performance ◽  
Multi Objective ◽  
Dynamic Voltage ◽  
And Performance ◽  
Performance Computing

Energy optimization is an increasingly important aspect of today’s high-performance computing applications. In particular, dynamic voltage and frequency scaling (DVFS) has become a widely adopted solution to balance performance and energy consumption, and hardware vendors provide management libraries that allow the programmer to change both memory and core frequencies manually to minimize energy consumption while maximizing performance. This article focuses on modeling the energy consumption and speedup of GPU applications while using different frequency configurations. The task is not straightforward, because of the large set of possible and uniformly distributed configurations and because of the multi-objective nature of the problem, which minimizes energy consumption and maximizes performance. This article proposes a machine learning-based method to predict the best core and memory frequency configurations on GPUs for an input OpenCL kernel. The method is based on two models for speedup and normalized energy predictions over the default frequency configuration. Those are later combined into a multi-objective approach that predicts a Pareto-set of frequency configurations. Results show that our approach is very accurate at predicting extema and the Pareto set, and finds frequency configurations that dominate the default configuration in either energy or performance.

Balancing Power And Performance In HPC Clouds

2020 ◽  
Vol 63 (6) ◽  
pp. 880-899
Author(s):  
Lixia Chen ◽  
Jian Li ◽  
Ruhui Ma ◽  
Haibing Guan ◽  
Hans-Arno Jacobsen
Keyword(s):  
Energy Consumption ◽  
High Performance ◽  
Virtual Machines ◽  
Support Vector ◽  
Frequency Scaling ◽  
Vm Placement ◽  
Placement Problem ◽  
Dynamic Voltage ◽  
And Performance ◽  
Save Energy

Abstract With energy consumption in high-performance computing clouds growing rapidly, energy saving has become an important topic. Virtualization provides opportunities to save energy by enabling one physical machine (PM) to host multiple virtual machines (VMs). Dynamic voltage and frequency scaling (DVFS) is another technology to reduce energy consumption. However, in heterogeneous cloud environments where DVFS may be applied at the chip level or the core level, it is a great challenge to combine these two technologies efficiently. On per-core DVFS servers, cloud managers should carefully determine VM placements to minimize performance interference. On full-chip DVFS servers, cloud managers further face the choice of whether to combine VMs with different characteristics to reduce performance interference or to combine VMs with similar characteristics to take better advantage of DVFS. This paper presents a novel mechanism combining a VM placement algorithm and a frequency scaling method. We formulate this VM placement problem as an integer programming (IP) to find appropriate placement configurations, and we utilize support vector machines to select suitable frequencies. We conduct detailed experiments and simulations, showing that our scheme effectively reduces energy consumption with modest impact on performance. Particularly, the total energy delay product is reduced by up to 60%.

scholarly journals Eeadoselfcloud: Energy Efficient Adaptive Depth Optimized Self Cloud Mechanism for VM Migration in Data Centers

2019 ◽  
Vol 8 (6) ◽  
pp. 475-482
Keyword(s):  
Power Consumption ◽  
Data Center ◽  
Data Centers ◽  
Virtual Machines ◽  
Algorithm Analysis ◽  
Virtual Machine Migration ◽  
Hop Count ◽  
Vm Migration ◽  
Bee Colony ◽  
Data Centres

Cloud computing is a paradigm where all resources like software, hardware and information are accessed over internet by using highly sophisticated virtual data centres. The cloud has a data center with a host of many features. Each machine is shared by many users, and virtual machines are used to use these machines. With a large number of data centers and data centers with a large number of physical hosts. Two important issues in cloud environment are Load balancing and power consumption which solved by virtual machine migration. In earlier learnings, Artificial Bee Colony (ABC)'s policy could lead to a compromise between productivity and energy consumption. There are, however, two ways in the ABC-based Abstract based approach: (1) How to find effective solutions across the globe. (2) how to reduce the time to decide to distribute BM.To overcome this issue, this project develop one novel VM migration scheme called eeadoSelfCloud. This proposed method introduces Bee Lion Optimization (BLO) for VM allocation. Data Center Utilization, Average Node Utilization, Request Rejection Ration, Number of Hop Count and Power Consumption are employed as parameters for the proposed algorithm analysis. The experimental results indicate that the proposed algorithm does better than the other available methods.

scholarly journals Randomized routing of virtual machines in IaaS data centers

2019 ◽  
Vol 5 ◽  
pp. e211
Author(s):  
Hadi Khani ◽  
Hamed Khanmirza
Keyword(s):  
Cloud Computing ◽  
Quality Of Service ◽  
Power Consumption ◽  
Analytical Model ◽  
Data Center ◽  
Large Scale ◽  
Data Centers ◽  
Virtual Machines ◽  
General Distribution

Cloud computing technology has been a game changer in recent years. Cloud computing providers promise cost-effective and on-demand resource computing for their users. Cloud computing providers are running the workloads of users as virtual machines (VMs) in a large-scale data center consisting a few thousands physical servers. Cloud data centers face highly dynamic workloads varying over time and many short tasks that demand quick resource management decisions. These data centers are large scale and the behavior of workload is unpredictable. The incoming VM must be assigned onto the proper physical machine (PM) in order to keep a balance between power consumption and quality of service. The scale and agility of cloud computing data centers are unprecedented so the previous approaches are fruitless. We suggest an analytical model for cloud computing data centers when the number of PMs in the data center is large. In particular, we focus on the assignment of VM onto PMs regardless of their current load. For exponential VM arrival with general distribution sojourn time, the mean power consumption is calculated. Then, we show the minimum power consumption under quality of service constraint will be achieved with randomize assignment of incoming VMs onto PMs. Extensive simulation supports the validity of our analytical model.

Resource clustering algorithm for cloud data centers

2021 ◽  
pp. 1-11
Author(s):  
Cail Song ◽  
Bin Liang ◽  
Jiao Li
Keyword(s):  
Energy Consumption ◽  
Data Center ◽  
Clustering Algorithm ◽  
Data Centers ◽  
Virtual Machines ◽  
Good Effect ◽  
Cloud Data Center ◽  
Cloud Data ◽  
Physical Machine ◽  
Cloud Data Centers

Recently, the virtual machine deployment algorithm uses physical machine less or consumes higher energy in data centers, resulting in declined service quality of cloud data centers or rising operational costs, which leads to a decrease in cloud service provider’s earnings finally. According to this situation, a resource clustering algorithm for cloud data centers is proposed. This algorithm systematically analyzes the cloud data center model and physical machine’s use ratio, establishes the dynamic resource clustering rules through k-means clustering algorithm, and deploys the virtual machines based on clustering results, so as to promote the use ratio of physical machine and bring down energy consumption in cloud data centers. The experimental results indicate that, regarding the compute-intensive virtual machines in cloud data centers, compared to contrast algorithm, the physical machine’s use ratio of this algorithm is improved by 12% on average, and its energy consumption in cloud data center is lowered by 15% on average. Regarding the general-purpose virtual machines in cloud data center, compared to contrast algorithm, the physical machine’s use ratio is improved by 14% on average, and its energy consumption in cloud data centers is lowered by 12% on average. Above results demonstrate that this method shows a good effect in the resource management of cloud data centers, which may provide reference to some extent.

Genetic algorithm based cooling energy optimization of data centers

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Jayati Athavale ◽  
Minami Yoda ◽  
Yogendra Joshi
Keyword(s):  
Genetic Algorithm ◽  
Energy Consumption ◽  
Power Consumption ◽  
Data Center ◽  
Load Distribution ◽  
Data Centers ◽  
Cooling Power ◽  
Temperature Prediction ◽  
Content Type ◽  
Optimization Framework

Purpose This study aims to present development of genetic algorithm (GA)-based framework aimed at minimizing data center cooling energy consumption by optimizing the cooling set-points while ensuring that thermal management criteria are satisfied. Design/methodology/approach Three key components of the developed framework include an artificial neural network-based model for rapid temperature prediction (Athavale et al., 2018a, 2019), a thermodynamic model for cooling energy estimation and GA-based optimization process. The static optimization framework informs the IT load distribution and cooling set-points in the data center room to simultaneously minimize cooling power consumption while maximizing IT load. The dynamic framework aims to minimize cooling power consumption in the data center during operation by determining most energy-efficient set-points for the cooling infrastructure while preventing temperature overshoots. Findings Results from static optimization framework indicate that among the three levels (room, rack and row) of IT load distribution granularity, Rack-level distribution consumes the least cooling power. A test case of 7.5 h implementing dynamic optimization demonstrated a reduction in cooling energy consumption between 21%–50% depending on current operation of data center. Research limitations/implications The temperature prediction model used being data-driven, is specific to the lab configuration considered in this study and cannot be directly applied to other scenarios. However, the overall framework can be generalized. Practical implications The developed framework can be implemented in data centers to optimize operation of cooling infrastructure and reduce energy consumption. Originality/value This paper presents a holistic framework for improving energy efficiency of data centers which is of critical value given the high (and increasing) energy consumption by these facilities.

Energy conscious scheduling with controlled threshold for precedence-constrained tasks on heterogeneous clusters

2016 ◽  
Vol 25 (3) ◽  
pp. 276-286 ◽  
Author(s):  
Nirmal Kaur ◽  
Savina Bansal ◽  
Rakesh Kumar Bansal
Keyword(s):  
Energy Consumption ◽  
High Performance Computing ◽  
High Performance ◽  
Scheduling Algorithm ◽  
Voltage Scaling ◽  
Dynamic Voltage Scaling ◽  
Heterogeneous Cluster ◽  
Dynamic Voltage ◽  
Computing Platforms ◽  
Performance Computing

Efficient task scheduling of concurrent tasks is one of the primary requirements for high-performance computing platforms. Recent advances in high-performance computing have resulted in widespread performance improvement though at the cost of increased energy consumption and other system resources. In this article, an energy conscious scheduling algorithm with controlled threshold has been developed for precedence-constrained tasks on heterogeneous cluster, which aims at lower makespan along with reduced energy consumption. Energy conscious scheduling with controlled threshold algorithm combines the benefits of dynamic voltage scaling with controlled threshold-based duplication strategy to achieve its objectives. Effectiveness of the proposed algorithm is analyzed in comparison with available duplication- and non-duplication-based scheduling algorithms (with and without dynamic voltage scaling approach) to ascertain its performance and energy consumption. Exhaustive simulation results on random and real-world graphs demonstrate that energy conscious scheduling algorithm with controlled threshold has the potential to reduce energy consumption and makespan.

Energy-Saving in Optical Data Center Networks

2013 ◽  
Vol 411-414 ◽  
pp. 634-637
Author(s):  
Pei Pei Jiang ◽  
Cun Qian Yu ◽  
Yu Huai Peng
Keyword(s):  
Energy Consumption ◽  
Energy Saving ◽  
Data Center ◽  
Data Centers ◽  
Virtual Machines ◽  
Rapid Expansion ◽  
Optical Data ◽  
Data Center Networks ◽  
Vm Migration ◽  
Server Consolidation

In recent years, with the rapid expansion of network scale and types of applications, cloud computing and virtualization technology have been widely used in the data centers, providing a fast, flexible and convenient service. However, energy efficiency has increased dramatically. The problem of energy consumption has been widespread concern around the world. In this paper, we study the energy-saving in optical data center networks. First, we summarize the traditional methods of energy-saving and meanwhile reveal that the predominant energy consuming resources are the servers installed in the data centers. Then we present the server virtualization technologies based on Virtual Machines (VMs) that have been used widely to reduce energy consumption of servers. Results show server consolidation based on VM migration can efficiently reduce the overall energy consumption compared with traditional energy-saving approaches by reducing energy consumption of the entire network infrastructure in data center networks. For future work, we will study server consolidation based on VM migration in actual environment and address QoS requirements and access latency.

Sign in / Sign up

Export Citation Format

Share Document