scholarly journals Scheduling Parallel Jobs Using Migration and Consolidation in the Cloud

2012 ◽  
Vol 2012 ◽  
pp. 1-18 ◽  
Author(s):  
Xiaocheng Liu ◽  
Bin Chen ◽  
Xiaogang Qiu ◽  
Ying Cai ◽  
Kedi Huang
Keyword(s):  
Quality Of Service ◽  
High Performance ◽  
Large Scale ◽  
Job Scheduling ◽  
Scheduling Algorithm ◽  
Parallel Applications ◽  
Parallel Job Scheduling ◽  
Parallel Job ◽  
Job Scheduling Problem

An increasing number of high performance computing parallel applications leverages the power of the cloud for parallel processing. How to schedule the parallel applications to improve the quality of service is the key to the successful host of parallel applications in the cloud. The large scale of the cloud makes the parallel job scheduling more complicated as even simple parallel job scheduling problem is NP-complete. In this paper, we propose a parallel job scheduling algorithm named MEASY. MEASY adopts migration and consolidation to enhance the most popular EASY scheduling algorithm. Our extensive experiments on well-known workloads show that our algorithm takes very good care of the quality of service. For two common parallel job scheduling objectives, our algorithm produces an up to 41.1% and an average of 23.1% improvement on the average response time; an up to 82.9% and an average of 69.3% improvement on the average slowdown. Our algorithm is robust even in terms that it allows inaccurate CPU usage estimation and high migration cost. Our approach involves trivial modification on EASY and requires no additional technique; it is practical and effective in the cloud environment.

QoS Oriented Enhancement based on the Analysis of Dynamic Job Scheduling in HPC

2014 ◽  
Vol 6 (4) ◽  
pp. 28-45 ◽  
Author(s):  
Reshmi Raveendran ◽  
D. Shanthi Saravanan
Keyword(s):  
Resource Allocation ◽  
Quality Of Service ◽  
High Performance ◽  
Large Scale ◽  
Job Scheduling ◽  
Dynamic Scheduling ◽  
Future Research ◽  
Dynamic Allocation ◽  
The Impact

With the advent of High Performance Computing (HPC) in the large-scale parallel computational environment, better job scheduling and resource allocation techniques are required to deliver Quality of Service (QoS). Therefore, job scheduling on a large-scale parallel system has been studied to minimize the queue time, response time, and to maximize the overall system utilization. The objective of this paper is to touch upon the recent methods used for dynamic resource allocation across multiple computing nodes and the impact of scheduling algorithms. In addition, a quantitative approach which explains a trend line analysis on dynamic allocation for batch processors is depicted. Throughout the survey, the trends in research on dynamic allocation and parallel computing is identified, besides, highlights the potential areas for future research and development. This study proposes the design for an efficient dynamic scheduling algorithm based on the Quality-of-Service. The analysis provides a compelling research platform to optimize dynamic scheduling of jobs in HPC.

Multi-Objective Scheduling for Parallel Jobs on Grid

2010 ◽  
Vol 439-440 ◽  
pp. 1281-1286 ◽  
Author(s):  
Peng Fei Liu ◽  
Shou Bin Dong
Keyword(s):  
Large Scale ◽  
Job Scheduling ◽  
Scheduling Algorithm ◽  
Turnaround Time ◽  
Experimental Result ◽  
Parallel Job Scheduling ◽  
Multi Objective ◽  
Parallel Jobs ◽  
Parallel Job ◽  
Average Turnaround Time

Focused on the complexity of the parallel job scheduling on heterogeneous Grid, the paper proposes a multi-objective optimization based scheduling algorithm. The algorithm first splits the parallel job up into a series of independent processes with constraints, and then adopts particles to represent the mapping of job-resource. Multi-objective PSO is employed to simultaneously optimize the scheduling objectives of throughput and average turnaround time. Experimental result indicates that the proposed approach is effective while dealing with large scale parallel jobs scheduling on heterogeneous Grid and outperforms other conventional algorithms.

scholarly journals ROA-CONS: Raccoon Optimization for Job Scheduling

Symmetry ◽  
2021 ◽  
Vol 13 (12) ◽  
pp. 2270
Author(s):  
Sina Zangbari Koohi ◽  
Nor Asilah Wati Abdul Hamid ◽  
Mohamed Othman ◽  
Gafurjan Ibragimov
Keyword(s):  
User Satisfaction ◽  
High Performance ◽  
Job Scheduling ◽  
Response Times ◽  
Scheduling Algorithm ◽  
Desktop Computer ◽  
Network Heterogeneity ◽  
Parallel Job Scheduling ◽  
Scheduling Method ◽  
And Performance

High-performance computing comprises thousands of processing powers in order to deliver higher performance computation than a typical desktop computer or workstation in order to solve large problems in science, engineering, or business. The scheduling of these machines has an important impact on their performance. HPC’s job scheduling is intended to develop an operational strategy which utilises resources efficiently and avoids delays. An optimised schedule results in greater efficiency of the parallel machine. In addition, processes and network heterogeneity is another difficulty for the scheduling algorithm. Another problem for parallel job scheduling is user fairness. One of the issues in this field of study is providing a balanced schedule that enhances efficiency and user fairness. ROA-CONS is a new job scheduling method proposed in this paper. It describes a new scheduling approach, which is a combination of an updated conservative backfilling approach further optimised by the raccoon optimisation algorithm. This algorithm also proposes a technique of selection that combines job waiting and response time optimisation with user fairness. It contributes to the development of a symmetrical schedule that increases user satisfaction and performance. In comparison with other well-known job scheduling algorithms, the simulation assesses the effectiveness of the proposed method. The results demonstrate that the proposed strategy offers improved schedules that reduce the overall system’s job waiting and response times.

Job Scheduling Algorithm for Cloud Computing Based on Particle Swarm Optimization

2013 ◽  
Vol 662 ◽  
pp. 957-960 ◽  
Author(s):  
Jing Liu ◽  
Xing Guo Luo ◽  
Xing Ming Zhang ◽  
Fan Zhang
Keyword(s):  
Cloud Computing ◽  
Particle Swarm Optimization ◽  
High Performance ◽  
Large Scale ◽  
Job Scheduling ◽  
Scheduling Algorithm ◽  
Autonomous Systems ◽  
Particle Swarm ◽  
Cost Benefit ◽  
Swarm Optimization

Cloud computing is an emerging high performance computing environment with a large scale, heterogeneous collection of autonomous systems and flexible computational architecture. The performance of the scheduling system influences the cost benefit of this computing paradigm. To reduce the energy consumption and improve the profit, a job scheduling model based on the particle swarm optimization(PSO) algorithm is established for cloud computing. Based on open source cloud computing simulation platform CloudSim, compared to GA and random scheduling algorithms, the results show that the proposed algorithm can obtain a better solution concerning the energy cost and profit.

QoS Oriented Enhancement based on the Analysis of Dynamic Job Scheduling in HPC

2016 ◽  
pp. 1800-1817
Author(s):  
Reshmi Raveendran ◽  
D. Shanthi Saravanan
Keyword(s):  
Resource Allocation ◽  
Quality Of Service ◽  
Large Scale ◽  
Job Scheduling ◽  
Dynamic Scheduling ◽  
Future Research ◽  
Dynamic Allocation ◽  
System Utilization ◽  
Batch Processors

With the advent of High Performance Computing (HPC) in the large-scale parallel computational environment, better job scheduling and resource allocation techniques are required to deliver Quality of Service (QoS). Therefore, job scheduling on a large-scale parallel system has been studied to minimize the queue time, response time, and to maximize the overall system utilization. The objective of this paper is to touch upon the recent methods used for dynamic resource allocation across multiple computing nodes and the impact of scheduling algorithms. In addition, a quantitative approach which explains a trend line analysis on dynamic allocation for batch processors is depicted. Throughout the survey, the trends in research on dynamic allocation and parallel computing is identified, besides, highlights the potential areas for future research and development. This study proposes the design for an efficient dynamic scheduling algorithm based on the Quality-of-Service. The analysis provides a compelling research platform to optimize dynamic scheduling of jobs in HPC.

EFFICIENT PARALLEL JOB SCHEDULING USING GANG SERVICE

2001 ◽  
Vol 12 (03) ◽  
pp. 265-284
Author(s):  
FABRICIO ALVES BARBOSA DA SILVA ◽  
ISAAC D. SCHERSON
Keyword(s):  
Competitive Ratio ◽  
Bin Packing ◽  
Job Scheduling ◽  
Multidimensional Case ◽  
Traditional Concept ◽  
Parallel Job Scheduling ◽  
Gang Scheduling ◽  
Parallel Job ◽  
Job Scheduling Problem ◽  
Simultaneous Scheduling

Gang scheduling has been widely used as a practical solution to the dynamic parallel job scheduling problem. To overcome some of the limitations of traditional Gang scheduling algorithms, Concurrent Gang is proposed as a class of scheduling policies which allows the flexible and simultaneous scheduling of multiple parallel jobs. It hence improves the space sharing characteristics of Gang scheduling while preserving all other advantages. To provide a sound analysis of Concurrent Gang performance, a novel methodology based on the traditional concept of competitive ratio is also introduced. Dubbed dynamic competitive ratio, the new method is used to compare dynamic bin packing algorithms used in this paper. These packing algorithms apply to the Concurrent Gang scheduling of a workload generated by a statistical model. Moreover, dynamic competitive ratio is the figure of merit used to evaluate and compare packing strategies for job scheduling under multiple constraints. It will be shown that for the unidimensional case there is a small difference between the performance of best fit and first fit; first fit can hence be used without significant system degradation. For the multidimensional case, when memory is also considered, we concluded that the packing algorithm must try to balance the resource utilization in all dimensions simulataneously, instead of given priority to only one dimension of the problem.

scholarly journals Statistical and machine learning models for optimizing energy in parallel applications

2019 ◽  
Vol 33 (6) ◽  
pp. 1079-1097 ◽  
Author(s):  
Mark Endrei ◽  
Chao Jin ◽  
Minh Ngoc Dinh ◽  
David Abramson ◽  
Heidi Poxon ◽  
...  
Keyword(s):  
Machine Learning ◽  
Energy Efficiency ◽  
High Performance ◽  
Large Scale ◽  
Energy Use ◽  
Parallel Applications ◽  
Learning Models ◽  
Trade Off ◽  
Time Required ◽  
Machine Learning Models

Rising power costs and constraints are driving a growing focus on the energy efficiency of high performance computing systems. The unique characteristics of a particular system and workload and their effect on performance and energy efficiency are typically difficult for application users to assess and to control. Settings for optimum performance and energy efficiency can also diverge, so we need to identify trade-off options that guide a suitable balance between energy use and performance. We present statistical and machine learning models that only require a small number of runs to make accurate Pareto-optimal trade-off predictions using parameters that users can control. We study model training and validation using several parallel kernels and more complex workloads, including Algebraic Multigrid (AMG), Large-scale Atomic Molecular Massively Parallel Simulator, and Livermore Unstructured Lagrangian Explicit Shock Hydrodynamics. We demonstrate that we can train the models using as few as 12 runs, with prediction error of less than 10%. Our AMG results identify trade-off options that provide up to 45% improvement in energy efficiency for around 10% performance loss. We reduce the sample measurement time required for AMG by 90%, from 13 h to 74 min.

Sign in / Sign up

Export Citation Format

Share Document