Integrating SDN-Enhanced MPI with Job Scheduler to Support Shared Clusters

2020 ◽  
pp. 149-159
Author(s):  
Keichi Takahashi ◽  
Susumu Date ◽  
Yasuhiro Watashiba ◽  
Yoshiyuki Kido ◽  
Shinji Shimojo
Keyword(s):  
Job Scheduler

A Reinforcement Learning-Based Mixed Job Scheduler Scheme for Grid or IaaS Cloud

2020 ◽  
Vol 8 (4) ◽  
pp. 1030-1039 ◽  
Author(s):  
Delong Cui ◽  
Zhiping Peng ◽  
Jianbin Xiong ◽  
Bo Xu ◽  
Weiwei Lin
Keyword(s):  
Reinforcement Learning ◽  
Job Scheduler

scholarly journals Location of Processor Allocator and Job Scheduler and Its Impact on CMP Performance

2012 ◽  
Vol 58 (1) ◽  
pp. 9-14 ◽  
Author(s):  
Dawid Zydek ◽  
Grzegorz Chmaj ◽  
Alaa Shawky ◽  
Henry Selvaraj
Keyword(s):  
High Performance ◽  
Chip Multiprocessors ◽  
Energy Savings ◽  
Processor Allocation ◽  
Processing Elements ◽  
Energy Models ◽  
Allocation Process ◽  
On Chip ◽  
Job Scheduler ◽  
Performance Computing

Location of Processor Allocator and Job Scheduler and Its Impact on CMP PerformanceHigh Performance Computing (HPC) architectures are being developed continually with an aim of achieving exascale capability by 2020. Processors that are being developed and used as nodes in HPC systems are Chip Multiprocessors (CMPs) with a number of cores. In this paper, we continue our effort towards a better processor allocation process. The Processor Allocator (PA) and Job Scheduler (JS) proposed and implemented in our previous works are explored in the context of its best location on the chip. We propose a system, where all locations on a chip can be analyzed, considering energy used by Network-on-Chip (NoC), PA and JS, and processing elements. We present energy models for the researched CMP components, mathematical model of the system, and experimentation system. Based on experimental results, proper placement of PA and JS on a chip can provide up to 45% NoC energy savings.

scholarly journals A Web Portal for Rice Crop Improvements

2018 ◽  
Vol 10 (2) ◽  
pp. 15-31 ◽  
Author(s):  
James W. Baurley ◽  
Arif Budiarto ◽  
Muhamad Fitra Kacamarga ◽  
Bens Pardamean
Keyword(s):  
Crop Production ◽  
Rice Crop ◽  
Complex Data ◽  
Web Interface ◽  
Web Portal ◽  
Rice Varieties ◽  
Computationally Intensive ◽  
Job Scheduler ◽  
Sqlite Database ◽  
Server Architecture

High quality models of factors influencing rice crop yield are needed in countries where rice is a staple food. These models can help select optimal rice varieties for expected field conditions. Development of a system to help scientist track and make decisions using this data is challenging. It involves incorporation of complex data structures - genomic, phenotypic, and remote sensing - with computationally intensive statistical modeling. In this article, the authors present a web portal designed to help researchers to manage and analyze their datasets, apply machine learning to detect how factors taken together influence crop production, and summarize the results to help scientists make decisions based on the learned models. The authors developed the system to be easily accessed by the entire team including rice scientist, genetics, and farmers. As such, they developed a system on a server architecture comprised of a SQLite database, a web interface developed in Python, the Celery job scheduler, and statistical computing in R.

Efficient Job Scheduling in Computational Grid Systems Using Wind Driven Optimization Technique

2018 ◽  
Vol 9 (1) ◽  
pp. 49-59
Author(s):  
Tarun Kumar Ghosh ◽  
Sanjoy Das
Keyword(s):  
Job Scheduling ◽  
Scheduling Algorithm ◽  
Optimization Technique ◽  
Computational Grid ◽  
Grid System ◽  
Swarm Optimization ◽  
Grid Systems ◽  
Geographically Distributed ◽  
Job Scheduler ◽  
Computational Resources

Computational Grid has been employed for solving complex and large computation-intensive problems with the help of geographically distributed, heterogeneous and dynamic resources. Job scheduling is a vital and challenging function of a computational Grid system. Job scheduler has to deal with many heterogeneous computational resources and to take decisions concerning the dynamic, efficient and effective execution of jobs. Optimization of the Grid performance is directly related with the efficiency of scheduling algorithm. To evaluate the efficiency of a scheduling algorithm, different parameters can be used, the most important of which are makespan and flowtime. In this paper, a very recent evolutionary heuristic algorithm known as Wind Driven Optimization (WDO) is used for efficiently allocating jobs to resources in a computational Grid system so that makespan and flowtime are minimized. In order to measure the efficacy of WDO, Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) are considered for comparison. This study proves that WDO produces best results.

scholarly journals Log Analysis-Based Resource and Execution Time Improvement in HPC: A Case Study

2020 ◽  
Vol 10 (7) ◽  
pp. 2634
Author(s):  
JunWeon Yoon ◽  
TaeYoung Hong ◽  
ChanYeol Park ◽  
Seo-Young Noh ◽  
HeonChang Yu
Keyword(s):  
Execution Time ◽  
High Performance ◽  
Large Scale ◽  
Experimental Result ◽  
Optimization Approach ◽  
Root Cause ◽  
Large Systems ◽  
Job Scheduler ◽  
Performance Computing

High-performance computing (HPC) uses many distributed computing resources to solve large computational science problems through parallel computation. Such an approach can reduce overall job execution time and increase the capacity of solving large-scale and complex problems. In the supercomputer, the job scheduler, the HPC’s flagship tool, is responsible for distributing and managing the resources of large systems. In this paper, we analyze the execution log of the job scheduler for a certain period of time and propose an optimization approach to reduce the idle time of jobs. In our experiment, it has been found that the main root cause of delayed job is highly related to resource waiting. The execution time of the entire job is affected and significantly delayed due to the increase in idle resources that must be ready when submitting the large-scale job. The backfilling algorithm can optimize the inefficiency of these idle resources and help to reduce the execution time of the job. Therefore, we propose the backfilling algorithm, which can be applied to the supercomputer. This experimental result shows that the overall execution time is reduced.

A new rule-based power-aware job scheduler for supercomputers

2018 ◽  
Vol 74 (6) ◽  
pp. 2508-2527
Author(s):  
Jun Wang ◽  
Dezhi Han ◽  
Ruijun Wang
Keyword(s):  
Rule Based ◽  
Job Scheduler
Sign in / Sign up

Export Citation Format

Share Document