Online Scheduling Algorithm for Heterogeneous Distributed Machine Learning Jobs

Author(s):  
Ruiting Zhou ◽  
Jinlong Pang ◽  
Qin Zhang ◽  
Chuan Wu ◽  
Lei Jiao ◽  
...  
2021 ◽  
Vol 51 (4) ◽  
pp. 75-81
Author(s):  
Ahad Mirza Baig ◽  
Alkida Balliu ◽  
Peter Davies ◽  
Michal Dory

Rachid Guerraoui was the rst keynote speaker, and he got things o to a great start by discussing the broad relevance of the research done in our community relative to both industry and academia. He rst argued that, in some sense, the fact that distributed computing is so pervasive nowadays could end up sti ing progress in our community by inducing people to work on marginal problems, and becoming isolated. His rst suggestion was to try to understand and incorporate new ideas coming from applied elds into our research, and argued that this has been historically very successful. He illustrated this point via the distributed payment problem, which appears in the context of blockchains, in particular Bitcoin, but then turned out to be very theoretically interesting; furthermore, the theoretical understanding of the problem inspired new practical protocols. He then went further to discuss new directions in distributed computing, such as the COVID tracing problem, and new challenges in Byzantine-resilient distributed machine learning. Another source of innovation Rachid suggested was hardware innovations, which he illustrated with work studying the impact of RDMA-based primitives on fundamental problems in distributed computing. The talk concluded with a very lively discussion.


2021 ◽  
Author(s):  
Junjie Shi ◽  
Jiang Bian ◽  
Jakob Richter ◽  
Kuan-Hsun Chen ◽  
Jörg Rahnenführer ◽  
...  

AbstractThe predictive performance of a machine learning model highly depends on the corresponding hyper-parameter setting. Hence, hyper-parameter tuning is often indispensable. Normally such tuning requires the dedicated machine learning model to be trained and evaluated on centralized data to obtain a performance estimate. However, in a distributed machine learning scenario, it is not always possible to collect all the data from all nodes due to privacy concerns or storage limitations. Moreover, if data has to be transferred through low bandwidth connections it reduces the time available for tuning. Model-Based Optimization (MBO) is one state-of-the-art method for tuning hyper-parameters but the application on distributed machine learning models or federated learning lacks research. This work proposes a framework $$\textit{MODES}$$ MODES that allows to deploy MBO on resource-constrained distributed embedded systems. Each node trains an individual model based on its local data. The goal is to optimize the combined prediction accuracy. The presented framework offers two optimization modes: (1) $$\textit{MODES}$$ MODES -B considers the whole ensemble as a single black box and optimizes the hyper-parameters of each individual model jointly, and (2) $$\textit{MODES}$$ MODES -I considers all models as clones of the same black box which allows it to efficiently parallelize the optimization in a distributed setting. We evaluate $$\textit{MODES}$$ MODES by conducting experiments on the optimization for the hyper-parameters of a random forest and a multi-layer perceptron. The experimental results demonstrate that, with an improvement in terms of mean accuracy ($$\textit{MODES}$$ MODES -B), run-time efficiency ($$\textit{MODES}$$ MODES -I), and statistical stability for both modes, $$\textit{MODES}$$ MODES outperforms the baseline, i.e., carry out tuning with MBO on each node individually with its local sub-data set.


2020 ◽  
Vol 44 (4) ◽  
Author(s):  
Donya Fozoonmayeh ◽  
Hai Vu Le ◽  
Ekaterina Wittfoth ◽  
Chong Geng ◽  
Natalie Ha ◽  
...  

SIMULATION ◽  
2021 ◽  
pp. 003754972110286
Author(s):  
Eduardo Pérez

Wind turbines experience stochastic loading due to seasonal variations in wind speed and direction. These harsh operational conditions lead to failures of wind turbines, which are difficult to predict. Consequently, it is challenging to schedule maintenance actions that will avoid failures. In this article, a simulation-driven online maintenance scheduling algorithm for wind farm operational planning is derived. Online scheduling is a suitable framework for this problem since it integrates data that evolve over time into the maintenance scheduling decisions. The computational study presented in this article compares the performance of the simulation-driven online scheduling algorithm against two benchmark algorithms commonly used in practice: scheduled maintenance and condition-based monitoring maintenance. An existing discrete event system specification simulation model was used to test and study the benefits of the proposed algorithm. The computational study demonstrates the importance of avoiding over-simplistic assumptions when making maintenance decisions for wind farms. For instance, most literature assumes maintenance lead times are constant. The computational results show that allowing lead times to be adjusted in an online fashion improves the performance of wind farm operations in terms of the number of turbine failures, availability capacity, and power generation.


Sign in / Sign up

Export Citation Format

Share Document