TAPP: DNN Training for Task Allocation through Pipeline Parallelism Based on Distributed Deep Reinforcement Learning

The rapid development of artificial intelligence technology has made deep neural networks (DNNs) widely used in various fields. DNNs have been continuously growing in order to improve the accuracy and quality of the models. Moreover, traditional data/model parallelism is hard to expand due to communication bottlenecks and hardware efficiency issues. However, pipeline parallelism trains multiple batches, reducing training overheads, so that it can achieve better acceleration effect. Considering the complexity of solving the pipeline parallel task allocation problem in heterogeneous computing resources, in this paper, a task allocation in pipeline parallelism (TAPP) based on deep reinforcement learning, is proposed. In TAPP, the predictive network is trained by a policy gradient until it obtains the optimal pipeline parallel task allocation scheme and speeds up the model training. Experimental results show that, on average, the single-step training time of TAPP is decreased by 1.37 times and the proportion of communication time is reduced by 48.92%, compared with the data parallelism, bulk synchronous parallel (BSP).

Download Full-text

Reinforcement-Learning-Based Asynchronous Formation Control Scheme for Multiple Unmanned Surface Vehicles

Applied Sciences ◽

10.3390/app11020546 ◽

2021 ◽

Vol 11 (2) ◽

pp. 546

Author(s):

Jiajia Xie ◽

Rui Zhou ◽

Yuan Liu ◽

Jun Luo ◽

Shaorong Xie ◽

...

Keyword(s):

Reinforcement Learning ◽

Formation Control ◽

Rapid Development ◽

Gradient Algorithm ◽

Robot System ◽

Physical Relationship ◽

Unmanned Surface Vehicles ◽

Main Challenge ◽

Control Scheme ◽

Multi Robot

The high performance and efficiency of multiple unmanned surface vehicles (multi-USV) promote the further civilian and military applications of coordinated USV. As the basis of multiple USVs’ cooperative work, considerable attention has been spent on developing the decentralized formation control of the USV swarm. Formation control of multiple USV belongs to the geometric problems of a multi-robot system. The main challenge is the way to generate and maintain the formation of a multi-robot system. The rapid development of reinforcement learning provides us with a new solution to deal with these problems. In this paper, we introduce a decentralized structure of the multi-USV system and employ reinforcement learning to deal with the formation control of a multi-USV system in a leader–follower topology. Therefore, we propose an asynchronous decentralized formation control scheme based on reinforcement learning for multiple USVs. First, a simplified USV model is established. Simultaneously, the formation shape model is built to provide formation parameters and to describe the physical relationship between USVs. Second, the advantage deep deterministic policy gradient algorithm (ADDPG) is proposed. Third, formation generation policies and formation maintenance policies based on the ADDPG are proposed to form and maintain the given geometry structure of the team of USVs during movement. Moreover, three new reward functions are designed and utilized to promote policy learning. Finally, various experiments are conducted to validate the performance of the proposed formation control scheme. Simulation results and contrast experiments demonstrate the efficiency and stability of the formation control scheme.

Download Full-text

Response threshold-based task allocation in a reinforcement learning robotic swarm

2014 IEEE 7th International Workshop on Computational Intelligence and Applications (IWCIA) ◽

10.1109/iwcia.2014.6988104 ◽

2014 ◽

Cited By ~ 4

Author(s):

Toshiyuki Yasuda ◽

Koki Kage ◽

Kazuhiro Ohkura

Keyword(s):

Reinforcement Learning ◽

Task Allocation ◽

Response Threshold ◽

Robotic Swarm

Download Full-text

Combining Planning with Reinforcement Learning for Multi-robot Task Allocation

Adaptive Agents and Multi-Agent Systems II - Lecture Notes in Computer Science ◽

10.1007/978-3-540-32274-0_17 ◽

2005 ◽

pp. 260-274 ◽

Cited By ~ 12

Author(s):

Malcolm Strens ◽

Neil Windelinckx

Keyword(s):

Reinforcement Learning ◽

Task Allocation ◽

Robot Task ◽

Multi Robot

Download Full-text

Intelligent Routing Control for MANET Based on Reinforcement Learning

MATEC Web of Conferences ◽

10.1051/matecconf/201823204002 ◽

2018 ◽

Vol 232 ◽

pp. 04002

Author(s):

Fang Dong ◽

Ou Li ◽

Min Tong

Keyword(s):

Reinforcement Learning ◽

Control Algorithm ◽

Rapid Development ◽

Selection Strategy ◽

Performance Simulation ◽

Multiple Parameters ◽

Intelligent Routing ◽

Routing Control ◽

The Cost

With the rapid development and wide use of MANET, the quality of service for various businesses is much higher than before. Aiming at the adaptive routing control with multiple parameters for universal scenes, we propose an intelligent routing control algorithm for MANET based on reinforcement learning, which can constantly optimize the node selection strategy through the interaction with the environment and converge to the optimal transmission paths gradually. There is no need to update the network state frequently, which can save the cost of routing maintenance while improving the transmission performance. Simulation results show that, compared with other algorithms, the proposed approach can choose appropriate paths under constraint conditions, and can obtain better optimization objective.

Download Full-text

Deep-Learning-Based Remaining Useful Life Prediction Based on a Multi-Scale Dilated Convolution Network

Mathematics ◽

10.3390/math9233035 ◽

2021 ◽

Vol 9 (23) ◽

pp. 3035

Author(s):

Feiyue Deng ◽

Yan Bi ◽

Yongqiang Liu ◽

Shaopu Yang

Keyword(s):

Deep Learning ◽

Rapid Development ◽

Operational Efficiency ◽

Remaining Useful Life ◽

Convolution Kernel ◽

Training Time ◽

Multi Scale ◽

Dilated Convolution ◽

Convolution Kernels ◽

Useful Life

Remaining useful life (RUL) prediction of key components is an important influencing factor in making accurate maintenance decisions for mechanical systems. With the rapid development of deep learning (DL) techniques, the research on RUL prediction based on the data-driven model is increasingly widespread. Compared with the conventional convolution neural networks (CNNs), the multi-scale CNNs can extract different-scale feature information, which exhibits a better performance in the RUL prediction. However, the existing multi-scale CNNs employ multiple convolution kernels with different sizes to construct the network framework. There are two main shortcomings of this approach: (1) the convolution operation based on multiple size convolution kernels requires enormous computation and has a low operational efficiency, which severely restricts its application in practical engineering. (2) The convolutional layer with a large size convolution kernel needs a mass of weight parameters, leading to a dramatic increase in the network training time and making it prone to overfitting in the case of small datasets. To address the above issues, a multi-scale dilated convolution network (MsDCN) is proposed for RUL prediction in this article. The MsDCN adopts a new multi-scale dilation convolution fusion unit (MsDCFU), in which the multi-scale network framework is composed of convolution operations with different dilated factors. This effectively expands the range of receptive field (RF) for the convolution kernel without an additional computational burden. Moreover, the MsDCFU employs the depthwise separable convolution (DSC) to further improve the operational efficiency of the prognostics model. Finally, the proposed method was validated with the accelerated degradation test data of rolling element bearings (REBs). The experimental results demonstrate that the proposed MSDCN has a higher RUL prediction accuracy compared to some typical CNNs and better operational efficiency than the existing multi-scale CNNs based on different convolution kernel sizes.

Download Full-text

Energy Optimization Heuristics for Budget-Constrained Workflow in Heterogeneous Computing System

Journal of Circuits System and Computers ◽

10.1142/s0218126619501597 ◽

2019 ◽

Vol 28 (09) ◽

pp. 1950159 ◽

Cited By ~ 3

Author(s):

Junqiang Jiang ◽

Wenbin Li ◽

Li Pan ◽

Bo Yang ◽

Xin Peng

Keyword(s):

Energy Consumption ◽

Heterogeneous Computing ◽

Rapid Development ◽

Energy Optimization ◽

Minimum Cost ◽

Computing System ◽

Directed Acyclic Graphs ◽

Optimization Heuristics ◽

Dynamic Voltage ◽

Heterogeneous Computing System

With the rapid development of commercialized computation, the heterogeneous computing system (HCS) has evolved into a new method of service provisioning based on utility computing models, in which the users consume services and resources based on their quality of service requirements. In certain models using the pay-as-you-go concept, the users are charged for accessed services based on their usage. In addition, the commercialized HCS provider also assumes the responsibility to reduce the energy consumption to protect the environment. This paper considers a basic model known as directed acyclic graphs (DAG), which is designed for workflow applications, and investigates heuristics that allows the scheduling of various tasks of a workflow into the dynamic voltage and frequency scaling enabled HCS. The proposed approaches, which are Minimum-Cost-Up-to-Budget (MCUB) and Maximum-Cost-Down-to-Budget (MCDB), could not only satisfy budget constrains but could also optimize overall energy consumption. The approaches along with their variants are implemented and evaluated using four types of basic DAGs. From the experimental results, we conclude that MCDB outperforms MCUB in energy optimization and makespan criterion while meeting budget constraints faced by users.

Download Full-text