A MULTI-AGENT REINFORCEMENT LEARNING FRAMEWORK FOR INTELLIGENT MANUFACTURING WITH AUTONOMOUS MOBILE ROBOTS

AbstractIntelligent manufacturing (IM) embraces Industry 4.0 design principles to advance autonomy and increase manufacturing efficiency. However, many IM systems are created ad hoc, which limits the potential for generalizable design principles and operational guidelines. This work offers a standardizing framework for integrated job scheduling and navigation control in an autonomous mobile robot driven shop floor, an increasingly common IM paradigm. We specifically propose a multi-agent framework involving mobile robots, machines, humans. Like any cyberphysical system, the performance of IM systems is influenced by the construction of the underlying software platforms and the choice of the constituent algorithms. In this work, we demonstrate the use of reinforcement learning on a sub-system of the proposed framework and test its effectiveness in a dynamic scenario. The case study demonstrates collaboration amongst robots to maximize throughput and safety on the shop floor. Moreover, we observe nuanced behavior, including the ability to autonomously compensate for processing delays, and machine and robot failures in real time.

Download Full-text

Building autonomic systems using collaborative reinforcement learning

The Knowledge Engineering Review ◽

10.1017/s0269888906000956 ◽

2006 ◽

Vol 21 (3) ◽

pp. 231-238 ◽

Cited By ~ 12

Author(s):

JIM DOWLING ◽

RAYMOND CUNNINGHAM ◽

EOIN CURRAN ◽

VINNY CAHILL

Keyword(s):

Reinforcement Learning ◽

Ad Hoc ◽

Optimization Problems ◽

System Optimization ◽

Multi Agent Systems ◽

Learning Agents ◽

Coordination Model ◽

Routing Performance ◽

Multi Agent ◽

Unpredictable Environment

This paper presents Collaborative Reinforcement Learning (CRL), a coordination model for online system optimization in decentralized multi-agent systems. In CRL system optimization problems are represented as a set of discrete optimization problems, each of whose solution cost is minimized by model-based reinforcement learning agents collaborating on their solution. CRL systems can be built to provide autonomic behaviours such as optimizing system performance in an unpredictable environment and adaptation to partial failures. We evaluate CRL using an ad hoc routing protocol that optimizes system routing performance in an unpredictable network environment.

Download Full-text

A novel multi-agent reinforcement learning approach for job scheduling in Grid computing

Future Generation Computer Systems ◽

10.1016/j.future.2010.10.009 ◽

2011 ◽

Vol 27 (5) ◽

pp. 430-439 ◽

Cited By ~ 51

Author(s):

Jun Wu ◽

Xin Xu ◽

Pengcheng Zhang ◽

Chunming Liu

Keyword(s):

Reinforcement Learning ◽

Grid Computing ◽

Job Scheduling ◽

Learning Approach ◽

Multi Agent

Download Full-text

Multi-Agent Machine Learning for Mobile Robots: A Reinforcement Learning Approach

Proceedings of the 5th International Conference of Control, Dynamic Systems, and Robotics (CDSR'19) ◽

10.11159/cdsr19.3 ◽

2019 ◽

Author(s):

Howard Schwarz

Keyword(s):

Machine Learning ◽

Reinforcement Learning ◽

Mobile Robots ◽

Learning Approach ◽

Multi Agent

Download Full-text

A centralized reinforcement learning method for multi-agent job scheduling in Grid

2016 6th International Conference on Computer and Knowledge Engineering (ICCKE) ◽

10.1109/iccke.2016.7802135 ◽

2016 ◽

Cited By ~ 5

Author(s):

Milad Moradi

Keyword(s):

Reinforcement Learning ◽

Job Scheduling ◽

Learning Method ◽

Multi Agent

Download Full-text

Construction of multi-agent mobile robots control system in the problem of persecution with using a modified reinforcement learning method based on neural networks

IOP Conference Series Materials Science and Engineering ◽

10.1088/1757-899x/312/1/012018 ◽

2018 ◽

Vol 312 ◽

pp. 012018

Author(s):

M L Patkin ◽

G N Rogachev

Keyword(s):

Neural Networks ◽

Control System ◽

Reinforcement Learning ◽

Mobile Robots ◽

Learning Method ◽

Multi Agent

Download Full-text

Real-time order dispatching for a fleet of autonomous mobile robots using multi-agent reinforcement learning

CIRP Annals ◽

10.1016/j.cirp.2020.04.001 ◽

2020 ◽

Vol 69 (1) ◽

pp. 397-400

Author(s):

Andreja Malus ◽

Dominik Kozjek ◽

Rok Vrabič

Keyword(s):

Reinforcement Learning ◽

Mobile Robots ◽

Real Time ◽

Autonomous Mobile Robots ◽

Time Order ◽

Multi Agent

Download Full-text

Using statistical network link model for routing in ad hoc networks with multi-agent reinforcement learning

2010 2nd International Conference on Advanced Computer Control ◽

10.1109/icacc.2010.5486814 ◽

2010 ◽

Cited By ~ 2

Author(s):

Zhang Binbin ◽

Liu Quan ◽

Zhao Shouling

Keyword(s):

Reinforcement Learning ◽

Ad Hoc Networks ◽

Ad Hoc ◽

Network Link ◽

Multi Agent ◽

Link Model ◽

Hoc Networks

Download Full-text

Multi-Agent Reinforcement Learning Using Linear Fuzzy Model Applied to Cooperative Mobile Robots

Symmetry ◽

10.3390/sym10100461 ◽

2018 ◽

Vol 10 (10) ◽

pp. 461 ◽

Cited By ~ 7

Author(s):

David Luviano-Cruz ◽

Francesco Garcia-Luna ◽

Luis Pérez-Domínguez ◽

S. Gadi

Keyword(s):

Reinforcement Learning ◽

Mobile Robots ◽

State Space ◽

Fuzzy Model ◽

Discrete State ◽

Continuous State Space ◽

Continuous State ◽

Multi Agent ◽

Successful Approach ◽

Q Function

A multi-agent system (MAS) is suitable for addressing tasks in a variety of domains without any programmed behaviors, which makes it ideal for the problems associated with the mobile robots. Reinforcement learning (RL) is a successful approach used in the MASs to acquire new behaviors; most of these select exact Q-values in small discrete state space and action space. This article presents a joint Q-function linearly fuzzified for a MAS’ continuous state space, which overcomes the dimensionality problem. Also, this article gives a proof for the convergence and existence of the solution proposed by the algorithm presented. This article also discusses the numerical simulations and experimental results that were carried out to validate the proposed algorithm.

Download Full-text