scholarly journals Message-Dropout: An Efficient Training Method for Multi-Agent Deep Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 6079-6086 ◽  
Author(s):  
Woojun Kim ◽  
Myungsik Cho ◽  
Youngchul Sung
Keyword(s):  
Reinforcement Learning ◽  
Dropout Rate ◽  
Learning Performance ◽  
Multi Agent Systems ◽  
Message Communication ◽  
Execution Phase ◽  
New Learning ◽  
Policy Gradient ◽  
Multi Agent ◽  
Decentralized Execution

In this paper, we propose a new learning technique named message-dropout to improve the performance for multi-agent deep reinforcement learning under two application scenarios: 1) classical multi-agent reinforcement learning with direct message communication among agents and 2) centralized training with decentralized execution. In the first application scenario of multi-agent systems in which direct message communication among agents is allowed, the messagedropout technique drops out the received messages from other agents in a block-wise manner with a certain probability in the training phase and compensates for this effect by multiplying the weights of the dropped-out block units with a correction probability. The applied message-dropout technique effectively handles the increased input dimension in multi-agent reinforcement learning with communication and makes learning robust against communication errors in the execution phase. In the second application scenario of centralized training with decentralized execution, we particularly consider the application of the proposed messagedropout to Multi-Agent Deep Deterministic Policy Gradient (MADDPG), which uses a centralized critic to train a decentralized actor for each agent. We evaluate the proposed message-dropout technique for several games, and numerical results show that the proposed message-dropout technique with proper dropout rate improves the reinforcement learning performance significantly in terms of the training speed and the steady-state performance in the execution phase.

Improvement on Supporting Machine Learning Algorithm for Solving Problem in Immediate Decision Making

2012 ◽  
Vol 566 ◽  
pp. 572-579
Author(s):  
Abdolkarim Niazi ◽  
Norizah Redzuan ◽  
Raja Ishak Raja Hamzah ◽  
Sara Esfandiari
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Multi Agent Systems ◽  
Combined Model ◽  
Q Learning ◽  
Agent Systems ◽  
Multi Agent ◽  
Case Base ◽  
Case Base Reasoning ◽  
Robotic Tool

In this paper, a new algorithm based on case base reasoning and reinforcement learning (RL) is proposed to increase the convergence rate of the reinforcement learning algorithms. RL algorithms are very useful for solving wide variety decision problems when their models are not available and they must make decision correctly in every state of system, such as multi agent systems, artificial control systems, robotic, tool condition monitoring and etc. In the propose method, we investigate how making improved action selection in reinforcement learning (RL) algorithm. In the proposed method, the new combined model using case base reasoning systems and a new optimized function is proposed to select the action, which led to an increase in algorithms based on Q-learning. The algorithm mentioned was used for solving the problem of cooperative Markov’s games as one of the models of Markov based multi-agent systems. The results of experiments Indicated that the proposed algorithms perform better than the existing algorithms in terms of speed and accuracy of reaching the optimal policy.

A Two-Layer Approach to Developing Self-Adaptive Multi-Agent Systems in Open Environment

2014 ◽  
Vol 6 (1) ◽  
pp. 65-85 ◽  
Author(s):  
Xinjun Mao ◽  
Menggao Dong ◽  
Haibin Zhu
Keyword(s):  
Reinforcement Learning ◽  
Adaptive Systems ◽  
Multi Agent Systems ◽  
Uncertain Environments ◽  
Implementation Framework ◽  
Fine Grain ◽  
Adaptation Mechanisms ◽  
Multi Agent ◽  
Self Adaptation ◽  
Self Adaptive

Development of self-adaptive systems situated in open and uncertain environments is a great challenge in the community of software engineering due to the unpredictability of environment changes and the variety of self-adaptation manners. Explicit specification of expected changes and various self-adaptations at design-time, an approach often adopted by developers, seems ineffective. This paper presents an agent-based approach that combines two-layer self-adaptation mechanisms and reinforcement learning together to support the development and running of self-adaptive systems. The approach takes self-adaptive systems as multi-agent organizations and enables the agent itself to make decisions on self-adaptation by learning at run-time and at different levels. The proposed self-adaptation mechanisms that are based on organization metaphors enable self-adaptation at two layers: fine-grain behavior level and coarse-grain organization level. Corresponding reinforcement learning algorithms on self-adaptation are designed and integrated with the two-layer self-adaptation mechanisms. This paper further details developmental technologies, based on the above approach, in establishing self-adaptive systems, including extended software architecture for self-adaptation, an implementation framework, and a development process. A case study and experiment evaluations are conducted to illustrate the effectiveness of the proposed approach.

Sign in / Sign up

Export Citation Format

Share Document