Robust experience replay sampling for multi-agent reinforcement learning

2021 ◽  
Author(s):  
Isack Thomas Nicholaus ◽  
Dae-Ki Kang
Keyword(s):  
Reinforcement Learning ◽  
Experience Replay ◽  
Multi Agent

scholarly journals SMIX(λ): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 7301-7308
Author(s):  
Chao Wen ◽  
Xinghu Yao ◽  
Yuhui Wang ◽  
Xiaoyang Tan
Keyword(s):  
Reinforcement Learning ◽  
Single Agent ◽  
Value Functions ◽  
Type Method ◽  
Agent Learning ◽  
Experience Replay ◽  
Multi Agent ◽  
Decentralized Execution ◽  
Single Agent Learning

This work presents a sample efficient and effective value-based method, named SMIX(λ), for reinforcement learning in multi-agent environments (MARL) within the paradigm of centralized training with decentralized execution (CTDE), in which learning a stable and generalizable centralized value function (CVF) is crucial. To achieve this, our method carefully combines different elements, including 1) removing the unrealistic centralized greedy assumption during the learning phase, 2) using the λ-return to balance the trade-off between bias and variance and to deal with the environment's non-Markovian property, and 3) adopting an experience-replay style off-policy training. Interestingly, it is revealed that there exists inherent connection between SMIX(λ) and previous off-policy Q(λ) approach for single-agent learning. Experiments on the StarCraft Multi-Agent Challenge (SMAC) benchmark show that the proposed SMIX(λ) algorithm outperforms several state-of-the-art MARL methods by a large margin, and that it can be used as a general tool to improve the overall performance of a CTDE-type method by enhancing the evaluation quality of its CVF. We open-source our code at: https://github.com/chaovven/SMIX.

scholarly journals Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13949-13950
Author(s):  
Wang Qisheng ◽  
Wang Qichao ◽  
Li Xiao
Keyword(s):  
Reinforcement Learning ◽  
Supervised Learning ◽  
Experimental Results ◽  
State Action ◽  
Reward Function ◽  
Current State ◽  
Learning Speed ◽  
Communication Method ◽  
Experience Replay ◽  
Multi Agent

Exploration efficiency challenges for multi-agent reinforcement learning (MARL), as the policy learned by confederate MARL depends on the interaction among agents. Less informative reward also restricts the learning speed of MARL in comparison with the informative label in supervised learning. This paper proposes a novel communication method which helps agents focus on different exploration subarea to guide MARL to accelerate exploration. We propose a predictive network to forecast the reward of current state-action pair and use the guidance learned by the predictive network to modify the reward function. An improved prioritized experience replay is employed to help agents better take advantage of the different knowledge learned by different agents. Experimental results demonstrate that the proposed algorithm outperforms existing methods in cooperative multi-agent environments.

Multi-agent deep reinforcement learning with type-based hierarchical group communication

2021 ◽  
Author(s):  
Hao Jiang ◽  
Dianxi Shi ◽  
Chao Xue ◽  
Yajie Wang ◽  
Gongju Wang ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Group Communication ◽  
Multi Agent ◽  
Hierarchical Group

Multi-Agent Deep Reinforcement Learning for Vehicular Computation Offloading in IoT

2020 ◽  
pp. 1-1 ◽  
Author(s):  
Xiaoyu Zhu ◽  
Yueyi Luo ◽  
Anfeng Liu ◽  
Md Zakirul Alam Bhuiyan ◽  
Shaobo Zhang
Keyword(s):  
Reinforcement Learning ◽  
Computation Offloading ◽  
Multi Agent

scholarly journals Multi-Agent Reinforcement Learning: A Review of Challenges and Applications

2021 ◽  
Vol 11 (11) ◽  
pp. 4948
Author(s):  
Lorenzo Canese ◽  
Gian Carlo Cardarilli ◽  
Luca Di Di Nunzio ◽  
Rocco Fazzolari ◽  
Daniele Giardino ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Mathematical Models ◽  
Learning Algorithms ◽  
Single Agent ◽  
Critical Issues ◽  
Multi Agent ◽  
Pros And Cons ◽  
Application Fields

In this review, we present an analysis of the most used multi-agent reinforcement learning algorithms. Starting with the single-agent reinforcement learning algorithms, we focus on the most critical issues that must be taken into account in their extension to multi-agent scenarios. The analyzed algorithms were grouped according to their features. We present a detailed taxonomy of the main multi-agent approaches proposed in the literature, focusing on their related mathematical models. For each algorithm, we describe the possible application fields, while pointing out its pros and cons. The described multi-agent algorithms are compared in terms of the most important characteristics for multi-agent reinforcement learning applications—namely, nonstationarity, scalability, and observability. We also describe the most common benchmark environments used to evaluate the performances of the considered methods.

Sign in / Sign up

Export Citation Format

Share Document