Inference-based Hierarchical Reinforcement Learning for Cooperative Multi-agent Navigation

2021 ◽  
Author(s):  
Lijun Xia ◽  
Chao Yu ◽  
Zifan Wu
Keyword(s):  
Reinforcement Learning ◽  
Hierarchical Reinforcement Learning ◽  
Multi Agent ◽  
Agent Navigation

Hierarchical Reinforcement Learning

2021 ◽  
Vol 54 (5) ◽  
pp. 1-35
Author(s):  
Shubham Pateria ◽  
Budhitama Subagdja ◽  
Ah-hwee Tan ◽  
Chai Quek
Keyword(s):  
Reinforcement Learning ◽  
Future Research ◽  
Comprehensive Overview ◽  
Open Problems ◽  
Practical Applications ◽  
Hierarchical Reinforcement Learning ◽  
The Past ◽  
Agent Learning ◽  
Multi Agent ◽  
Supplementary Material

Hierarchical Reinforcement Learning (HRL) enables autonomous decomposition of challenging long-horizon decision-making tasks into simpler subtasks. During the past years, the landscape of HRL research has grown profoundly, resulting in copious approaches. A comprehensive overview of this vast landscape is necessary to study HRL in an organized manner. We provide a survey of the diverse HRL approaches concerning the challenges of learning hierarchical policies, subtask discovery, transfer learning, and multi-agent learning using HRL. The survey is presented according to a novel taxonomy of the approaches. Based on the survey, a set of important open problems is proposed to motivate the future research in HRL. Furthermore, we outline a few suitable task domains for evaluating the HRL approaches and a few interesting examples of the practical applications of HRL in the Supplementary Material.

ANALYTICAL REVIEW OF MULTI-AGENT REINFORCEMENT LEARNING PROBLEMS

2020 ◽  
pp. 48-56
Author(s):  
Yu. V. Dubenko
Keyword(s):  
Reinforcement Learning ◽  
Intelligent Agents ◽  
Russian Language ◽  
Learning Problems ◽  
Multi Agent Systems ◽  
Hierarchical Reinforcement Learning ◽  
Collective Interaction ◽  
Analytical Review ◽  
Multi Agent ◽  
Partially Observable

This paper is devoted to the problem of collective artificial intelligence in solving problems by intelligent agents in external environments. The environments may be: fully or partially observable, deterministic or stochastic, static or dynamic, discrete or continuous. The paper identifies problems of collective interaction of intelligent agents when they solve a class of tasks, which need to coordinate actions of agent group, e. g. task of exploring the territory of a complex infrastructure facility. It is revealed that the problem of reinforcement training in multi-agent systems is poorly presented in the press, especially in Russian-language publications. The article analyzes reinforcement learning, describes hierarchical reinforcement learning, presents basic methods to implement reinforcement learning. The concept of macro-action by agents integrated in groups is introduced. The main problems of intelligent agents collective interaction for problem solving (i. e. calculation of individual rewards for each agent; agent coordination issues; application of macro actions by agents integrated into groups; exchange of experience generated by various agents as part of solving a collective problem) are identified. The model of multi-agent reinforcement learning is described in details. The article describes problems of this approach building on existing solutions. Basic problems of multi-agent reinforcement learning are formulated in conclusion.

scholarly journals SYSTEM ANALYSIS IN HIERARCHICAL INTELLIGENT MULTI-AGENT SYSTEMS

2021 ◽  
pp. 33-46
Author(s):  
V. S. Simankov ◽  
Yu. V. Dubenko
Keyword(s):  
Reinforcement Learning ◽  
System Analysis ◽  
Three Dimensional ◽  
Label Propagation ◽  
Multi Agent System ◽  
Multi Agent Systems ◽  
Agent Systems ◽  
Hierarchical Reinforcement Learning ◽  
Multi Agent ◽  
Weight Coefficients

The system analysis of the hierarchical intelligent multi-agent system in general, as well as its main structural unit, the intelligent agent, its major subsystems identified. As part of the analysis of the computer vision subsystem, it was concluded that the considered sources have insufficiently worked out issues related to the processing of occlusions, with the automation of the process of reconstruction of three-dimensional scenes, with the implementation of the processing of an unstructured set of images. The structure of the block for the reconstruction of three-dimensional scenes is proposed, the implementation of which is aimed at eliminating the indicated problems characteristic of the machine vision subsystem. The analysis of the main methods of implementing unsupervised learning is carried out, based on the results of which it is concluded that it is advisable to use reinforcement learning when implementing systems of this type. Such types of reinforcement learning as hierarchical reinforcement learning and multi-agent reinforcement learning are considered. A method for segmentation of macro actions is proposed, based on the implementation of clustering by the method of label propagation, in which the number of transitions is formalized in the form of weight coefficients of edges.

Sign in / Sign up

Export Citation Format

Share Document