Multi-agent Reinforcement Learning for Control Systems: Challenges and Proposals

2015 ◽  
pp. 18-25
Author(s):  
Manuel Graña ◽  
Borja Fernandez-Gauna
Keyword(s):  
Reinforcement Learning ◽  
Control Systems ◽  
Multi Agent

scholarly journals Spatial-Temporal Traffic Flow Control on Motorways Using Distributed Multi-Agent Reinforcement Learning

Mathematics ◽  
2021 ◽  
Vol 9 (23) ◽  
pp. 3081
Author(s):  
Krešimir Kušić ◽  
Edouard Ivanjko ◽  
Filip Vrbanić ◽  
Martin Gregurić ◽  
Ivana Dusparic
Keyword(s):  
Reinforcement Learning ◽  
Control Systems ◽  
Traffic Control ◽  
Adaptive Design ◽  
Control Process ◽  
Leading Edge ◽  
Speed Limit ◽  
Multi Agent ◽  
Local Policies ◽  
The Impact

The prevailing variable speed limit (VSL) systems as an effective strategy for traffic control on motorways have the disadvantage that they only work with static VSL zones. Under changing traffic conditions, VSL systems with static VSL zones may perform suboptimally. Therefore, the adaptive design of VSL zones is required in traffic scenarios where congestion characteristics vary widely over space and time. To address this problem, we propose a novel distributed spatial-temporal multi-agent VSL (DWL-ST-VSL) approach capable of dynamically adjusting the length and position of VSL zones to complement the adjustment of speed limits in current VSL control systems. To model DWL-ST-VSL, distributed W-learning (DWL), a reinforcement learning (RL)-based algorithm for collaborative agent-based self-optimization toward multiple policies, is used. Each agent uses RL to learn local policies, thereby maximizing travel speed and eliminating congestion. In addition to local policies, through the concept of remote policies, agents learn how their actions affect their immediate neighbours and which policy or action is preferred in a given situation. To assess the impact of deploying additional agents in the control loop and the different cooperation levels on the control process, DWL-ST-VSL is evaluated in a four-agent configuration (DWL4-ST-VSL). This evaluation is done via SUMO microscopic simulations using collaborative agents controlling four segments upstream of the congestion in traffic scenarios with medium and high traffic loads. DWL also allows for heterogeneity in agents’ policies; cooperating agents in DWL4-ST-VSL implement two speed limit sets with different granularity. DWL4-ST-VSL outperforms all baselines (W-learning-based VSL and simple proportional speed control), which use static VSL zones. Finally, our experiments yield insights into the new concept of VSL control. This may trigger further research on using advanced learning-based technology to design a new generation of adaptive traffic control systems to meet the requirements of operating in a nonstationary environment and at the leading edge of emerging connected and autonomous vehicles in general.

scholarly journals Cooperative Multi-Agent Reinforcement Learning for Multi-Component Robotic Systems: guidelines for future research

2011 ◽  
Vol 2 (2) ◽  
Author(s):  
Manuel Graña ◽  
Borja Fernandez-Gauna ◽  
Jose Manuel Lopez-Guede
Keyword(s):  
Reinforcement Learning ◽  
Control Systems ◽  
Robot Control ◽  
Robotic Systems ◽  
Future Research ◽  
Multi Agent Systems ◽  
Feedback Information ◽  
Agent Systems ◽  
Innovative Solutions ◽  
Multi Agent

AbstractReinforcement Learning (RL) as a paradigm aims to develop algorithms that allow to train an agent to optimally achieve a goal with minimal feedback information about the desired behavior, which is not precisely specified. Scalar rewards are returned to the agent as response to its actions endorsing or opposing them. RL algorithms have been successfully applied to robot control design. The extension of the RL paradigm to cope with the design of control systems for Multi-Component Robotic Systems (MCRS) poses new challenges, mainly related to coping with scaling up of complexity due to the exponential state space growth, coordination issues, and the propagation of rewards among agents. In this paper, we identify the main issues which offer opportunities to develop innovative solutions towards fully-scalable cooperative multi-agent systems.

Multi-agent deep reinforcement learning with type-based hierarchical group communication

2021 ◽  
Author(s):  
Hao Jiang ◽  
Dianxi Shi ◽  
Chao Xue ◽  
Yajie Wang ◽  
Gongju Wang ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Group Communication ◽  
Multi Agent ◽  
Hierarchical Group

Multi-Agent Deep Reinforcement Learning for Vehicular Computation Offloading in IoT

2020 ◽  
pp. 1-1 ◽  
Author(s):  
Xiaoyu Zhu ◽  
Yueyi Luo ◽  
Anfeng Liu ◽  
Md Zakirul Alam Bhuiyan ◽  
Shaobo Zhang
Keyword(s):  
Reinforcement Learning ◽  
Computation Offloading ◽  
Multi Agent

scholarly journals Multi-Agent Reinforcement Learning: A Review of Challenges and Applications

2021 ◽  
Vol 11 (11) ◽  
pp. 4948
Author(s):  
Lorenzo Canese ◽  
Gian Carlo Cardarilli ◽  
Luca Di Di Nunzio ◽  
Rocco Fazzolari ◽  
Daniele Giardino ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Mathematical Models ◽  
Learning Algorithms ◽  
Single Agent ◽  
Critical Issues ◽  
Multi Agent ◽  
Pros And Cons ◽  
Application Fields

In this review, we present an analysis of the most used multi-agent reinforcement learning algorithms. Starting with the single-agent reinforcement learning algorithms, we focus on the most critical issues that must be taken into account in their extension to multi-agent scenarios. The analyzed algorithms were grouped according to their features. We present a detailed taxonomy of the main multi-agent approaches proposed in the literature, focusing on their related mathematical models. For each algorithm, we describe the possible application fields, while pointing out its pros and cons. The described multi-agent algorithms are compared in terms of the most important characteristics for multi-agent reinforcement learning applications—namely, nonstationarity, scalability, and observability. We also describe the most common benchmark environments used to evaluate the performances of the considered methods.

Sign in / Sign up

Export Citation Format

Share Document