Dynamic holding control to avoid bus bunching: A multi-agent deep reinforcement learning framework

2020 ◽  
Vol 116 ◽  
pp. 102661 ◽  
Author(s):  
Jiawei Wang ◽  
Lijun Sun
Keyword(s):  
Reinforcement Learning ◽  
Learning Framework ◽  
Bus Bunching ◽  
Multi Agent

scholarly journals Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning

2021 ◽  
Author(s):  
Jiawei Wang ◽  
Lijun Sun
Keyword(s):  
Reinforcement Learning ◽  
Traffic Control ◽  
Inductive Learning ◽  
Traffic Conditions ◽  
Passenger Demand ◽  
Smart Card Data ◽  
Proposed Model ◽  
Bus Bunching ◽  
Multi Agent ◽  
Bus Services

The bus system is a critical component of sustainable urban transportation. However, due to the significant uncertainties in passenger demand and traffic conditions, bus operation is unstable in nature and bus bunching has become a common phenomenon that undermines the reliability and efficiency of bus services. Despite recent advances in multi-agent reinforcement learning (MARL) on traffic control, little research has focused on bus fleet control due to the tricky asynchronous characteristic---control actions only happen when a bus arrives at a bus stop and thus agents do not act simultaneously. In this study, we formulate route-level bus fleet control as an asynchronous multi-agent reinforcement learning (ASMR) problem and extend the classical actor-critic architecture to handle the asynchronous issue. Specifically, we design a novel critic network to effectively approximate the marginal contribution for other agents, in which graph attention neural network is used to conduct inductive learning for policy evaluation. The critic structure also helps the ego agent optimize its policy more efficiently. We evaluate the proposed framework on real-world bus services and actual passenger demand derived from smart card data. Our results show that the proposed model outperforms both traditional headway-based control methods and existing MARL methods.

scholarly journals DTDE: A new cooperative multi-agent reinforcement learning framework

2021 ◽  
pp. 100162
Author(s):  
Guanghui Wen ◽  
Junjie Fu ◽  
Pengcheng Dai ◽  
Jialing Zhou
Keyword(s):  
Reinforcement Learning ◽  
Learning Framework ◽  
Multi Agent

scholarly journals Reinforcement Learning under Threats

2019 ◽  
Vol 33 ◽  
pp. 9939-9940 ◽  
Author(s):  
Victor Gallego ◽  
Roi Naveiro ◽  
David Rios Insua
Keyword(s):  
Reinforcement Learning ◽  
Single Agent ◽  
Potential Threat ◽  
Q Learning ◽  
Learning Framework ◽  
Opponent Modeling ◽  
Theoretical Approaches ◽  
New Learning ◽  
Markov Decision ◽  
Multi Agent

In several reinforcement learning (RL) scenarios, mainly in security settings, there may be adversaries trying to interfere with the reward generating process. However, when non-stationary environments as such are considered, Q-learning leads to suboptimal results (Busoniu, Babuska, and De Schutter 2010). Previous game-theoretical approaches to this problem have focused on modeling the whole multi-agent system as a game. Instead, we shall face the problem of prescribing decisions to a single agent (the supported decision maker, DM) against a potential threat model (the adversary). We augment the MDP to account for this threat, introducing Threatened Markov Decision Processes (TMDPs). Furthermore, we propose a level-k thinking scheme resulting in a new learning framework to deal with TMDPs. We empirically test our framework, showing the benefits of opponent modeling.

Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning

2022 ◽  
pp. 1-20
Author(s):  
D. Xu ◽  
G. Chen
Keyword(s):  
Reinforcement Learning ◽  
Safety Factor ◽  
Cooperative Control ◽  
Learning Framework ◽  
Control Stage ◽  
Autonomous Planning ◽  
Operational Safety ◽  
Policy Gradient ◽  
Multi Agent ◽  
Reward Mechanism

Abstract In this paper, we expolore Multi-Agent Reinforcement Learning (MARL) methods for unmanned aerial vehicle (UAV) cluster. Considering that the current UAV cluster is still in the program control stage, the fully autonomous and intelligent cooperative combat has not been realised. In order to realise the autonomous planning of the UAV cluster according to the changing environment and cooperate with each other to complete the combat goal, we propose a new MARL framework. It adopts the policy of centralised training with decentralised execution, and uses Actor-Critic network to select the execution action and then to make the corresponding evaluation. The new algorithm makes three key improvements on the basis of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The first is to improve learning framework; it makes the calculated Q value more accurate. The second is to add collision avoidance setting, which can increase the operational safety factor. And the third is to adjust reward mechanism; it can effectively improve the cluster’s cooperative ability. Then the improved MADDPG algorithm is tested by performing two conventional combat missions. The simulation results show that the learning efficiency is obviously improved, and the operational safety factor is further increased compared with the previous algorithm.

scholarly journals Multi-Agent Deep Reinforcement Learning for Online 3D Human Poses Estimation

2021 ◽  
Vol 13 (19) ◽  
pp. 3995
Author(s):  
Zhen Fan ◽  
Xiu Li ◽  
Yipeng Li
Keyword(s):  
Reinforcement Learning ◽  
Pose Estimation ◽  
Fixed Number ◽  
Agent Based ◽  
View Selection ◽  
Learning Framework ◽  
Multi Agent ◽  
Identical Number ◽  
Human Pose ◽  
3D Information

Most multi-view based human pose estimation techniques assume the cameras are fixed. While in dynamic scenes, the cameras should be able to move and seek the best views to avoid occlusions and extract 3D information of the target collaboratively. In this paper, we address the problem of online view selection for a fixed number of cameras to estimate multi-person 3D poses actively. The proposed method exploits a distributed multi-agent based deep reinforcement learning framework, where each camera is modeled as an agent, to optimize the action of all the cameras. An inter-agent communication protocol was developed to transfer the cameras’ relative positions between agents for better collaboration. Experiments on the Panoptic dataset show that our method outperforms other view selection methods by a large margin given an identical number of cameras. To the best of our knowledge, our method is the first to address online active multi-view 3D pose estimation with multi-agent reinforcement learning.

Sign in / Sign up

Export Citation Format

Share Document