scholarly journals Application of Multi-Agent Deep Reinforcement Learning to Optimize Real-World Traffic Signal Controls

2021 ◽  
Author(s):  
Maxim Friesen ◽  
Tian Tan ◽  
Jürgen Jasperneite ◽  
Jie Wang
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Traffic Congestion ◽  
Traffic Signal ◽  
Traffic Model ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Logic ◽  
Rule Based ◽  
Multi Agent

Increasing traffic congestion leads to significant costs associated by additional travel delays, whereby poorly configured signaled intersections are a common bottleneck and root cause. Traditional traffic signal control (TSC) systems employ rule-based or heuristic methods to decide signal timings, while adaptive TSC solutions utilize a traffic-actuated control logic to increase their adaptability to real-time traffic changes. However, such systems are expensive to deploy and are often not flexible enough to adequately adapt to the volatility of today's traffic dynamics. More recently, this problem became a frontier topic in the domain of deep reinforcement learning (DRL) and enabled the development of multi-agent DRL approaches that could operate in environments with several agents present, such as traffic systems with multiple signaled intersections. However, most of these proposed approaches were validated using artificial traffic grids. This paper therefore presents a case study, where real-world traffic data from the town of Lemgo in Germany is used to create a realistic road model within VISSIM. A multi-agent DRL setup, comprising multiple independent deep Q-networks, is applied to the simulated traffic network. Traditional rule-based signal controls, currently employed in the real world at the studied intersections, are integrated in the traffic model with LISA+ and serve as a performance baseline. Our performance evaluation indicates a significant reduction of traffic congestion when using the RL-based signal control policy over the conventional TSC approach in LISA+. Consequently, this paper reinforces the applicability of RL concepts in the domain of TSC engineering by employing a highly realistic traffic model.

scholarly journals Application of Multi-Agent Deep Reinforcement Learning to Optimize Real-World Traffic Signal Controls

2021 ◽  
Author(s):  
Maxim Friesen ◽  
Tian Tan ◽  
Jürgen Jasperneite ◽  
Jie Wang
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Traffic Congestion ◽  
Traffic Signal ◽  
Traffic Model ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Logic ◽  
Rule Based ◽  
Multi Agent

Increasing traffic congestion leads to significant costs associated by additional travel delays, whereby poorly configured signaled intersections are a common bottleneck and root cause. Traditional traffic signal control (TSC) systems employ rule-based or heuristic methods to decide signal timings, while adaptive TSC solutions utilize a traffic-actuated control logic to increase their adaptability to real-time traffic changes. However, such systems are expensive to deploy and are often not flexible enough to adequately adapt to the volatility of today's traffic dynamics. More recently, this problem became a frontier topic in the domain of deep reinforcement learning (DRL) and enabled the development of multi-agent DRL approaches that could operate in environments with several agents present, such as traffic systems with multiple signaled intersections. However, most of these proposed approaches were validated using artificial traffic grids. This paper therefore presents a case study, where real-world traffic data from the town of Lemgo in Germany is used to create a realistic road model within VISSIM. A multi-agent DRL setup, comprising multiple independent deep Q-networks, is applied to the simulated traffic network. Traditional rule-based signal controls, currently employed in the real world at the studied intersections, are integrated in the traffic model with LISA+ and serve as a performance baseline. Our performance evaluation indicates a significant reduction of traffic congestion when using the RL-based signal control policy over the conventional TSC approach in LISA+. Consequently, this paper reinforces the applicability of RL concepts in the domain of TSC engineering by employing a highly realistic traffic model.

Recent Advances in Reinforcement Learning for Traffic Signal Control

2021 ◽  
Vol 22 (2) ◽  
pp. 12-18 ◽  
Author(s):  
Hua Wei ◽  
Guanjie Zheng ◽  
Vikash Gayah ◽  
Zhenhui Li
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Intelligent Transportation Systems ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Methods ◽  
Advantages And Disadvantages ◽  
Recent Advances

Traffic signal control is an important and challenging real-world problem that has recently received a large amount of interest from both transportation and computer science communities. In this survey, we focus on investigating the recent advances in using reinforcement learning (RL) techniques to solve the traffic signal control problem. We classify the known approaches based on the RL techniques they use and provide a review of existing models with analysis on their advantages and disadvantages. Moreover, we give an overview of the simulation environments and experimental settings that have been developed to evaluate the traffic signal control methods. Finally, we explore future directions in the area of RLbased traffic signal control methods. We hope this survey could provide insights to researchers dealing with real-world applications in intelligent transportation systems

scholarly journals An Edge Based Multi-Agent Auto Communication Method for Traffic Light Control

Sensors ◽  
2020 ◽  
Vol 20 (15) ◽  
pp. 4291 ◽  
Author(s):  
Qiang Wu ◽  
Jianqing Wu ◽  
Jun Shen ◽  
Binbin Yong ◽  
Qingguo Zhou
Keyword(s):  
Reinforcement Learning ◽  
Control Method ◽  
Communication Protocol ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Light Control ◽  
Traffic Light ◽  
Traffic Light Control ◽  
Multi Agent

With smart city infrastructures growing, the Internet of Things (IoT) has been widely used in the intelligent transportation systems (ITS). The traditional adaptive traffic signal control method based on reinforcement learning (RL) has expanded from one intersection to multiple intersections. In this paper, we propose a multi-agent auto communication (MAAC) algorithm, which is an innovative adaptive global traffic light control method based on multi-agent reinforcement learning (MARL) and an auto communication protocol in edge computing architecture. The MAAC algorithm combines multi-agent auto communication protocol with MARL, allowing an agent to communicate the learned strategies with others for achieving global optimization in traffic signal control. In addition, we present a practicable edge computing architecture for industrial deployment on IoT, considering the limitations of the capabilities of network transmission bandwidth. We demonstrate that our algorithm outperforms other methods over 17% in experiments in a real traffic simulation environment.

scholarly journals MetaLight: Value-Based Meta-Reinforcement Learning for Traffic Signal Control

2020 ◽  
Vol 34 (01) ◽  
pp. 1153-1160 ◽  
Author(s):  
Xinshi Zang ◽  
Huaxiu Yao ◽  
Guanjie Zheng ◽  
Nan Xu ◽  
Kai Xu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Learning Algorithm ◽  
Traffic Signal ◽  
Training Data ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Individual Level ◽  
Real World Datasets ◽  
Reinforcement Learning Models

Using reinforcement learning for traffic signal control has attracted increasing interests recently. Various value-based reinforcement learning methods have been proposed to deal with this classical transportation problem and achieved better performances compared with traditional transportation methods. However, current reinforcement learning models rely on tremendous training data and computational resources, which may have bad consequences (e.g., traffic jams or accidents) in the real world. In traffic signal control, some algorithms have been proposed to empower quick learning from scratch, but little attention is paid to learning by transferring and reusing learned experience. In this paper, we propose a novel framework, named as MetaLight, to speed up the learning process in new scenarios by leveraging the knowledge learned from existing scenarios. MetaLight is a value-based meta-reinforcement learning workflow based on the representative gradient-based meta-learning algorithm (MAML), which includes periodically alternate individual-level adaptation and global-level adaptation. Moreover, MetaLight improves the-state-of-the-art reinforcement learning model FRAP in traffic signal control by optimizing its model structure and updating paradigm. The experiments on four real-world datasets show that our proposed MetaLight not only adapts more quickly and stably in new traffic scenarios, but also achieves better performance.

Sign in / Sign up

Export Citation Format

Share Document