Recent Advances in Reinforcement Learning for Traffic Signal Control

2021 ◽  
Vol 22 (2) ◽  
pp. 12-18 ◽  
Author(s):  
Hua Wei ◽  
Guanjie Zheng ◽  
Vikash Gayah ◽  
Zhenhui Li
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Intelligent Transportation Systems ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Methods ◽  
Advantages And Disadvantages ◽  
Recent Advances

Traffic signal control is an important and challenging real-world problem that has recently received a large amount of interest from both transportation and computer science communities. In this survey, we focus on investigating the recent advances in using reinforcement learning (RL) techniques to solve the traffic signal control problem. We classify the known approaches based on the RL techniques they use and provide a review of existing models with analysis on their advantages and disadvantages. Moreover, we give an overview of the simulation environments and experimental settings that have been developed to evaluate the traffic signal control methods. Finally, we explore future directions in the area of RLbased traffic signal control methods. We hope this survey could provide insights to researchers dealing with real-world applications in intelligent transportation systems

scholarly journals MetaLight: Value-Based Meta-Reinforcement Learning for Traffic Signal Control

2020 ◽  
Vol 34 (01) ◽  
pp. 1153-1160 ◽  
Author(s):  
Xinshi Zang ◽  
Huaxiu Yao ◽  
Guanjie Zheng ◽  
Nan Xu ◽  
Kai Xu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Learning Algorithm ◽  
Traffic Signal ◽  
Training Data ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Individual Level ◽  
Real World Datasets ◽  
Reinforcement Learning Models

Using reinforcement learning for traffic signal control has attracted increasing interests recently. Various value-based reinforcement learning methods have been proposed to deal with this classical transportation problem and achieved better performances compared with traditional transportation methods. However, current reinforcement learning models rely on tremendous training data and computational resources, which may have bad consequences (e.g., traffic jams or accidents) in the real world. In traffic signal control, some algorithms have been proposed to empower quick learning from scratch, but little attention is paid to learning by transferring and reusing learned experience. In this paper, we propose a novel framework, named as MetaLight, to speed up the learning process in new scenarios by leveraging the knowledge learned from existing scenarios. MetaLight is a value-based meta-reinforcement learning workflow based on the representative gradient-based meta-learning algorithm (MAML), which includes periodically alternate individual-level adaptation and global-level adaptation. Moreover, MetaLight improves the-state-of-the-art reinforcement learning model FRAP in traffic signal control by optimizing its model structure and updating paradigm. The experiments on four real-world datasets show that our proposed MetaLight not only adapts more quickly and stably in new traffic scenarios, but also achieves better performance.

KNOWLEDGE BASED TRAFFIC SIGNAL CONTROL MODEL FOR SIGNALIZED INTERSECTION

Transport ◽  
2012 ◽  
Vol 27 (3) ◽  
pp. 263-267 ◽  
Author(s):  
Henrikas Pranevičius ◽  
Tadas Kraujalis
Keyword(s):  
Intelligent Transportation Systems ◽  
Control Method ◽  
Intelligent Transportation ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Signalized Intersection ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Signal Timing ◽  
Traffic Conditions

Intelligent transportation systems have received increasing attention in academy and industry. Being able to handle uncertainties and complexity, expert systems are applied in vast areas of real life including intelligent transportation systems. This paper presents a traffic signal control method based on expert knowledge for an isolated signalized intersection. The proposed method has the adaptive signal timing ability to adjust its signal timing in response to changing traffic conditions. Based on the traffic conditions, the system determines to extend or terminate the current green signal group. Using the information from its traffic detectors of isolated intersection, the proposed controller gives optimal signals to adapt the phase lengths to the traffic conditions. A comparative analysis between proposed control algorithm, fuzzy logic (FLC) and fixed-timed (pre-timed) controllers has been made in traffic flows control, with varying traffic volume levels, by using simulation software ‘Arena’. Simulation results show that the proposed traffic signal control method (EKC) has better performance over fuzzy logic and conventional pre-time controllers under light and heavy traffic conditions.

scholarly journals Application of Multi-Agent Deep Reinforcement Learning to Optimize Real-World Traffic Signal Controls

2021 ◽  
Author(s):  
Maxim Friesen ◽  
Tian Tan ◽  
Jürgen Jasperneite ◽  
Jie Wang
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Traffic Congestion ◽  
Traffic Signal ◽  
Traffic Model ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Logic ◽  
Rule Based ◽  
Multi Agent

Increasing traffic congestion leads to significant costs associated by additional travel delays, whereby poorly configured signaled intersections are a common bottleneck and root cause. Traditional traffic signal control (TSC) systems employ rule-based or heuristic methods to decide signal timings, while adaptive TSC solutions utilize a traffic-actuated control logic to increase their adaptability to real-time traffic changes. However, such systems are expensive to deploy and are often not flexible enough to adequately adapt to the volatility of today's traffic dynamics. More recently, this problem became a frontier topic in the domain of deep reinforcement learning (DRL) and enabled the development of multi-agent DRL approaches that could operate in environments with several agents present, such as traffic systems with multiple signaled intersections. However, most of these proposed approaches were validated using artificial traffic grids. This paper therefore presents a case study, where real-world traffic data from the town of Lemgo in Germany is used to create a realistic road model within VISSIM. A multi-agent DRL setup, comprising multiple independent deep Q-networks, is applied to the simulated traffic network. Traditional rule-based signal controls, currently employed in the real world at the studied intersections, are integrated in the traffic model with LISA+ and serve as a performance baseline. Our performance evaluation indicates a significant reduction of traffic congestion when using the RL-based signal control policy over the conventional TSC approach in LISA+. Consequently, this paper reinforces the applicability of RL concepts in the domain of TSC engineering by employing a highly realistic traffic model.

Reinforcement Learning-based Traffic Signal Control under Real-World Constraints

2021 ◽  
Vol 48 (8) ◽  
pp. 871-877
Author(s):  
Mingyu Pi ◽  
Hunsoon Lee ◽  
Moonyoung Chung
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control

scholarly journals Application of Multi-Agent Deep Reinforcement Learning to Optimize Real-World Traffic Signal Controls

2021 ◽  
Author(s):  
Maxim Friesen ◽  
Tian Tan ◽  
Jürgen Jasperneite ◽  
Jie Wang
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Traffic Congestion ◽  
Traffic Signal ◽  
Traffic Model ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Logic ◽  
Rule Based ◽  
Multi Agent

Increasing traffic congestion leads to significant costs associated by additional travel delays, whereby poorly configured signaled intersections are a common bottleneck and root cause. Traditional traffic signal control (TSC) systems employ rule-based or heuristic methods to decide signal timings, while adaptive TSC solutions utilize a traffic-actuated control logic to increase their adaptability to real-time traffic changes. However, such systems are expensive to deploy and are often not flexible enough to adequately adapt to the volatility of today's traffic dynamics. More recently, this problem became a frontier topic in the domain of deep reinforcement learning (DRL) and enabled the development of multi-agent DRL approaches that could operate in environments with several agents present, such as traffic systems with multiple signaled intersections. However, most of these proposed approaches were validated using artificial traffic grids. This paper therefore presents a case study, where real-world traffic data from the town of Lemgo in Germany is used to create a realistic road model within VISSIM. A multi-agent DRL setup, comprising multiple independent deep Q-networks, is applied to the simulated traffic network. Traditional rule-based signal controls, currently employed in the real world at the studied intersections, are integrated in the traffic model with LISA+ and serve as a performance baseline. Our performance evaluation indicates a significant reduction of traffic congestion when using the RL-based signal control policy over the conventional TSC approach in LISA+. Consequently, this paper reinforces the applicability of RL concepts in the domain of TSC engineering by employing a highly realistic traffic model.

Sign in / Sign up

Export Citation Format

Share Document