scholarly journals MetaLight: Value-Based Meta-Reinforcement Learning for Traffic Signal Control

2020 ◽  
Vol 34 (01) ◽  
pp. 1153-1160 ◽  
Author(s):  
Xinshi Zang ◽  
Huaxiu Yao ◽  
Guanjie Zheng ◽  
Nan Xu ◽  
Kai Xu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Learning Algorithm ◽  
Traffic Signal ◽  
Training Data ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Individual Level ◽  
Real World Datasets ◽  
Reinforcement Learning Models

Using reinforcement learning for traffic signal control has attracted increasing interests recently. Various value-based reinforcement learning methods have been proposed to deal with this classical transportation problem and achieved better performances compared with traditional transportation methods. However, current reinforcement learning models rely on tremendous training data and computational resources, which may have bad consequences (e.g., traffic jams or accidents) in the real world. In traffic signal control, some algorithms have been proposed to empower quick learning from scratch, but little attention is paid to learning by transferring and reusing learned experience. In this paper, we propose a novel framework, named as MetaLight, to speed up the learning process in new scenarios by leveraging the knowledge learned from existing scenarios. MetaLight is a value-based meta-reinforcement learning workflow based on the representative gradient-based meta-learning algorithm (MAML), which includes periodically alternate individual-level adaptation and global-level adaptation. Moreover, MetaLight improves the-state-of-the-art reinforcement learning model FRAP in traffic signal control by optimizing its model structure and updating paradigm. The experiments on four real-world datasets show that our proposed MetaLight not only adapts more quickly and stably in new traffic scenarios, but also achieves better performance.

Recent Advances in Reinforcement Learning for Traffic Signal Control

2021 ◽  
Vol 22 (2) ◽  
pp. 12-18 ◽  
Author(s):  
Hua Wei ◽  
Guanjie Zheng ◽  
Vikash Gayah ◽  
Zhenhui Li
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Intelligent Transportation Systems ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Methods ◽  
Advantages And Disadvantages ◽  
Recent Advances

Traffic signal control is an important and challenging real-world problem that has recently received a large amount of interest from both transportation and computer science communities. In this survey, we focus on investigating the recent advances in using reinforcement learning (RL) techniques to solve the traffic signal control problem. We classify the known approaches based on the RL techniques they use and provide a review of existing models with analysis on their advantages and disadvantages. Moreover, we give an overview of the simulation environments and experimental settings that have been developed to evaluate the traffic signal control methods. Finally, we explore future directions in the area of RLbased traffic signal control methods. We hope this survey could provide insights to researchers dealing with real-world applications in intelligent transportation systems

scholarly journals Application of Multi-Agent Deep Reinforcement Learning to Optimize Real-World Traffic Signal Controls

2021 ◽  
Author(s):  
Maxim Friesen ◽  
Tian Tan ◽  
Jürgen Jasperneite ◽  
Jie Wang
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Traffic Congestion ◽  
Traffic Signal ◽  
Traffic Model ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Logic ◽  
Rule Based ◽  
Multi Agent

Increasing traffic congestion leads to significant costs associated by additional travel delays, whereby poorly configured signaled intersections are a common bottleneck and root cause. Traditional traffic signal control (TSC) systems employ rule-based or heuristic methods to decide signal timings, while adaptive TSC solutions utilize a traffic-actuated control logic to increase their adaptability to real-time traffic changes. However, such systems are expensive to deploy and are often not flexible enough to adequately adapt to the volatility of today's traffic dynamics. More recently, this problem became a frontier topic in the domain of deep reinforcement learning (DRL) and enabled the development of multi-agent DRL approaches that could operate in environments with several agents present, such as traffic systems with multiple signaled intersections. However, most of these proposed approaches were validated using artificial traffic grids. This paper therefore presents a case study, where real-world traffic data from the town of Lemgo in Germany is used to create a realistic road model within VISSIM. A multi-agent DRL setup, comprising multiple independent deep Q-networks, is applied to the simulated traffic network. Traditional rule-based signal controls, currently employed in the real world at the studied intersections, are integrated in the traffic model with LISA+ and serve as a performance baseline. Our performance evaluation indicates a significant reduction of traffic congestion when using the RL-based signal control policy over the conventional TSC approach in LISA+. Consequently, this paper reinforces the applicability of RL concepts in the domain of TSC engineering by employing a highly realistic traffic model.

scholarly journals Cooperative Traffic Signal Control with Traffic Flow Prediction in Multi-Intersection

Sensors ◽  
2019 ◽  
Vol 20 (1) ◽  
pp. 137 ◽  
Author(s):  
Daeho Kim ◽  
Okran Jeong
Keyword(s):  
Reinforcement Learning ◽  
Traffic Flow ◽  
Learning Algorithm ◽  
Traffic Signal ◽  
Traffic Information ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Traffic Flow Prediction ◽  
Traffic Conditions ◽  
Flow Prediction

As traffic congestion in cities becomes serious, intelligent traffic signal control has been actively studied. Deep Q-Network (DQN), a representative deep reinforcement learning algorithm, is applied to various domains from fully-observable game environment to traffic signal control. Due to the effective performance of DQN, deep reinforcement learning has improved speeds and various DQN extensions have been introduced. However, most traffic signal control researches were performed at a single intersection, and because of the use of virtual simulators, there are limitations that do not take into account variables that affect actual traffic conditions. In this paper, we propose a cooperative traffic signal control with traffic flow prediction (TFP-CTSC) for a multi-intersection. A traffic flow prediction model predicts future traffic state and considers the variables that affect actual traffic conditions. In addition, for cooperative traffic signal control in multi-intersection, each intersection is modeled as an agent, and each agent is trained to take best action by receiving traffic states from the road environment. To deal with multi-intersection efficiently, agents share their traffic information with other adjacent intersections. In the experiment, TFP-CTSC is compared with existing traffic signal control algorithms in a 4 × 4 intersection environment. We verify our traffic flow prediction and cooperative method.

Reinforcement Learning-based Traffic Signal Control under Real-World Constraints

2021 ◽  
Vol 48 (8) ◽  
pp. 871-877
Author(s):  
Mingyu Pi ◽  
Hunsoon Lee ◽  
Moonyoung Chung
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control

A Survey on Reinforcement Learning Models and Algorithms for Traffic Signal Control

2017 ◽  
Vol 50 (3) ◽  
pp. 1-38 ◽  
Author(s):  
Kok-Lim Alvin Yau ◽  
Junaid Qadir ◽  
Hooi Ling Khoo ◽  
Mee Hong Ling ◽  
Peter Komisarczuk
Keyword(s):  
Reinforcement Learning ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Learning Models ◽  
Reinforcement Learning Models

scholarly journals Traffic Signal Control Model on Isolated Intersection Using Reinforcement Learning: A Case Study on Algiers City, Algeria

2021 ◽  
Vol 35 (5) ◽  
pp. 417-424
Author(s):  
Fares Bouriachi ◽  
Hicham Zatla ◽  
Bilal Tolbi ◽  
Koceila Becha ◽  
Allaeddine Ghermoul
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Traffic Light ◽  
Traffic Lights ◽  
On The Road ◽  
Isolated Intersection

Traffic jams and congestion in our cities are a major problem because of the huge increase in the number of cars on the road. To remedy this problem, several control methods are proposed to prevent or reduce traffic congestion based on traffic lights. There are few works using reinforcement learning technique for traffic light control and recent studies have shown promising results. However, existing works have not yet tested the methods on the real-world traffic data and they only focus on studying the rewards without interpreting the policies. In this paper, we proposed a reinforcement learning algorithm to address the traffic signal control problem in real multi-phases isolated intersection. A case study based on Algiers city is conducted the simulation results from the different scenarios show that our proposed scheme reduces the total travel time of the vehicles compared to those obtained with traffic-adaptive control.

Sign in / Sign up

Export Citation Format

Share Document