scholarly journals Deep Q-network-based traffic signal control models

PLoS ONE ◽  
2021 ◽  
Vol 16 (9) ◽  
pp. e0256405
Author(s):  
Sangmin Park ◽  
Eum Han ◽  
Sungho Park ◽  
Harim Jeong ◽  
Ilsoo Yun
Keyword(s):  
Reinforcement Learning ◽  
Urban Areas ◽  
Control Method ◽  
Fixed Time ◽  
Traffic Signal ◽  
Time Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Models ◽  
Isolated Intersection

Traffic congestion has become common in urban areas worldwide. To solve this problem, the method of searching a solution using artificial intelligence has recently attracted widespread attention because it can solve complex problems such as traffic signal control. This study developed two traffic signal control models using reinforcement learning and a microscopic simulation-based evaluation for an isolated intersection and two coordinated intersections. To develop these models, a deep Q-network (DQN) was used, which is a promising reinforcement learning algorithm. The performance was evaluated by comparing the developed traffic signal control models in this research with the fixed-time signal optimized by Synchro model, which is a traffic signal optimization model. The evaluation showed that the developed traffic signal control model of the isolated intersection was validated, and the coordination of intersections was superior to that of the fixed-time signal control method.

scholarly journals Deep Reinforcement Learning-Based Traffic Signal Control Using High-Resolution Event-Based Data

Entropy ◽  
2019 ◽  
Vol 21 (8) ◽  
pp. 744 ◽  
Author(s):  
Song Wang ◽  
Xu Xie ◽  
Kedi Huang ◽  
Junjie Zeng ◽  
Zimin Cai
Keyword(s):  
Reinforcement Learning ◽  
High Resolution ◽  
Control Strategy ◽  
Control Method ◽  
Control Strategies ◽  
Fixed Time ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Event Based

Reinforcement learning (RL)-based traffic signal control has been proven to have great potential in alleviating traffic congestion. The state definition, which is a key element in RL-based traffic signal control, plays a vital role. However, the data used for state definition in the literature are either coarse or difficult to measure directly using the prevailing detection systems for signal control. This paper proposes a deep reinforcement learning-based traffic signal control method which uses high-resolution event-based data, aiming to achieve cost-effective and efficient adaptive traffic signal control. High-resolution event-based data, which records the time when each vehicle-detector actuation/de-actuation event occurs, is informative and can be collected directly from vehicle-actuated detectors (e.g., inductive loops) with current technologies. Given the event-based data, deep learning techniques are employed to automatically extract useful features for traffic signal control. The proposed method is benchmarked with two commonly used traffic signal control strategies, i.e., the fixed-time control strategy and the actuated control strategy, and experimental results reveal that the proposed method significantly outperforms the commonly used control strategies.

scholarly journals An Edge Based Multi-Agent Auto Communication Method for Traffic Light Control

Sensors ◽  
2020 ◽  
Vol 20 (15) ◽  
pp. 4291 ◽  
Author(s):  
Qiang Wu ◽  
Jianqing Wu ◽  
Jun Shen ◽  
Binbin Yong ◽  
Qingguo Zhou
Keyword(s):  
Reinforcement Learning ◽  
Control Method ◽  
Communication Protocol ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Light Control ◽  
Traffic Light ◽  
Traffic Light Control ◽  
Multi Agent

With smart city infrastructures growing, the Internet of Things (IoT) has been widely used in the intelligent transportation systems (ITS). The traditional adaptive traffic signal control method based on reinforcement learning (RL) has expanded from one intersection to multiple intersections. In this paper, we propose a multi-agent auto communication (MAAC) algorithm, which is an innovative adaptive global traffic light control method based on multi-agent reinforcement learning (MARL) and an auto communication protocol in edge computing architecture. The MAAC algorithm combines multi-agent auto communication protocol with MARL, allowing an agent to communicate the learned strategies with others for achieving global optimization in traffic signal control. In addition, we present a practicable edge computing architecture for industrial deployment on IoT, considering the limitations of the capabilities of network transmission bandwidth. We demonstrate that our algorithm outperforms other methods over 17% in experiments in a real traffic simulation environment.

scholarly journals Self-organisation in traffic signal control algorithms under light traffic conditions

ORiON ◽  
2019 ◽  
Vol 35 (1) ◽  
pp. 57-87
Author(s):  
SJ Movius ◽  
JH Van Vuuren
Keyword(s):  
Simulation Model ◽  
Traffic Simulation ◽  
Control Method ◽  
Fixed Time ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Light Traffic ◽  
Traffic Conditions ◽  
Self Organisation

Fixed-time control and vehicle-actuated control are two distinct types of traffic signal control. The latter control method involves switching traffic signals based on detected traffic flows and thus offers more flexibility (appropriate for lighter traffic conditions) than the former, which relies solely on cyclic, predetermined signal phases that are better suited for heavier traffic conditions. The notion of self-organisation has relatively recently been proposed as an alternative approach towards improving traffic signal control, particularly under light traffic conditions, due to its flexible nature and its potential to result in emergent behaviour. The effectiveness of five existing self-organising traffic signal control strategies from the literature and a fixed-control strategy are compared in this paper within a newly designed agent-based, microscopic traffic simulation model. Various shortcomings of three of these algorithms are identified and algorithmic improvements are suggested to remedy these deficiencies. The relative performance improvements resulting from these algorithmic modifications are then quantified by their implementation in the aforementioned traffic simulation model. Finally, a new self-organising algorithm is proposed that is particularly effective under lighter traffic conditions.

scholarly journals A real-time signal control strategy at an isolated pedestrian crossing based on radar data

2019 ◽  
Vol 11 (3) ◽  
pp. 168781401982590 ◽  
Author(s):  
Xu Qu ◽  
Tangyi Guo ◽  
Jin Guo ◽  
Yi Lin ◽  
Bin Ran
Keyword(s):  
Real Time ◽  
Control Strategy ◽  
Fixed Time ◽  
Radar Data ◽  
Traffic Signal ◽  
Time Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Traffic Delay ◽  
Pedestrian Crossing

Fixed-time traffic signal control strategy in an isolated pedestrian crossing tends to reduce traffic capacity and expose vulnerable road users to more danger. To mitigate the negative impact of previous control strategy, this study proposed an optimal real-time signal timing strategy to protect pedestrian crossing and at the same time minimize the system-wide traffic delay. With the application of a wide-area radar data, the features of vehicles, pedestrians, and the passing time of non-motor vehicles and pedestrian were captured considering conflicts and traffic delay. The support vector machine for regression was utilized to hypothesize traffic delay by training. The discrete values of hypothetical passing time will be tested. The minimum value of delay can be recognized and the corresponding hypothetical passing time will be recommended as the green time for crossing. The performance of the proposed ORSTS outperformed the fixed-time traffic signal control strategy in reducing traffic delay by 22.3%.

scholarly journals Continuous residual reinforcement learning for traffic signal control optimization

2018 ◽  
Vol 45 (8) ◽  
pp. 690-702 ◽  
Author(s):  
Mohammad Aslani ◽  
Stefan Seipel ◽  
Marco Wiering
Keyword(s):  
Reinforcement Learning ◽  
Fixed Time ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Microscopic Traffic Simulation ◽  
Control Optimization ◽  
Large State Space ◽  
Average Travel Time ◽  
Control Traffic

Traffic signal control can be naturally regarded as a reinforcement learning problem. Unfortunately, it is one of the most difficult classes of reinforcement learning problems owing to its large state space. A straightforward approach to address this challenge is to control traffic signals based on continuous reinforcement learning. Although they have been successful in traffic signal control, they may become unstable and fail to converge to near-optimal solutions. We develop adaptive traffic signal controllers based on continuous residual reinforcement learning (CRL-TSC) that is more stable. The effect of three feature functions is empirically investigated in a microscopic traffic simulation. Furthermore, the effects of departing streets, more actions, and the use of the spatial distribution of the vehicles on the performance of CRL-TSCs are assessed. The results show that the best setup of the CRL-TSC leads to saving average travel time by 15% in comparison to an optimized fixed-time controller.

scholarly journals Traffic Signal Control via Reinforcement Learning for Reducing Global Vehicle Emission

2021 ◽  
Vol 13 (20) ◽  
pp. 11254
Author(s):  
Bálint Kővári ◽  
Lászlo Szőke ◽  
Tamás Bécsi ◽  
Szilárd Aradi ◽  
Péter Gáspár
Keyword(s):  
Reinforcement Learning ◽  
Queue Length ◽  
Emission Reduction ◽  
Control Method ◽  
Traffic Signal ◽  
Vehicle Emission ◽  
Signal Control ◽  
Traffic Signal Control ◽  
The Core ◽  
Reward Shaping

The traffic signal control problem is an extensively researched area providing different approaches, from classic methods to machine learning based ones. Different aspects can be considered to find an optima, from which this paper emphasises emission reduction. The core of our solution is a novel rewarding concept for deep reinforcement learning (DRL) which does not utilize any reward shaping, hence exposes new insights into the traffic signal control (TSC) problem. Despite the omission of the standard measures in the rewarding scheme, the proposed approach can outperform a modern actuated control method in classic performance measures such as waiting time and queue length. Moreover, the sustainability of the realized controls is also placed under investigation to evaluate their environmental impacts. Our results show that the proposed solution goes beyond the actuated control not just in the classic measures but in emission-related measures too.

Sign in / Sign up

Export Citation Format

Share Document