scholarly journals Multi-Agent Reinforcement Learning for Optimizing Traffic Signal Timing

2021 ◽  
Author(s):  
Areej Salaymeh ◽  
Loren Schwiebert ◽  
Stephen Remias
Keyword(s):  
Reinforcement Learning ◽  
Traffic Signals ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Signal Timing ◽  
Model Free ◽  
Proposed Model ◽  
Multi Agent ◽  
Traffic Signal Timing

Designing efficient transportation systems is crucial to save time and money for drivers and for the economy as whole. One of the most important components of traffic systems are traffic signals. Currently, most traffic signal systems are configured using fixed timing plans, which are based on limited vehicle count data. Past research has introduced and designed intelligent traffic signals; however, machine learning and deep learning have only recently been used in systems that aim to optimize the timing of traffic signals in order to reduce travel time. A very promising field in Artificial Intelligence is Reinforcement Learning. Reinforcement learning (RL) is a data driven method that has shown promising results in optimizing traffic signal timing plans to reduce traffic congestion. However, model-based and centralized methods are impractical here due to the high dimensional state-action space in complex urban traffic network. In this paper, a model-free approach is used to optimize signal timing for complicated multiple four-phase signalized intersections. We propose a multi-agent deep reinforcement learning framework that aims to optimize traffic flow using data within traffic signal intersections and data coming from other intersections in a Multi-Agent Environment in what is called Multi-Agent Reinforcement Learning (MARL). The proposed model consists of state-of-art techniques such as Double Deep Q-Network and Hindsight Experience Replay (HER). This research uses HER to allow our framework to quickly learn on sparse reward settings. We tested and evaluated our proposed model via a Simulation of Urban MObility simulation (SUMO). Our results show that the proposed method is effective in reducing congestion in both peak and off-peak times.

scholarly journals Multi-Agent-Based Data-Driven Distributed Adaptive Cooperative Control in Urban Traffic Signal Timing

Energies ◽  
2019 ◽  
Vol 12 (7) ◽  
pp. 1402 ◽  
Author(s):  
Haibo Zhang ◽  
Xiaoming Liu ◽  
Honghai Ji ◽  
Zhongsheng Hou ◽  
Lingling Fan
Keyword(s):  
Cooperative Control ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Data Driven ◽  
Experimental Comparison ◽  
Signal Control ◽  
Signal Timing ◽  
Agent Based ◽  
Multi Agent ◽  
Traffic Signal Timing

Data-driven intelligent transportation systems (D2ITSs) have drawn significant attention lately. This work investigates a novel multi-agent-based data-driven distributed adaptive cooperative control (MA-DD-DACC) method for multi-direction queuing strength balance with changeable cycle in urban traffic signal timing. Compared with the conventional signal control strategies, the proposed MA-DD-DACC method combined with an online parameter learning law can be applied for traffic signal control in a distributed manner by merely utilizing the collected I/O traffic queueing length data and network topology of multi-direction signal controllers at a single intersection. A Lyapunov-based stability analysis shows that the proposed approach guarantees uniform ultimate boundedness of the distributed consensus coordinated errors of queuing strength. The numerical and experimental comparison simulations are performed on a VISSIM-VB-MATLAB joint simulation platform to verify the effectiveness of the proposed approach.

scholarly journals Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

2020 ◽  
Vol 2020 ◽  
pp. 1-14
Author(s):  
Duowei Li ◽  
Jianping Wu ◽  
Ming Xu ◽  
Ziheng Wang ◽  
Kezhen Hu
Keyword(s):  
Reinforcement Learning ◽  
Waiting Time ◽  
Traffic Signals ◽  
Control Model ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Average Waiting Time ◽  
Adaptive Traffic Signal Control ◽  
Proposed Model

Controlling traffic signals to alleviate increasing traffic pressure is a concept that has received public attention for a long time. However, existing systems and methodologies for controlling traffic signals are insufficient for addressing the problem. To this end, we build a truly adaptive traffic signal control model in a traffic microsimulator, i.e., “Simulation of Urban Mobility” (SUMO), using the technology of modern deep reinforcement learning. The model is proposed based on a deep Q-network algorithm that precisely represents the elements associated with the problem: agents, environments, and actions. The real-time state of traffic, including the number of vehicles and the average speed, at one or more intersections is used as an input to the model. To reduce the average waiting time, the agents provide an optimal traffic signal phase and duration that should be implemented in both single-intersection cases and multi-intersection cases. The co-operation between agents enables the model to achieve an improvement in overall performance in a large road network. By testing with data sets pertaining to three different traffic conditions, we prove that the proposed model is better than other methods (e.g., Q-learning method, longest queue first method, and Webster fixed timing control method) for all cases. The proposed model reduces both the average waiting time and travel time, and it becomes more advantageous as the traffic environment becomes more complex.

Influence of Traffic Signal Timing on Red-Light Running and Potential Vehicle Conflicts at Urban Intersections

1997 ◽  
Vol 1595 (1) ◽  
pp. 1-7 ◽  
Author(s):  
Richard A. Retting ◽  
Michael A. Greene
Keyword(s):  
Urban Areas ◽  
Motor Vehicle ◽  
Red Light ◽  
Traffic Signals ◽  
Traffic Signal ◽  
Signal Timing ◽  
Red Light Running ◽  
Sustained Effects ◽  
Traffic Violation ◽  
Traffic Signal Timing

Motor vehicle crashes at traffic signals are a major source of injuries and property damage, especially in urban areas. Many crashes result from vehicles entering the intersection after the onset of a red light, a traffic violation that may be affected by the duration of the change interval (the yellow and all-red periods of the traffic signal). The purpose of this study was to examine short-term and sustained effects on red-light compliance and potential vehicle conflicts as a result of an increase in change intervals to values associated with the Institute of Traffic Engineers (ITE) proposed recommended practice for determining vehicle change intervals. Data were collected during an experiment in an urban location involving changes in signal timing at some 10 intersections. Observations included the proportion of signal cycles with vehicles entering on a red light and the proportion of vehicles exiting the intersection after the onset of a conflicting green signal. Results indicate that change intervals set closer to ITE’s proposed recommended practice can reduce red-light violations and potential right-angle vehicle conflicts and that such safety benefits can be sustained.

Sign in / Sign up

Export Citation Format

Share Document