scholarly journals Traffic Signal Optimization for Multiple Intersections Based on Reinforcement Learning

2021 ◽  
Vol 11 (22) ◽  
pp. 10688
Author(s):  
Jaun Gu ◽  
Minhyuck Lee ◽  
Chulmin Jun ◽  
Yohee Han ◽  
Youngchan Kim ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Mathematical Optimization ◽  
Fixed Time ◽  
Traffic Signal ◽  
Real Field ◽  
Urban Mobility ◽  
Signal Optimization ◽  
Time Model ◽  
Proposed Model ◽  
Multiple Intersections

In order to deal with dynamic traffic flow, adaptive traffic signal controls using reinforcement learning are being studied. However, most of the related studies are difficult to apply to the real field considering only mathematical optimization. In this study, we propose a reinforcement learning-based signal optimization model with constraints. The proposed model maintains the sequence of typical signal phases and considers the minimum green time. The model was trained using Simulation of Urban MObility (SUMO), a microscopic traffic simulator. The model was evaluated in the virtual environment similar to a real road with multiple intersections connected. The performance of the proposed model was analyzed by comparing the delay and number of stops with a reinforcement learning model that did not consider constraints and a fixed-time model. In a peak hour, the proposed model reduced the delay from 3 min 15 s to 2 min 15 s and the number of stops from 11 to 4.7 compared to the fixed-time model.

scholarly journals Multi-Agent Reinforcement Learning for Optimizing Traffic Signal Timing

2021 ◽  
Author(s):  
Areej Salaymeh ◽  
Loren Schwiebert ◽  
Stephen Remias
Keyword(s):  
Reinforcement Learning ◽  
Traffic Signals ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Signal Timing ◽  
Model Free ◽  
Proposed Model ◽  
Multi Agent ◽  
Traffic Signal Timing

Designing efficient transportation systems is crucial to save time and money for drivers and for the economy as whole. One of the most important components of traffic systems are traffic signals. Currently, most traffic signal systems are configured using fixed timing plans, which are based on limited vehicle count data. Past research has introduced and designed intelligent traffic signals; however, machine learning and deep learning have only recently been used in systems that aim to optimize the timing of traffic signals in order to reduce travel time. A very promising field in Artificial Intelligence is Reinforcement Learning. Reinforcement learning (RL) is a data driven method that has shown promising results in optimizing traffic signal timing plans to reduce traffic congestion. However, model-based and centralized methods are impractical here due to the high dimensional state-action space in complex urban traffic network. In this paper, a model-free approach is used to optimize signal timing for complicated multiple four-phase signalized intersections. We propose a multi-agent deep reinforcement learning framework that aims to optimize traffic flow using data within traffic signal intersections and data coming from other intersections in a Multi-Agent Environment in what is called Multi-Agent Reinforcement Learning (MARL). The proposed model consists of state-of-art techniques such as Double Deep Q-Network and Hindsight Experience Replay (HER). This research uses HER to allow our framework to quickly learn on sparse reward settings. We tested and evaluated our proposed model via a Simulation of Urban MObility simulation (SUMO). Our results show that the proposed method is effective in reducing congestion in both peak and off-peak times.

scholarly journals Evaluation and Application of Urban Traffic Signal Optimizing Control Strategy Based on Reinforcement Learning

2018 ◽  
Vol 2018 ◽  
pp. 1-9
Author(s):  
Yizhe Wang ◽  
Xiaoguang Yang ◽  
Yangdong Liu ◽  
Hailun Liang
Keyword(s):  
Reinforcement Learning ◽  
Hierarchical Control ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Learning Ability ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Multidimensional Space ◽  
High Flows ◽  
Multiple Intersections

Reinforcement learning method has a self-learning ability in complex multidimensional space because it does not need accurate mathematical model and due to the low requirement for prior knowledge of the environment. The single intersection, arterial lines, and regional road network of a group of multiple intersections are taken as the research object on the paper. Based on the three key parameters of cycle, arterial coordination offset, and green split, a set of hierarchical control algorithms based on reinforcement learning is constructed to optimize and improve the current signal timing scheme. However, the traffic signal optimization strategy based on reinforcement learning is suitable for complex traffic environments (high flows and multiple intersections), and the effects of which are better than the current optimization methods in the conditions of high flows in single intersections, arteries, and regional multi-intersection. In a word, the problem of insufficient traffic signal control capability is studied, and the hierarchical control algorithm based on reinforcement learning is applied to traffic signal control, so as to provide new ideas and methods for traffic signal control theory.

scholarly journals Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning

2020 ◽  
Vol 2020 ◽  
pp. 1-14
Author(s):  
Duowei Li ◽  
Jianping Wu ◽  
Ming Xu ◽  
Ziheng Wang ◽  
Kezhen Hu
Keyword(s):  
Reinforcement Learning ◽  
Waiting Time ◽  
Traffic Signals ◽  
Control Model ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Average Waiting Time ◽  
Adaptive Traffic Signal Control ◽  
Proposed Model

Controlling traffic signals to alleviate increasing traffic pressure is a concept that has received public attention for a long time. However, existing systems and methodologies for controlling traffic signals are insufficient for addressing the problem. To this end, we build a truly adaptive traffic signal control model in a traffic microsimulator, i.e., “Simulation of Urban Mobility” (SUMO), using the technology of modern deep reinforcement learning. The model is proposed based on a deep Q-network algorithm that precisely represents the elements associated with the problem: agents, environments, and actions. The real-time state of traffic, including the number of vehicles and the average speed, at one or more intersections is used as an input to the model. To reduce the average waiting time, the agents provide an optimal traffic signal phase and duration that should be implemented in both single-intersection cases and multi-intersection cases. The co-operation between agents enables the model to achieve an improvement in overall performance in a large road network. By testing with data sets pertaining to three different traffic conditions, we prove that the proposed model is better than other methods (e.g., Q-learning method, longest queue first method, and Webster fixed timing control method) for all cases. The proposed model reduces both the average waiting time and travel time, and it becomes more advantageous as the traffic environment becomes more complex.

scholarly journals Mathematical optimization for planning and design of cycle paths

2016 ◽  
Author(s):  
Roberto José LIÑÁN Ruiz ◽  
Jorge Pérez Aracil ◽  
Víctor Cabrera Cañizares
Keyword(s):  
Programming Model ◽  
Mathematical Optimization ◽  
Urban Mobility ◽  
The Road ◽  
Planning And Design ◽  
Bicycle Model ◽  
Proposed Model ◽  
Bicycle Infrastructure ◽  
Mode Of Transport ◽  
The City

The daily need for citizens to move for different activities, whatever its nature, has been greatly affected by the changes. The advantages resulting from the inclusion of the bicycle as a mode of transport and the proliferation of its use among citizens are numerous and extend both in the field of urban mobility and sustainable development.Currently, there are a number of programs for the implementation, promotion or increased public participation related to cycling in cities. But ultimately, each and every one of these initiatives have the same goal, to create a mesh of effective, useful and cycling trails that allow the use of bicycles in preferred routes with high guarantees of security, incorporating bicycle model intermodal urban transport.With the gradual implementation of bike lanes, many people have begun to use them to get around the city. But everything again needs a period of adaptation, and the reality is that the road network for these vehicles is full of obstacles to the rider. The current situation has led to the proposal that many kilometers of cycle paths needed to supply the demand of this mode of transport and, if implemented and planned are correct and sufficient.This paper presents a mathematical programming model for optimal design of a network for cyclists is presented. Specifically, the model determines a network of bicycle infrastructure, appropriate to the characteristics of a network of existing roads.As an application of the proposed model, the result of these experiments give a number of useful conclusions for planning and designing networks of cycle paths from a social perspective, applied to the case in the city of Malaga.DOI: http://dx.doi.org/10.4995/CIT2016.2016.4089

scholarly journals Continuous residual reinforcement learning for traffic signal control optimization

2018 ◽  
Vol 45 (8) ◽  
pp. 690-702 ◽  
Author(s):  
Mohammad Aslani ◽  
Stefan Seipel ◽  
Marco Wiering
Keyword(s):  
Reinforcement Learning ◽  
Fixed Time ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Microscopic Traffic Simulation ◽  
Control Optimization ◽  
Large State Space ◽  
Average Travel Time ◽  
Control Traffic

Traffic signal control can be naturally regarded as a reinforcement learning problem. Unfortunately, it is one of the most difficult classes of reinforcement learning problems owing to its large state space. A straightforward approach to address this challenge is to control traffic signals based on continuous reinforcement learning. Although they have been successful in traffic signal control, they may become unstable and fail to converge to near-optimal solutions. We develop adaptive traffic signal controllers based on continuous residual reinforcement learning (CRL-TSC) that is more stable. The effect of three feature functions is empirically investigated in a microscopic traffic simulation. Furthermore, the effects of departing streets, more actions, and the use of the spatial distribution of the vehicles on the performance of CRL-TSCs are assessed. The results show that the best setup of the CRL-TSC leads to saving average travel time by 15% in comparison to an optimized fixed-time controller.

scholarly journals Deep Reinforcement Learning-Based Traffic Signal Control Using High-Resolution Event-Based Data

Entropy ◽  
2019 ◽  
Vol 21 (8) ◽  
pp. 744 ◽  
Author(s):  
Song Wang ◽  
Xu Xie ◽  
Kedi Huang ◽  
Junjie Zeng ◽  
Zimin Cai
Keyword(s):  
Reinforcement Learning ◽  
High Resolution ◽  
Control Strategy ◽  
Control Method ◽  
Control Strategies ◽  
Fixed Time ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Event Based

Reinforcement learning (RL)-based traffic signal control has been proven to have great potential in alleviating traffic congestion. The state definition, which is a key element in RL-based traffic signal control, plays a vital role. However, the data used for state definition in the literature are either coarse or difficult to measure directly using the prevailing detection systems for signal control. This paper proposes a deep reinforcement learning-based traffic signal control method which uses high-resolution event-based data, aiming to achieve cost-effective and efficient adaptive traffic signal control. High-resolution event-based data, which records the time when each vehicle-detector actuation/de-actuation event occurs, is informative and can be collected directly from vehicle-actuated detectors (e.g., inductive loops) with current technologies. Given the event-based data, deep learning techniques are employed to automatically extract useful features for traffic signal control. The proposed method is benchmarked with two commonly used traffic signal control strategies, i.e., the fixed-time control strategy and the actuated control strategy, and experimental results reveal that the proposed method significantly outperforms the commonly used control strategies.

Sign in / Sign up

Export Citation Format

Share Document