scholarly journals Trajectory Optimization of CAVs in Freeway Work Zone considering Car-Following Behaviors Using Online Multiagent Reinforcement Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Tong Zhu ◽  
Xiaohu Li ◽  
Wei Fan ◽  
Changshuai Wang ◽  
Haoxue Liu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Trajectory Optimization ◽  
Autonomous Vehicle ◽  
Work Zone ◽  
Operating Efficiency ◽  
Lane Changing ◽  
Car Following ◽  
Advisory Service ◽  
Online Computation ◽  
Multiagent Reinforcement Learning

Work zone areas are frequent congested sections considered as the freeway bottleneck. Connected and autonomous vehicle (CAV) trajectory optimization can improve the operating efficiency in bottleneck areas by harmonizing vehicles’ manipulations. This study presents a joint trajectory optimization of cooperative lane changing, merging, and car-following actions for CAV control at a local merging point together with upstream points. The multiagent reinforcement learning (MARL) method is applied in this system, with one agent providing a merging advisory service at the merging point and controlling the inner-lane vehicles’ headway for smooth outer-lane vehicle merging, while other agents provide lane-changing advisory services at advance lane-changing points to control how vehicles make lane changes in advance and perform corresponding headway adjustment, similar to and jointly with the merging advisory service. Uniting all agents, the coordination graph (CG) method is applied to seek the global optimum, overcoming the exponential growth problem in MARL. Using MATLAB and the VISSIM COM interface, an online simulation platform is established. The simulation results show that MARL is effective for online computation with in-timing response. More importantly, comparisons of the results obtained in various scenarios demonstrate that the proposed system obtained smoother vehicle trajectories in all controlled sections, rather than only in the merging area, indicating that it can achieve better traffic conditions in freeway work zone areas.

Development of Operation Guidelines for Leader–Follower Autonomous Maintenance Vehicles at Work Zone Locations

2021 ◽  
pp. 036119812110566
Author(s):  
Qing Tang ◽  
Xianbiao Hu ◽  
Ruwen Qin
Keyword(s):  
Decision Making ◽  
Autonomous Vehicle ◽  
Field Testing ◽  
Work Zone ◽  
Lane Changing ◽  
The Public ◽  
Car Following ◽  
Technical Requirements ◽  
Car Following Model ◽  
Niche Area

The rapid advancement of connected and autonomous vehicle (CAV) technologies, although possibly years away from wide application to the general public travel, are receiving attention from many state Departments of Transportation (DOT) in the niche area of using autonomous maintenance technology (AMT) to reduce fatalities of DOT workers in work zone locations. Although promising results are shown in testing and deployments in several states, current autonomous truck mounted attenuator (ATMA) system operators are not provided with much practical driving guidance on how to drive these new vehicle systems in a way that is safe to both the public and themselves. To this end, this manuscript aims to model and develop a set of rules and instructions for ATMA system operators, particularly when it comes to critical locations where essential decision making is needed. Specifically, three technical requirements are investigated: car-following distance, critical lane-changing gap distance, and intersection clearance time. Newell’s simplified car-following model, and the classic lane-changing behavior model are modified, with roll-ahead distance taken into account, to model the driving behaviors of the ATMA vehicles at those critical decision-making locations. Data are collected from real-world field testing to calibrate and validate the developed models. The modeling outputs suggest important thresholds for ATMA system operators to follow. For example, on a freeway with a speed limit of 70 mph and ATMA operating speed of 10 mph, car-following distance should be no less than 75 ft for the lead truck and 100 ft for the follower truck, the critical lane-changing gap distance is 912 ft, and a minimum intersection clearance is 15 s, which are all much higher than the requirements for a general vehicle.

Research on decision-making of autonomous vehicle following based on reinforcement learning method

2019 ◽  
Vol 46 (3) ◽  
pp. 444-452 ◽  
Author(s):  
Hongbo Gao ◽  
Guanya Shi ◽  
Kelong Wang ◽  
Guotao Xie ◽  
Yuchao Liu
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Learning System ◽  
Explicit Calculation ◽  
Gradient Descent Method ◽  
Learning Method ◽  
Content Type ◽  
Car Following

Purpose Over the past decades, there has been significant research effort dedicated to the development of autonomous vehicles. The decision-making system, which is responsible for driving safety, is one of the most important technologies for autonomous vehicles. The purpose of this study is the use of an intensive learning method combined with car-following data by a driving simulator to obtain an explanatory learning following algorithm and establish an anthropomorphic car-following model. Design/methodology/approach This paper proposed car-following method based on reinforcement learning for autonomous vehicles decision-making. An approximator is used to approximate the value function by determining state space, action space and state transition relationship. A gradient descent method is used to solve the parameter. Findings The effect of car-following on certain driving styles is initially achieved through the simulation of step conditions. The effect of car-following initially proves that the reinforcement learning system is more adaptive to car following and that it has certain explanatory and stability based on the explicit calculation of R. Originality/value The simulation results show that the car-following method based on reinforcement learning for autonomous vehicle decision-making realizes reliable car-following decision-making and has the advantages of simple sample, small amount of data, simple algorithm and good robustness.

Simulation and evaluation of speed and lane-changing advisory of CAVS at work zones in heterogeneous traffic flow

2020 ◽  
Vol 34 (21) ◽  
pp. 2050201
Author(s):  
Wenjing Wu ◽  
Renchao Sun ◽  
Anning Ni ◽  
Zhikang Liang ◽  
Hongfei Jia
Keyword(s):  
Traffic Flow ◽  
Control Strategy ◽  
Control Strategies ◽  
Heavy Traffic ◽  
Autonomous Vehicle ◽  
Heterogeneous Traffic ◽  
Work Zone ◽  
Lane Changing ◽  
Advantages And Disadvantages ◽  
Car Following Model

Emerging connected autonomous vehicle (CAV) technologies provide an opportunity to the vehicle motion control to improve the traffic performance. This study simulated and evaluated the CAV-based speed and lane-changing (LC) control strategies at the expressway work zone in heterogeneous traffic flow. The control strategies of CAV are optimized by the multi-layer control structure based on model predictive control. The heterogeneous traffic flow composed of human-driven vehicles and CAVs is constructed based on cellular automata by the proposed Expected Distance-based Symmetric Two-lane Cellular Automate (ED-STCA) LC model and CAV car-following model. The six control strategies composed of variable speed limits (VSL), LC and their coordinated control strategies are experimented. The average travel time and throughput are selected to assess the advantages and disadvantages of each strategy under each combination of vehicles’ arrival rates and CAV mixed ratios. The numerical results show that: (i) the effect of the control strategy on the traffic is not obvious under free flow, and the control strategy may worsen the traffic under medium traffic. (ii) Early lane-changing control (ELC) is better than late lane-changing control (LLC) under medium traffic, and LLC is better under heavy traffic. (iii) [Formula: see text] is the best choice under heavy traffic and the mixed rate of CAVs is high. The simulation results obtained in the paper would provide some practical references for transportation agencies to manage the traffic in work zone under networking environment in the future.

scholarly journals Conditional Artificial Potential Field-Based Autonomous Vehicle Safety Control with Interference of Lane Changing in Mixed Traffic Scenario

Sensors ◽  
2019 ◽  
Vol 19 (19) ◽  
pp. 4199 ◽  
Author(s):  
Kai Gao ◽  
Di Yan ◽  
Fan Yang ◽  
Jin Xie ◽  
Li Liu ◽  
...  
Keyword(s):  
Potential Field ◽  
Ride Comfort ◽  
Dynamic Performance ◽  
Autonomous Vehicle ◽  
Support Vector ◽  
Artificial Potential Field ◽  
Lane Change ◽  
Lane Changing ◽  
Safety Control ◽  
Car Following

Car-following is an essential trajectory control strategy for the autonomous vehicle, which not only improves traffic efficiency, but also reduces fuel consumption and emissions. However, the prediction of lane change intentions in adjacent lanes is problematic, and will significantly affect the car-following control of the autonomous vehicle, especially when the vehicle changing lanes is only a connected unintelligent vehicle without expensive and accurate sensors. Autonomous vehicles suffer from adjacent vehicles’ abrupt lane changes, which may reduce ride comfort and increase energy consumption, and even lead to a collision. A machine learning-based lane change intention prediction and real time autonomous vehicle controller is proposed to respond to this problem. First, an interval-based support vector machine is designed to predict the vehicles’ lane change intention utilizing limited low-level vehicle status through vehicle-to-vehicle communication. Then, a conditional artificial potential field method is used to design the car-following controller by incorporating the lane-change intentions of the vehicle. Experimental results reveal that the proposed method can estimate a vehicle’s lane change intention more accurately. The autonomous vehicle avoids collisions with a lane-changing connected unintelligent vehicle with reliable safety and favorable dynamic performance.

Lane-Change Gaming Decision Control Based on Multiple Targets Evaluation for Autonomous Vehicle

2021 ◽  
pp. 036119812110111
Author(s):  
Yangyang Wang ◽  
Hangyun Deng ◽  
Guangda Chen
Keyword(s):  
Ride Comfort ◽  
Autonomous Vehicle ◽  
Objective Evaluation ◽  
Smooth Transition ◽  
Longitudinal Motion ◽  
Lane Change ◽  
Lane Changing ◽  
New Model ◽  
Car Following ◽  
Traffic Efficiency

Automatic lane change is one of the most important highway operations. It seriously affects traffic efficiency and safety. It is also an important driving technology for automatic driving. To achieve the best automatic lane-change control, it is necessary to achieve the control from the perspective of multi-objective evaluation. In this paper, to make it applicable for a hybrid condition of car following and lane change, the traditional car-following model is modified by regarding the longitudinal motion during the lane-changing process as a transition of the car-following behavior in the two lanes before and after a certain lane-change behavior. A hyperbolic tangent transition function is introduced to connect the model to achieve a smooth transition of the model output. Then, the discretionary lane-change decision process of highway autonomous vehicles is modeled into a two-vehicle game model, and a comprehensive loss function concerning safety, efficiency, and ride comfort is proposed for the evaluation of the strategies. The optimal strategy is obtained by minimizing the expectation of losses. Finally, to verify the performance of the proposed new model, simulations of different car-following and lane-changing models are carried out, which is for multi-target simulation conditions. The results of the simulation show that the new model exhibits higher traffic efficiency, better homogeneity, and stability.

Optimization of lane-changing advisory of connected and autonomous vehicles at a multi-lane work zone

2021 ◽  
pp. 2250020
Author(s):  
Wenjing Wu ◽  
Yongbin Zhan ◽  
Lili Yang ◽  
Renchao Sun ◽  
Anning Ni
Keyword(s):  
Autonomous Vehicles ◽  
Control Strategies ◽  
Autonomous Vehicle ◽  
Heterogeneous Traffic ◽  
Work Zone ◽  
Lane Changing ◽  
The Road ◽  
Traffic Efficiency ◽  
Traffic Distribution ◽  
Traffic Performance

The work zone with lane closure will be an active bottleneck due to vehicles’ mandatory lane-changing conflicts. The emerging Connected Autonomous Vehicle (CAV) technology provides opportunities for vehicle motion planning to improve traffic performance. However, the literature using CAV technology mainly focuses on single-lane lane-changing control in the merging area. The algorithm dealing with multi-lane lane-changing control is absent. In this paper, a simulation system with a lane-changing optimal strategy embedded for the multi-lane work zone is presented under the heterogeneous traffic flow. First, the road upstream of the work zone is divided into several segments, and an optimal multi-lane lane-changing algorithm is designed. It is recommended that CAVs, on the closure lane and the merged lane, change lanes on each segment to balance traffic distribution and minimize traffic delay. Second, to validate the algorithm proposed, a typical three-lane freeway with one-lane closed for the work zone is researched, and the simulation platform based on cellular automata is developed. Third, the advantages of multi-lane control strategies are studied and discussed in traffic efficiency improvement and collision risk reduction by comparing previous lane-changing control algorithms.

scholarly journals Car-following method based on inverse reinforcement learning for autonomous vehicle decision-making

2018 ◽  
Vol 15 (6) ◽  
pp. 172988141881716 ◽  
Author(s):  
Hongbo Gao ◽  
Guanya Shi ◽  
Guotao Xie ◽  
Bo Cheng
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Optimization Problems ◽  
Learning Algorithm ◽  
Autonomous Vehicle ◽  
Sequential Decision ◽  
Inverse Reinforcement Learning ◽  
Car Following ◽  
Reward Function ◽  
Automatic Driving

There are still some problems need to be solved though there are a lot of achievements in the fields of automatic driving. One of those problems is the difficulty of designing a car-following decision-making system for complex traffic conditions. In recent years, reinforcement learning shows the potential in solving sequential decision optimization problems. In this article, we establish the reward function R of each driver data based on the inverse reinforcement learning algorithm, and r visualization is carried out, and then driving characteristics and following strategies are analyzed. At last, we show the efficiency of the proposed method by simulation in a highway environment.

Sign in / Sign up

Export Citation Format

Share Document