Research on decision-making of autonomous vehicle following based on reinforcement learning method

2019 ◽  
Vol 46 (3) ◽  
pp. 444-452 ◽  
Author(s):  
Hongbo Gao ◽  
Guanya Shi ◽  
Kelong Wang ◽  
Guotao Xie ◽  
Yuchao Liu
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Learning System ◽  
Explicit Calculation ◽  
Gradient Descent Method ◽  
Learning Method ◽  
Content Type ◽  
Car Following

Purpose Over the past decades, there has been significant research effort dedicated to the development of autonomous vehicles. The decision-making system, which is responsible for driving safety, is one of the most important technologies for autonomous vehicles. The purpose of this study is the use of an intensive learning method combined with car-following data by a driving simulator to obtain an explanatory learning following algorithm and establish an anthropomorphic car-following model. Design/methodology/approach This paper proposed car-following method based on reinforcement learning for autonomous vehicles decision-making. An approximator is used to approximate the value function by determining state space, action space and state transition relationship. A gradient descent method is used to solve the parameter. Findings The effect of car-following on certain driving styles is initially achieved through the simulation of step conditions. The effect of car-following initially proves that the reinforcement learning system is more adaptive to car following and that it has certain explanatory and stability based on the explicit calculation of R. Originality/value The simulation results show that the car-following method based on reinforcement learning for autonomous vehicle decision-making realizes reliable car-following decision-making and has the advantages of simple sample, small amount of data, simple algorithm and good robustness.

scholarly journals Car-following method based on inverse reinforcement learning for autonomous vehicle decision-making

2018 ◽  
Vol 15 (6) ◽  
pp. 172988141881716 ◽  
Author(s):  
Hongbo Gao ◽  
Guanya Shi ◽  
Guotao Xie ◽  
Bo Cheng
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Optimization Problems ◽  
Learning Algorithm ◽  
Autonomous Vehicle ◽  
Sequential Decision ◽  
Inverse Reinforcement Learning ◽  
Car Following ◽  
Reward Function ◽  
Automatic Driving

There are still some problems need to be solved though there are a lot of achievements in the fields of automatic driving. One of those problems is the difficulty of designing a car-following decision-making system for complex traffic conditions. In recent years, reinforcement learning shows the potential in solving sequential decision optimization problems. In this article, we establish the reward function R of each driver data based on the inverse reinforcement learning algorithm, and r visualization is carried out, and then driving characteristics and following strategies are analyzed. At last, we show the efficiency of the proposed method by simulation in a highway environment.

scholarly journals A Game Theory-Based Approach for Modeling Autonomous Vehicle Behavior in Congested, Urban Lane-Changing Scenarios

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1523
Author(s):  
Nikita Smirnov ◽  
Yuzhou Liu ◽  
Aso Validi ◽  
Walter Morales-Alvarez ◽  
Cristina Olaverri-Monreal
Keyword(s):  
Game Theory ◽  
Decision Making ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Urban Traffic ◽  
Road User ◽  
Lane Change ◽  
Traffic Light ◽  
User Interactions ◽  
Lane Changing

Autonomous vehicles are expected to display human-like behavior, at least to the extent that their decisions can be intuitively understood by other road users. If this is not the case, the coexistence of manual and autonomous vehicles in a mixed environment might affect road user interactions negatively and might jeopardize road safety. To this end, it is highly important to design algorithms that are capable of analyzing human decision-making processes and of reproducing them. In this context, lane-change maneuvers have been studied extensively. However, not all potential scenarios have been considered, since most works have focused on highway rather than urban scenarios. We contribute to the field of research by investigating a particular urban traffic scenario in which an autonomous vehicle needs to determine the level of cooperation of the vehicles in the adjacent lane in order to proceed with a lane change. To this end, we present a game theory-based decision-making model for lane changing in congested urban intersections. The model takes as input driving-related parameters related to vehicles in the intersection before they come to a complete stop. We validated the model by relying on the Co-AutoSim simulator. We compared the prediction model outcomes with actual participant decisions, i.e., whether they allowed the autonomous vehicle to drive in front of them. The results are promising, with the prediction accuracy being 100% in all of the cases in which the participants allowed the lane change and 83.3% in the other cases. The false predictions were due to delays in resuming driving after the traffic light turned green.

scholarly journals An Efficiency Enhancing Methodology for Multiple Autonomous Vehicles in an Urban Network Adopting Deep Reinforcement Learning

2021 ◽  
Vol 11 (4) ◽  
pp. 1514 ◽  
Author(s):  
Quang-Duy Tran ◽  
Sang-Hoon Bae
Keyword(s):  
Reinforcement Learning ◽  
Traffic Congestion ◽  
Autonomous Vehicles ◽  
Penetration Rate ◽  
Autonomous Vehicle ◽  
Effective Means ◽  
Urban Network ◽  
Learning Agents ◽  
Policy Optimization ◽  
The Impact

To reduce the impact of congestion, it is necessary to improve our overall understanding of the influence of the autonomous vehicle. Recently, deep reinforcement learning has become an effective means of solving complex control tasks. Accordingly, we show an advanced deep reinforcement learning that investigates how the leading autonomous vehicles affect the urban network under a mixed-traffic environment. We also suggest a set of hyperparameters for achieving better performance. Firstly, we feed a set of hyperparameters into our deep reinforcement learning agents. Secondly, we investigate the leading autonomous vehicle experiment in the urban network with different autonomous vehicle penetration rates. Thirdly, the advantage of leading autonomous vehicles is evaluated using entire manual vehicle and leading manual vehicle experiments. Finally, the proximal policy optimization with a clipped objective is compared to the proximal policy optimization with an adaptive Kullback–Leibler penalty to verify the superiority of the proposed hyperparameter. We demonstrate that full automation traffic increased the average speed 1.27 times greater compared with the entire manual vehicle experiment. Our proposed method becomes significantly more effective at a higher autonomous vehicle penetration rate. Furthermore, the leading autonomous vehicles could help to mitigate traffic congestion.

scholarly journals Deep reinforcement learning based control for Autonomous Vehicles in CARLA

2022 ◽  
Author(s):  
Óscar Pérez-Gil ◽  
Rafael Barea ◽  
Elena López-Guillén ◽  
Luis M. Bergasa ◽  
Carlos Gómez-Huélamo ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Vehicle Control ◽  
Data Sources ◽  
Simulation Environment ◽  
Urban Simulation ◽  
Policy Gradient ◽  
Almost All ◽  
Control Layer

AbstractNowadays, Artificial Intelligence (AI) is growing by leaps and bounds in almost all fields of technology, and Autonomous Vehicles (AV) research is one more of them. This paper proposes the using of algorithms based on Deep Learning (DL) in the control layer of an autonomous vehicle. More specifically, Deep Reinforcement Learning (DRL) algorithms such as Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG) are implemented in order to compare results between them. The aim of this work is to obtain a trained model, applying a DRL algorithm, able of sending control commands to the vehicle to navigate properly and efficiently following a determined route. In addition, for each of the algorithms, several agents are presented as a solution, so that each of these agents uses different data sources to achieve the vehicle control commands. For this purpose, an open-source simulator such as CARLA is used, providing to the system with the ability to perform a multitude of tests without any risk into an hyper-realistic urban simulation environment, something that is unthinkable in the real world. The results obtained show that both DQN and DDPG reach the goal, but DDPG obtains a better performance. DDPG perfoms trajectories very similar to classic controller as LQR. In both cases RMSE is lower than 0.1m following trajectories with a range 180-700m. To conclude, some conclusions and future works are commented.

scholarly journals Heuristics-oriented overtaking decision making for autonomous vehicles using reinforcement learning

2020 ◽  
Vol 10 (4) ◽  
pp. 417-424
Author(s):  
Teng Liu ◽  
Bing Huang ◽  
Zejian Deng ◽  
Hong Wang ◽  
Xiaolin Tang ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Autonomous Vehicles

Development of Operation Guidelines for Leader–Follower Autonomous Maintenance Vehicles at Work Zone Locations

2021 ◽  
pp. 036119812110566
Author(s):  
Qing Tang ◽  
Xianbiao Hu ◽  
Ruwen Qin
Keyword(s):  
Decision Making ◽  
Autonomous Vehicle ◽  
Field Testing ◽  
Work Zone ◽  
Lane Changing ◽  
The Public ◽  
Car Following ◽  
Technical Requirements ◽  
Car Following Model ◽  
Niche Area

The rapid advancement of connected and autonomous vehicle (CAV) technologies, although possibly years away from wide application to the general public travel, are receiving attention from many state Departments of Transportation (DOT) in the niche area of using autonomous maintenance technology (AMT) to reduce fatalities of DOT workers in work zone locations. Although promising results are shown in testing and deployments in several states, current autonomous truck mounted attenuator (ATMA) system operators are not provided with much practical driving guidance on how to drive these new vehicle systems in a way that is safe to both the public and themselves. To this end, this manuscript aims to model and develop a set of rules and instructions for ATMA system operators, particularly when it comes to critical locations where essential decision making is needed. Specifically, three technical requirements are investigated: car-following distance, critical lane-changing gap distance, and intersection clearance time. Newell’s simplified car-following model, and the classic lane-changing behavior model are modified, with roll-ahead distance taken into account, to model the driving behaviors of the ATMA vehicles at those critical decision-making locations. Data are collected from real-world field testing to calibrate and validate the developed models. The modeling outputs suggest important thresholds for ATMA system operators to follow. For example, on a freeway with a speed limit of 70 mph and ATMA operating speed of 10 mph, car-following distance should be no less than 75 ft for the lead truck and 100 ft for the follower truck, the critical lane-changing gap distance is 912 ft, and a minimum intersection clearance is 15 s, which are all much higher than the requirements for a general vehicle.

Robot hand-eye cooperation based on improved inverse reinforcement learning

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Ning Yu ◽  
Lin Nan ◽  
Tao Ku
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Visual Information ◽  
Industrial Robots ◽  
Robot Hand ◽  
Inverse Reinforcement Learning ◽  
Generative Adversarial Network ◽  
Content Type ◽  
Coordination Model ◽  
Proposed Model

Purpose How to make accurate action decisions based on visual information is one of the important research directions of industrial robots. The purpose of this paper is to design a highly optimized hand-eye coordination model of the robot to improve the robots’ on-site decision-making ability. Design/methodology/approach The combination of inverse reinforcement learning (IRL) algorithm and generative adversarial network can effectively reduce the dependence on expert samples and robots can obtain the decision-making performance that the degree of optimization is not lower than or even higher than that of expert samples. Findings The performance of the proposed model is verified in the simulation environment and real scene. By monitoring the reward distribution of the reward function and the trajectory of the robot, the proposed model is compared with other existing methods. The experimental results show that the proposed model has better decision-making performance in the case of less expert data. Originality/value A robot hand-eye cooperation model based on improved IRL is proposed and verified. Empirical investigations on real experiments reveal that overall, the proposed approach tends to improve the real efficiency by more than 10% when compared to alternative hand-eye cooperation methods.

China will make rapid progress in autonomous vehicles

2018 ◽  
Keyword(s):  
Autonomous Vehicles ◽  
Elderly Population ◽  
Autonomous Vehicle ◽  
National Development ◽  
Rapid Progress ◽  
Dual Use ◽  
Content Type ◽  
Autonomous Cars ◽  
Smart Cars ◽  
The Military

Subject China's policies to develop the autonomous vehicle sector. Significance Chinese policymakers believe the size of their domestic market will give China’s vehicle makers the scale to lead the world in autonomous cars. The National Development and Reform Commission expects that 50% of new vehicles sold in China by 2020 will be ‘smart cars’, that is, with partial or fully autonomous functions. Impacts Policies do not explicitly favour fleet vehicles over private cars, but fleet vehicles are likely to lead adoption. Self-driving fleet services are a future way to provide mobility for a growing elderly population. Regardless of international concerns about protectionism, all levels of government will use preferential procurement to support the sector. China's civilian autonomous vehicle sector will benefit from dual-use technology developed by the military.

Sign in / Sign up

Export Citation Format

Share Document