scholarly journals Decision-Making System for Lane Change Using Deep Reinforcement Learning in Connected and Automated Driving

Electronics ◽  
2019 ◽  
Vol 8 (5) ◽  
pp. 543 ◽  
Author(s):  
HongIl An ◽  
Jae-il Jung
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Learning Algorithm ◽  
The State ◽  
Lane Change ◽  
Vehicular Communication ◽  
Automated Driving ◽  
Lane Changing ◽  
State Information ◽  
Gradient Learning

Lane changing systems have consistently received attention in the fields of vehicular communication and autonomous vehicles. In this paper, we propose a lane change system that combines deep reinforcement learning and vehicular communication. A host vehicle, trying to change lanes, receives the state information of the host vehicle and a remote vehicle that are both equipped with vehicular communication devices. A deep deterministic policy gradient learning algorithm in the host vehicle determines the high-level action of the host vehicle from the state information. The proposed system learns straight-line driving and collision avoidance actions without vehicle dynamics knowledge. Finally, we consider the update period for the state information from the host and remote vehicles.

scholarly journals End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm

Sensors ◽  
2020 ◽  
Vol 20 (18) ◽  
pp. 5443
Author(s):  
Hongyu Hu ◽  
Ziyang Lu ◽  
Qi Wang ◽  
Chengyuan Zheng
Keyword(s):  
Autonomous Vehicles ◽  
The State ◽  
Steering Wheel ◽  
Lane Change ◽  
Simulation Environment ◽  
Longitudinal Control ◽  
Driving Style ◽  
Lane Changing ◽  
Policy Gradient ◽  
End To End

Changing lanes while driving requires coordinating the lateral and longitudinal controls of a vehicle, considering its running state and the surrounding environment. Although the existing rule-based automated lane-changing method is simple, it is unsuitable for unpredictable scenarios encountered in practice. Therefore, using a deep deterministic policy gradient (DDPG) algorithm, we propose an end-to-end method for automated lane changing based on lidar data. The distance state information of the lane boundary and the surrounding vehicles obtained by the agent in a simulation environment is denoted as the state space for an automated lane-change problem based on reinforcement learning. The steering wheel angle and longitudinal acceleration are used as the action space, and both the state and action spaces are continuous. In terms of the reward function, avoiding collision and setting different expected lane-changing distances that represent different driving styles are considered for security, and the angular velocity of the steering wheel and jerk are considered for comfort. The minimum speed limit for lane changing and the control of the agent for a quick lane change are considered for efficiency. For a one-way two-lane road, a visual simulation environment scene is constructed using Pyglet. By comparing the lane-changing process tracks of two driving styles in a simplified traffic flow scene, we study the influence of driving style on the lane-changing process and lane-changing time. Through the training and adjustment of the combined lateral and longitudinal control of autonomous vehicles with different driving styles in complex traffic scenes, the vehicles could complete a series of driving tasks while considering driving-style differences. The experimental results show that autonomous vehicles can reflect the differences in the driving styles at the time of lane change at the same speed. Under the combined lateral and longitudinal control, the autonomous vehicles exhibit good robustness to different speeds and traffic density in different road sections. Thus, autonomous vehicles trained using the proposed method can learn an automated lane-changing policy while considering safety, comfort, and efficiency.

scholarly journals A Game Theory-Based Approach for Modeling Autonomous Vehicle Behavior in Congested, Urban Lane-Changing Scenarios

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1523
Author(s):  
Nikita Smirnov ◽  
Yuzhou Liu ◽  
Aso Validi ◽  
Walter Morales-Alvarez ◽  
Cristina Olaverri-Monreal
Keyword(s):  
Game Theory ◽  
Decision Making ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Urban Traffic ◽  
Road User ◽  
Lane Change ◽  
Traffic Light ◽  
User Interactions ◽  
Lane Changing

Autonomous vehicles are expected to display human-like behavior, at least to the extent that their decisions can be intuitively understood by other road users. If this is not the case, the coexistence of manual and autonomous vehicles in a mixed environment might affect road user interactions negatively and might jeopardize road safety. To this end, it is highly important to design algorithms that are capable of analyzing human decision-making processes and of reproducing them. In this context, lane-change maneuvers have been studied extensively. However, not all potential scenarios have been considered, since most works have focused on highway rather than urban scenarios. We contribute to the field of research by investigating a particular urban traffic scenario in which an autonomous vehicle needs to determine the level of cooperation of the vehicles in the adjacent lane in order to proceed with a lane change. To this end, we present a game theory-based decision-making model for lane changing in congested urban intersections. The model takes as input driving-related parameters related to vehicles in the intersection before they come to a complete stop. We validated the model by relying on the Co-AutoSim simulator. We compared the prediction model outcomes with actual participant decisions, i.e., whether they allowed the autonomous vehicle to drive in front of them. The results are promising, with the prediction accuracy being 100% in all of the cases in which the participants allowed the lane change and 83.3% in the other cases. The false predictions were due to delays in resuming driving after the traffic light turned green.

Characterizing Lane Changes via Digitized Infrastructure and Low-Cost GPS

2019 ◽  
Vol 2673 (8) ◽  
pp. 298-309 ◽  
Author(s):  
Ishtiak Ahmed ◽  
Alan Karr ◽  
Nagui M. Rouphail ◽  
Gyounghoon Chun ◽  
Shams Tanvir
Keyword(s):  
Autonomous Vehicles ◽  
Lateral Displacement ◽  
Low Cost ◽  
Detection Algorithm ◽  
Lane Change ◽  
Trajectory Data ◽  
Lane Changing ◽  
Lane Changes ◽  
Detection Systems ◽  
Level Information

With the expected increase in the availability of trajectory-level information from connected and autonomous vehicles, issues of lane changing behavior that were difficult to assess with traditional freeway detection systems can now begin to be addressed. This study presents the development and application of a lane change detection algorithm that uses trajectory data from a low-cost GPS-equipped fleet, supplemented with digitized lane markings. The proposed algorithm minimizes the effect of GPS errors by constraining the temporal duration and lateral displacement of a lane change detected using preliminary lane positioning. The algorithm was applied to 637 naturalistic trajectories traversing a long weaving segment and validated through a series of controlled lane change experiments. Analysis of naturalistic trajectory data revealed that ramp-to-freeway trips had the highest number of discretionary lane changes in excess of 1 lane change/vehicle. Overall, excessive lane change rates were highest between the two middle freeway lanes at 0.86 lane changes/vehicle. These results indicate that extreme lane changing behavior may significantly contribute to the peak-hour congestion at the site. The average lateral speed during lane change was 2.7 fps, consistent with the literature, with several freeway–freeway and ramp–ramp trajectories showing speeds up to 7.7 fps. All ramp-to-freeway vehicles executed their first mandatory lane change within 62.5% of the total weaving length, although other weaving lane changes were spread over the entire segment. These findings can be useful for implementing strategies to lessen abrupt and excessive lane changes through better lane pre-positioning.

scholarly journals An Improved Reinforcement Learning Algorithm for Cooperative Behaviors of Mobile Robots

2014 ◽  
Vol 2014 ◽  
pp. 1-8 ◽  
Author(s):  
Yong Song ◽  
Yibin Li ◽  
Xiaoli Wang ◽  
Xin Ma ◽  
Jiuhong Ruan
Keyword(s):  
Reinforcement Learning ◽  
Mobile Robots ◽  
Knowledge Sharing ◽  
State Space ◽  
Learning Algorithm ◽  
The State ◽  
Convergence Speed ◽  
Exponential Increase ◽  
Cooperative Behaviors ◽  
Reinforcement Learning Algorithm

Reinforcement learning algorithm for multirobot will become very slow when the number of robots is increasing resulting in an exponential increase of state space. A sequentialQ-learning based on knowledge sharing is presented. The rule repository of robots behaviors is firstly initialized in the process of reinforcement learning. Mobile robots obtain present environmental state by sensors. Then the state will be matched to determine if the relevant behavior rule has been stored in the database. If the rule is present, an action will be chosen in accordance with the knowledge and the rules, and the matching weight will be refined. Otherwise the new rule will be appended to the database. The robots learn according to a given sequence and share the behavior database. We examine the algorithm by multirobot following-surrounding behavior, and find that the improved algorithm can effectively accelerate the convergence speed.

scholarly journals Model-based estimation of the state of vehicle automation as derived from the driver’s spontaneous visual strategies

2021 ◽  
Vol 12 (3) ◽  
Author(s):  
Damien Schnebelen ◽  
Camilo Charron ◽  
Franck Mars
Keyword(s):  
Autonomous Vehicles ◽  
Autonomous Driving ◽  
The State ◽  
Partial Least Square ◽  
Least Square ◽  
Transition Matrices ◽  
Automated Driving ◽  
Gaze Behaviour ◽  
Motor Actions ◽  
Dynamic Indicators

When manually steering a car, the driver’s visual perception of the driving scene and his or her motor actions to control the vehicle are closely linked. Since motor behaviour is no longer required in an automated vehicle, the sampling of the visual scene is affected. Autonomous driving typically results in less gaze being directed towards the road centre and a broader exploration of the driving scene, compared to manual driving. To examine the corollary of this situation, this study estimated the state of automation (manual or automated) on the basis of gaze behaviour. To do so, models based on partial least square regressions were computed by considering the gaze behaviour in multiple ways, using static indicators (percentage of time spent gazing at 13 areas of interests), dynamic indicators (transition matrices between areas) or both together. Analysis of the quality of predictions for the different models showed that the best result was obtained by considering both static and dynamic indicators. However, gaze dynamics played the most important role in distinguishing between manual and automated driving. This study may be relevant to the issue of driver monitoring in autonomous vehicles.

scholarly journals A Simplified Pursuit-evasion Game with Reinforcement Learning

2021 ◽  
Vol 65 (2) ◽  
pp. 160-166
Author(s):  
Gabor Paczolay ◽  
Istvan Harmati
Keyword(s):  
Linear Programming ◽  
Reinforcement Learning ◽  
Programming Problem ◽  
Collision Avoidance ◽  
Linear Programming Problem ◽  
The State ◽  
State Information ◽  
Pursuit Evasion ◽  
Pursuit And Evasion ◽  
Evasion Game

In this paper we visit the problem of pursuit and evasion and specifically, the collision avoidance during the problem. Two distinct tasks are visited: the first is a scenario when the agents can communicate with each other online, meanwhile in the second scenario they have to only rely on the state information and the knowledge about other agents' actions. We propose a method combining the already existing Minimax-Q and Nash-Q algorithms to provide a solution that can better take the enemy as well as friendly agents' actions into consideration. This combination is a simple weighting of the two algorithms with the Minimax-Q algorithm being based on a linear programming problem.

scholarly journals Inferring the Driver’s Lane Change Intention through LiDAR-Based Environment Analysis Using Convolutional Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 475
Author(s):  
Alberto Díaz-Álvarez ◽  
Miguel Clavijo ◽  
Felipe Jiménez ◽  
Francisco Serradilla
Keyword(s):  
Autonomous Vehicles ◽  
Lane Change ◽  
Automated Driving ◽  
Driver Assistance Systems ◽  
Data Set ◽  
Environment Analysis ◽  
Process Support ◽  
Surrounding Environment ◽  
Wide Range ◽  
Input Variables

Most of the tactic manoeuvres during driving require a certain understanding of the surrounding environment from which to devise our future behaviour. In this paper, a Convolutional Neural Network (CNN) approach is used to model the lane change behaviour to identify when a driver is going to perform this manoeuvre. To that end, a slightly modified CNN architecture adapted to both spatial (i.e., surrounding environment) and non-spatial (i.e., rest of variables such as relative speed to the front vehicle) input variables. Anticipating a driver’s lane change intention means it is possible to use this information as a new source of data in wide range of different scenarios. One example of such scenarios might be the decision making process support for human drivers through Advanced Driver Assistance Systems (ADAS) fed with the data of the surrounding cars in an inter-vehicular network. Another example might even be its use in autonomous vehicles by using the data of a specific driver profile to make automated driving more human-like. Several CNN architectures have been tested on a simulation environment to assess their performance. Results show that the selected architecture provides a higher degree of accuracy than random guessing (i.e., assigning a class randomly for each observation in the data set), and it can capture subtle differences in behaviour between different driving profiles.

Sign in / Sign up

Export Citation Format

Share Document