scholarly journals End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm

Sensors ◽  
2020 ◽  
Vol 20 (18) ◽  
pp. 5443
Author(s):  
Hongyu Hu ◽  
Ziyang Lu ◽  
Qi Wang ◽  
Chengyuan Zheng
Keyword(s):  
Autonomous Vehicles ◽  
The State ◽  
Steering Wheel ◽  
Lane Change ◽  
Simulation Environment ◽  
Longitudinal Control ◽  
Driving Style ◽  
Lane Changing ◽  
Policy Gradient ◽  
End To End

Changing lanes while driving requires coordinating the lateral and longitudinal controls of a vehicle, considering its running state and the surrounding environment. Although the existing rule-based automated lane-changing method is simple, it is unsuitable for unpredictable scenarios encountered in practice. Therefore, using a deep deterministic policy gradient (DDPG) algorithm, we propose an end-to-end method for automated lane changing based on lidar data. The distance state information of the lane boundary and the surrounding vehicles obtained by the agent in a simulation environment is denoted as the state space for an automated lane-change problem based on reinforcement learning. The steering wheel angle and longitudinal acceleration are used as the action space, and both the state and action spaces are continuous. In terms of the reward function, avoiding collision and setting different expected lane-changing distances that represent different driving styles are considered for security, and the angular velocity of the steering wheel and jerk are considered for comfort. The minimum speed limit for lane changing and the control of the agent for a quick lane change are considered for efficiency. For a one-way two-lane road, a visual simulation environment scene is constructed using Pyglet. By comparing the lane-changing process tracks of two driving styles in a simplified traffic flow scene, we study the influence of driving style on the lane-changing process and lane-changing time. Through the training and adjustment of the combined lateral and longitudinal control of autonomous vehicles with different driving styles in complex traffic scenes, the vehicles could complete a series of driving tasks while considering driving-style differences. The experimental results show that autonomous vehicles can reflect the differences in the driving styles at the time of lane change at the same speed. Under the combined lateral and longitudinal control, the autonomous vehicles exhibit good robustness to different speeds and traffic density in different road sections. Thus, autonomous vehicles trained using the proposed method can learn an automated lane-changing policy while considering safety, comfort, and efficiency.

scholarly journals Decision-Making System for Lane Change Using Deep Reinforcement Learning in Connected and Automated Driving

Electronics ◽  
2019 ◽  
Vol 8 (5) ◽  
pp. 543 ◽  
Author(s):  
HongIl An ◽  
Jae-il Jung
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Learning Algorithm ◽  
The State ◽  
Lane Change ◽  
Vehicular Communication ◽  
Automated Driving ◽  
Lane Changing ◽  
State Information ◽  
Gradient Learning

Lane changing systems have consistently received attention in the fields of vehicular communication and autonomous vehicles. In this paper, we propose a lane change system that combines deep reinforcement learning and vehicular communication. A host vehicle, trying to change lanes, receives the state information of the host vehicle and a remote vehicle that are both equipped with vehicular communication devices. A deep deterministic policy gradient learning algorithm in the host vehicle determines the high-level action of the host vehicle from the state information. The proposed system learns straight-line driving and collision avoidance actions without vehicle dynamics knowledge. Finally, we consider the update period for the state information from the host and remote vehicles.

scholarly journals A Game Theory-Based Approach for Modeling Autonomous Vehicle Behavior in Congested, Urban Lane-Changing Scenarios

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1523
Author(s):  
Nikita Smirnov ◽  
Yuzhou Liu ◽  
Aso Validi ◽  
Walter Morales-Alvarez ◽  
Cristina Olaverri-Monreal
Keyword(s):  
Game Theory ◽  
Decision Making ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Urban Traffic ◽  
Road User ◽  
Lane Change ◽  
Traffic Light ◽  
User Interactions ◽  
Lane Changing

Autonomous vehicles are expected to display human-like behavior, at least to the extent that their decisions can be intuitively understood by other road users. If this is not the case, the coexistence of manual and autonomous vehicles in a mixed environment might affect road user interactions negatively and might jeopardize road safety. To this end, it is highly important to design algorithms that are capable of analyzing human decision-making processes and of reproducing them. In this context, lane-change maneuvers have been studied extensively. However, not all potential scenarios have been considered, since most works have focused on highway rather than urban scenarios. We contribute to the field of research by investigating a particular urban traffic scenario in which an autonomous vehicle needs to determine the level of cooperation of the vehicles in the adjacent lane in order to proceed with a lane change. To this end, we present a game theory-based decision-making model for lane changing in congested urban intersections. The model takes as input driving-related parameters related to vehicles in the intersection before they come to a complete stop. We validated the model by relying on the Co-AutoSim simulator. We compared the prediction model outcomes with actual participant decisions, i.e., whether they allowed the autonomous vehicle to drive in front of them. The results are promising, with the prediction accuracy being 100% in all of the cases in which the participants allowed the lane change and 83.3% in the other cases. The false predictions were due to delays in resuming driving after the traffic light turned green.

scholarly journals Deep reinforcement learning based control for Autonomous Vehicles in CARLA

2022 ◽  
Author(s):  
Óscar Pérez-Gil ◽  
Rafael Barea ◽  
Elena López-Guillén ◽  
Luis M. Bergasa ◽  
Carlos Gómez-Huélamo ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Vehicle Control ◽  
Data Sources ◽  
Simulation Environment ◽  
Urban Simulation ◽  
Policy Gradient ◽  
Almost All ◽  
Control Layer

AbstractNowadays, Artificial Intelligence (AI) is growing by leaps and bounds in almost all fields of technology, and Autonomous Vehicles (AV) research is one more of them. This paper proposes the using of algorithms based on Deep Learning (DL) in the control layer of an autonomous vehicle. More specifically, Deep Reinforcement Learning (DRL) algorithms such as Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG) are implemented in order to compare results between them. The aim of this work is to obtain a trained model, applying a DRL algorithm, able of sending control commands to the vehicle to navigate properly and efficiently following a determined route. In addition, for each of the algorithms, several agents are presented as a solution, so that each of these agents uses different data sources to achieve the vehicle control commands. For this purpose, an open-source simulator such as CARLA is used, providing to the system with the ability to perform a multitude of tests without any risk into an hyper-realistic urban simulation environment, something that is unthinkable in the real world. The results obtained show that both DQN and DDPG reach the goal, but DDPG obtains a better performance. DDPG perfoms trajectories very similar to classic controller as LQR. In both cases RMSE is lower than 0.1m following trajectories with a range 180-700m. To conclude, some conclusions and future works are commented.

Characterizing Lane Changes via Digitized Infrastructure and Low-Cost GPS

2019 ◽  
Vol 2673 (8) ◽  
pp. 298-309 ◽  
Author(s):  
Ishtiak Ahmed ◽  
Alan Karr ◽  
Nagui M. Rouphail ◽  
Gyounghoon Chun ◽  
Shams Tanvir
Keyword(s):  
Autonomous Vehicles ◽  
Lateral Displacement ◽  
Low Cost ◽  
Detection Algorithm ◽  
Lane Change ◽  
Trajectory Data ◽  
Lane Changing ◽  
Lane Changes ◽  
Detection Systems ◽  
Level Information

With the expected increase in the availability of trajectory-level information from connected and autonomous vehicles, issues of lane changing behavior that were difficult to assess with traditional freeway detection systems can now begin to be addressed. This study presents the development and application of a lane change detection algorithm that uses trajectory data from a low-cost GPS-equipped fleet, supplemented with digitized lane markings. The proposed algorithm minimizes the effect of GPS errors by constraining the temporal duration and lateral displacement of a lane change detected using preliminary lane positioning. The algorithm was applied to 637 naturalistic trajectories traversing a long weaving segment and validated through a series of controlled lane change experiments. Analysis of naturalistic trajectory data revealed that ramp-to-freeway trips had the highest number of discretionary lane changes in excess of 1 lane change/vehicle. Overall, excessive lane change rates were highest between the two middle freeway lanes at 0.86 lane changes/vehicle. These results indicate that extreme lane changing behavior may significantly contribute to the peak-hour congestion at the site. The average lateral speed during lane change was 2.7 fps, consistent with the literature, with several freeway–freeway and ramp–ramp trajectories showing speeds up to 7.7 fps. All ramp-to-freeway vehicles executed their first mandatory lane change within 62.5% of the total weaving length, although other weaving lane changes were spread over the entire segment. These findings can be useful for implementing strategies to lessen abrupt and excessive lane changes through better lane pre-positioning.

scholarly journals End-to-End Deep Neural Network Architectures for Speed and Steering Wheel Angle Prediction in Autonomous Driving

Electronics ◽  
2021 ◽  
Vol 10 (11) ◽  
pp. 1266
Author(s):  
Pedro J. Navarro ◽  
Leanne Miller ◽  
Francisca Rosique ◽  
Carlos Fernández-Isla ◽  
Alberto Gila-Navarro
Keyword(s):  
Autonomous Vehicles ◽  
High Performance ◽  
Autonomous Driving ◽  
Optimization Process ◽  
Mixed Data ◽  
Steering Wheel ◽  
Driver Assistance Systems ◽  
Convolutional Networks ◽  
Perception System ◽  
End To End

The complex decision-making systems used for autonomous vehicles or advanced driver-assistance systems (ADAS) are being replaced by end-to-end (e2e) architectures based on deep-neural-networks (DNN). DNNs can learn complex driving actions from datasets containing thousands of images and data obtained from the vehicle perception system. This work presents the classification, design and implementation of six e2e architectures capable of generating the driving actions of speed and steering wheel angle directly on the vehicle control elements. The work details the design stages and optimization process of the convolutional networks to develop six e2e architectures. In the metric analysis the architectures have been tested with different data sources from the vehicle, such as images, XYZ accelerations and XYZ angular speeds. The best results were obtained with a mixed data e2e architecture that used front images from the vehicle and angular speeds to predict the speed and steering wheel angle with a mean error of 1.06%. An exhaustive optimization process of the convolutional blocks has demonstrated that it is possible to design lightweight e2e architectures with high performance more suitable for the final implementation in autonomous driving.

Digital economy: conceptual and strategic aspects of development

2020 ◽  
pp. 40-52
Author(s):  
D. Volkov
Keyword(s):  
Public Administration ◽  
Digital Technologies ◽  
Digital Economy ◽  
The State ◽  
Long Wave ◽  
End To End ◽  
Advanced Development ◽  
Digital Sphere

The article proves the need to "return" the state to the economy in order to implement digital mobilization and form a new mechanism of public administration, including the article analyzes the key conditions for Russia’s transition to the path of "advanced development", reveals not only the content of the levels of the digital sphere, but also its end-to-end digital technologies, all the challenges and threats generated by the development of the digital economy, examines the need and possibility of Russia’s movement to the sixth technological order, provides an algorithm for the transition to the phase of a new long wave (the big or Kondratiev cycle).

Vision-Based Navigation of Autonomous Vehicles in Roadway Environments with Unexpected Hazards

2019 ◽  
Vol 2673 (12) ◽  
pp. 494-507 ◽  
Author(s):  
Mhafuzul Islam ◽  
Mashrur Chowdhury ◽  
Hongda Li ◽  
Hongxin Hu
Keyword(s):  
Object Detection ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Semantic Segmentation ◽  
Steering Wheel ◽  
Potential Hazard ◽  
Driving System ◽  
Hazardous Object ◽  
Vision Based Navigation ◽  
Navigational System

Vision-based navigation of autonomous vehicles primarily depends on the deep neural network (DNN) based systems in which the controller obtains input from sensors/detectors, such as cameras, and produces a vehicle control output, such as a steering wheel angle to navigate the vehicle safely in a roadway traffic environment. Typically, these DNN-based systems in the autonomous vehicle are trained through supervised learning; however, recent studies show that a trained DNN-based system can be compromised by perturbation or adverse inputs. Similarly, this perturbation can be introduced into the DNN-based systems of autonomous vehicles by unexpected roadway hazards, such as debris or roadblocks. In this study, we first introduce a hazardous roadway environment that can compromise the DNN-based navigational system of an autonomous vehicle, and produce an incorrect steering wheel angle, which could cause crashes resulting in fatality or injury. Then, we develop a DNN-based autonomous vehicle driving system using object detection and semantic segmentation to mitigate the adverse effect of this type of hazard, which helps the autonomous vehicle to navigate safely around such hazards. We find that our developed DNN-based autonomous vehicle driving system, including hazardous object detection and semantic segmentation, improves the navigational ability of an autonomous vehicle to avoid a potential hazard by 21% compared with the traditional DNN-based autonomous vehicle driving system.

Hidden Markov Model of Lane-Changing-Based Car-Following Behavior on Freeways using Naturalistic Driving Data

2021 ◽  
pp. 036119812199938
Author(s):  
Li Zhao ◽  
Laurence Rilett ◽  
Mm Shakiul Haque
Keyword(s):  
Markov Model ◽  
Hidden Markov Model ◽  
Hidden Markov ◽  
Gaussian Mixture ◽  
Vehicle Body ◽  
Lane Change ◽  
Lane Changing ◽  
Car Following ◽  
Pilot Model ◽  
Naturalistic Driving

This paper develops a methodology for simultaneously modeling lane-changing and car-following behavior of automated vehicles on freeways. Naturalistic driving data from the Safety Pilot Model Deployment (SPMD) program are used. First, a framework to process the SPMD data is proposed using various data analytics techniques including data fusion, data mining, and machine learning. Second, pairs of automated host vehicle and their corresponding front vehicle are identified along with their lane-change and car-following relationship data. Using these data, a lane-changing-based car-following (LCCF) model, which explicitly considers lane-change and car-following behavior simultaneously, is developed. The LCCF model is based on Gaussian-mixture-based hidden Markov model theory and is disaggregated into two processes: LCCF association and LCCF dissociation. These categories are based on the result of the lane change. The overall goal is to predict a driver’s lane-change intention using the LCCF model. Results show that the model can predict the lane-change event in the order of 0.6 to 1.3 s before the moment of the vehicle body across the lane boundary. In addition, the execution times of lane-change maneuvers average between 0.55 and 0.86 s. The LCCF model allows the intention time and execution time of driver’s lane-change behavior to be forecast, which will help to develop better advanced driver assistance systems for vehicle controls with respect to lane-change and car-following warning functions.

scholarly journals Automatic Lane-Changing Decision Based on Single-Step Dynamic Game with Incomplete Information and Collision-Free Path Planning

Actuators ◽  
2021 ◽  
Vol 10 (8) ◽  
pp. 173
Author(s):  
Hongbo Wang ◽  
Shihan Xu ◽  
Longze Deng
Keyword(s):  
Path Planning ◽  
Dynamic Game ◽  
Single Step ◽  
Bézier Curve ◽  
Vehicle Safety ◽  
Bezier Curve ◽  
Lane Change ◽  
Hardware In The Loop ◽  
Power Performance ◽  
Lane Changing

Traffic accidents are often caused by improper lane changes. Although the safety of lane-changing has attracted extensive attention in the vehicle and traffic fields, there are few studies considering the lateral comfort of vehicle users in lane-changing decision-making. Lane-changing decision-making by single-step dynamic game with incomplete information and path planning based on Bézier curve are proposed in this paper to coordinate vehicle lane-changing performance from safety payoff, velocity payoff, and comfort payoff. First, the lane-changing safety distance which is improved by collecting lane-changing data through simulated driving, and lane-changing time obtained by Bézier curve path planning are introduced into the game payoff, so that the selection of the lane-changing start time considers the vehicle safety, power performance and passenger comfort of the lane-changing process. Second, the lane-changing path without collision to the forward vehicle is obtained through the constrained Bézier curve, and the Bézier curve is further constrained to obtain a smoother lane-changing path. The path tracking sliding mode controller of front wheel angle compensation by radical basis function neural network is designed. Finally, the model in the loop simulation and the hardware in the loop experiment are carried out to verify the advantages of the proposed method. The results of three lane-changing conditions designed in the hardware in the loop experiment show that the vehicle safety, power performance, and passenger comfort of the vehicle controlled by the proposed method are better than that of human drivers in discretionary lane change and mandatory lane change scenarios.

Sign in / Sign up

Export Citation Format

Share Document