Deep Reinforcement Learning for Quadrotor Path Following and Obstacle Avoidance

2021 ◽  
pp. 563-633
Author(s):  
Bartomeu Rubí ◽  
Bernardo Morcego ◽  
Ramon Pérez
Keyword(s):  
Reinforcement Learning ◽  
Obstacle Avoidance ◽  
Path Following

scholarly journals Quadrotor Path Following and Reactive Obstacle Avoidance with Deep Reinforcement Learning

2021 ◽  
Vol 103 (4) ◽  
Author(s):  
Bartomeu Rubí ◽  
Bernardo Morcego ◽  
Ramon Pérez
Keyword(s):  
Reinforcement Learning ◽  
Obstacle Avoidance ◽  
Low Cost ◽  
Path Following ◽  
The State ◽  
Gradient Algorithm ◽  
Avoidance Task ◽  
Learning Approaches ◽  
Reward Function ◽  
Novel Structure

AbstractA deep reinforcement learning approach for solving the quadrotor path following and obstacle avoidance problem is proposed in this paper. The problem is solved with two agents: one for the path following task and another one for the obstacle avoidance task. A novel structure is proposed, where the action computed by the obstacle avoidance agent becomes the state of the path following agent. Compared to traditional deep reinforcement learning approaches, the proposed method allows to interpret the training process outcomes, is faster and can be safely trained on the real quadrotor. Both agents implement the Deep Deterministic Policy Gradient algorithm. The path following agent was developed in a previous work. The obstacle avoidance agent uses the information provided by a low-cost LIDAR to detect obstacles around the vehicle. Since LIDAR has a narrow field-of-view, an approach for providing the agent with a memory of the previously seen obstacles is developed. A detailed description of the process of defining the state vector, the reward function and the action of this agent is given. The agents are programmed in python/tensorflow and are trained and tested in the RotorS/gazebo platform. Simulations results prove the validity of the proposed approach.

Modeling human-like longitudinal driver model for intelligent vehicles based on reinforcement learning

2021 ◽  
pp. 095440702098357
Author(s):  
Ju Xie ◽  
Xing Xu ◽  
Feng Wang ◽  
Haobin Jiang
Keyword(s):  
Reinforcement Learning ◽  
Comprehensive Evaluation ◽  
Path Following ◽  
Intelligent Vehicles ◽  
Driver Model ◽  
Control Center ◽  
Training Performance ◽  
Learning Agents ◽  
System A ◽  
And Control

The driver model is the decision-making and control center of intelligent vehicle. In order to improve the adaptability of intelligent vehicles under complex driving conditions, and simulate the manipulation characteristics of the skilled driver under the driver-vehicle-road closed-loop system, a kind of human-like longitudinal driver model for intelligent vehicles based on reinforcement learning is proposed. This paper builds the lateral driver model for intelligent vehicles based on optimal preview control theory. Then, the control correction link of longitudinal driver model is established to calculate the throttle opening or brake pedal travel for the desired longitudinal acceleration. Moreover, the reinforcement learning agents for longitudinal driver model is parallel trained by comprehensive evaluation index and skilled driver data. Lastly, training performance and scenarios verification between the simulation experiment and the real car test are performed to verify the effectiveness of the reinforcement learning based longitudinal driver model. The results show that the proposed human-like longitudinal driver model based on reinforcement learning can help intelligent vehicles effectively imitate the speed control behavior of the skilled driver in various path-following scenarios.

Robot obstacle avoidance system using deep reinforcement learning

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Xiaojun Zhu ◽  
Yinghao Liang ◽  
Hanxu Sun ◽  
Xueqian Wang ◽  
Bin Ren
Keyword(s):  
Reinforcement Learning ◽  
Collision Avoidance ◽  
Obstacle Avoidance ◽  
Learning Algorithm ◽  
Optimal Path ◽  
Environmental Parameters ◽  
Working Environment ◽  
Content Type ◽  
Practical Applications ◽  
Human Operators

Purpose Most manufacturing plants choose the easy way of completely separating human operators from robots to prevent accidents, but as a result, it dramatically affects the overall quality and speed that is expected from human–robot collaboration. It is not an easy task to ensure human safety when he/she has entered a robot’s workspace, and the unstructured nature of those working environments makes it even harder. The purpose of this paper is to propose a real-time robot collision avoidance method to alleviate this problem. Design/methodology/approach In this paper, a model is trained to learn the direct control commands from the raw depth images through self-supervised reinforcement learning algorithm. To reduce the effect of sample inefficiency and safety during initial training, a virtual reality platform is used to simulate a natural working environment and generate obstacle avoidance data for training. To ensure a smooth transfer to a real robot, the automatic domain randomization technique is used to generate randomly distributed environmental parameters through the obstacle avoidance simulation of virtual robots in the virtual environment, contributing to better performance in the natural environment. Findings The method has been tested in both simulations with a real UR3 robot for several practical applications. The results of this paper indicate that the proposed approach can effectively make the robot safety-aware and learn how to divert its trajectory to avoid accidents with humans within the workspace. Research limitations/implications The method has been tested in both simulations with a real UR3 robot in several practical applications. The results indicate that the proposed approach can effectively make the robot be aware of safety and learn how to change its trajectory to avoid accidents with persons within the workspace. Originality/value This paper provides a novel collision avoidance framework that allows robots to work alongside human operators in unstructured and complex environments. The method uses end-to-end policy training to directly extract the optimal path from the visual inputs for the scene.

Sign in / Sign up

Export Citation Format

Share Document