End-to-End Autonomous Exploration with Deep Reinforcement Learning and Intrinsic Motivation

Developing artificial intelligence (AI) agents is challenging for efficient exploration in visually rich and complex environments. In this study, we formulate the exploration question as a reinforcement learning problem and rely on intrinsic motivation to guide exploration behavior. Such intrinsic motivation is driven by curiosity and is calculated based on episode memory. To distribute the intrinsic motivation, we use a count-based method and temporal distance to generate it synchronously. We tested our approach in 3D maze-like environments and validated its performance in exploration tasks through extensive experiments. The experimental results show that our agent can learn exploration ability from raw sensory input and accomplish autonomous exploration across different mazes. In addition, the learned policy is not biased by stochastic objects. We also analyze the effects of different training methods and driving forces on exploration policy.

Efficient and Safe Robotic Autonomous Environment Exploration Using Integrated Frontier Detection and Multiple Path Evaluation

Autonomous exploration and remote sensing using robots have gained increasing attention in recent years and aims to maximize information collection regarding the external world without human intervention. However, incomplete frontier detection, an inability to eliminate inefficient frontiers, and incomplete evaluation limit further improvements in autonomous exploration efficiency. This article provides a systematic solution for ground mobile robot exploration with high efficiency. Firstly, an integrated frontier detection and maintenance method is proposed, which incrementally discovers potential frontiers and achieves incremental maintenance of the safe and informative frontiers by updating the distance map locally. Secondly, we propose a novel multiple paths planning method to generate multiple paths from the robot position to the unexplored frontiers. Then, we use the proposed utility function to select the optimal path and improve its smoothness using an iterative optimization strategy. Ultimately, the model predictive control (MPC) method is applied to track the smooth path. Simulation experiments on typical environments demonstrate that compared with the benchmark methods, the proposed method reduce the path length by 27.07% and the exploration time by 27.09% on average. The real-world experimental results also reveal that our proposed method can achieve complete mapping with fewer repetitive paths.

Considering Autonomous Exploration in Healthy Environments: Reflections from an Urban Wildscape

Autonomous exploration should be considered in the creation of healthy environments since autonomy is an important developmental experience for children. For a group of boys in Raleigh, N.C., U.S. during the period 2002–2006, autonomous exploration was a meaningful experience. Results of a qualitative research project (n = 5) which highlight the importance of autonomous exploration are organized within a proposed framework for thick description. The framework creates verisimilitude by reporting on the context, social action and cultural context, and behavior and intentionality. The context of Raleigh and urban wildscapes furnished areas ripe for exploration. The social action and cultural context of attachment supported the autonomous exploration through scaffolded experiences of autonomy. The intentionality of the behavior was a desire to distinct themselves through a focus on individual development and the pursuit of extraordinary experiences. The ultimate outcomes of autonomous exploration for the boys were the development of long-term, intimate friendships and confidence in their decision-making ability. As cities become more health-focused, attention should be paid to preserve the rough edges of a city for children to explore.

Efficient and High Path Quality Autonomous Exploration and Trajectory Planning of UAV in an Unknown Environment

The ability of an autonomous Unmanned Aerial Vehicle (UAV) in an unknown environment is a prerequisite for its execution of complex tasks and is the main research direction in related fields. The autonomous navigation of UAVs in unknown environments requires solving the problem of autonomous exploration of the surrounding environment and path planning, which determines whether the drones can complete mission-based flights safely and efficiently. Existing UAV autonomous flight systems hardly perform well in terms of efficient exploration and flight trajectory quality. This paper establishes an integrated solution for autonomous exploration and path planning. In terms of autonomous exploration, frontier-based and sampling-based exploration strategies are integrated to achieve fast and effective exploration performance. In the study of path planning in complex environments, an advanced Rapidly Exploring Random Tree (RRT) algorithm combining the adaptive weights and dynamic step size is proposed, which effectively solves the problem of balancing flight time and trajectory quality. Then, this paper uses the Hermite difference polynomial to optimization the trajectory generated by the RRT algorithm. We named proposed UAV autonomous flight system as Frontier and Sampling-based Exploration and Advanced RRT Planner system (FSEPlanner). Simulation performs in both apartment and maze environment, and results show that the proposed FSEPlanner algorithm achieves greatly improved time consumption and path distances, and the smoothed path is more in line with the actual flight needs of a UAV.

FAST RRT* 3D-Sliced Planner for Autonomous Exploration Using MAVs

This paper addresses the challenge to build an autonomous exploration system using Micro-Aerial Vehicles (MAVs). MAVs are capable of flying autonomously, generating collision-free paths to navigate in unknown areas and also reconstructing the environment at which they are deployed. One of the contributions of our system is the “3D-Sliced Planner” for exploration. The main innovation is the low computational resources needed. This is because Optimal-Frontier-Points (OFP) to explore are computed in 2D slices of the 3D environment using a global Rapidly-exploring Random Tree (RRT) frontier detector. Then, the MAV can plan path routes to these points to explore the surroundings with our new proposed local “FAST RRT* Planner” that uses a tree reconnection algorithm based on cost, and a collision checking algorithm based on Signed Distance Field (SDF). The results show the proposed explorer takes 43.95% less time to compute exploration points and paths when compared with the State-of-the-Art represented by the Receding Horizon Next Best View Planner (RH-NBVP) in Gazebo simulations.

Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph

In this paper, we address the problem of autonomous exploration in unknown environments for ground mobile robots with deep reinforcement learning (DRL). To effectively explore unknown environments, we construct an exploration graph considering historical trajectories, frontier waypoints, landmarks, and obstacles. Meanwhile, to take full advantage of the spatiotemporal feature and historical information in the autonomous exploration task, we propose a novel network called Spatiotemporal Neural Network on Graph (Graph-STNN). Specifically, the proposed Graph-STNN extracts the spatial feature using graph convolutional network (GCN) and the temporal feature using temporal convolutional network (TCN). Then, gated recurrent unit (GRU) is performed to synthesize the spatial feature, the temporal feature, and the historical state information into the current state feature. Combined with DRL, our Graph-STNN helps estimation of the optimal target point through extracted hybrid features. The simulation experiment shows that our approach is more effective than the GCN-based approach and the information entropy-based approach. Moreover, Graph-STNN also performs better generalization ability than GCN-based, information entropy-based, and random methods. Finally, we validate our approach on the simulation platform Stage with the actual robot model.

From ERL to MBZIRC: Development of An Aerial-Ground Robotic Team for Search and Rescue

This chapter describes the efforts of the LARICS team in the 2019 European Robotics League (ERL) Emergency Robots and the 2020 Mohamed Bin Zayed International Robotics Challenge (MBZIRC) robotics competitions. We focus on the implementation of hardware and software modules that enable the deployment of aerial-ground robotic teams in unstructured environments for joint missions. In addition to the overall system specification, we outline the main algorithms for operation in such conditions: autonomous exploration of unknown environments and detection of objects of interest. Analysis of the results shows the success of the developed system in the competition arena of two of the largest outdoor robotics challenges. Throughout the chapter, we highlight the evolution of the robotic system based on the experience gained in the ERL competition. We conclude the chapter with key findings and additional improvement ideas to advance the state of the art in search and rescue applications of heterogeneous robotic teams.