Advanced Double Layered Multi-Agent Systems Based on A3C in Real-Time Path Planning

In this paper, we propose an advanced double layered multi-agent system to reduce learning time, expressing a state space using a 2D grid. This system is based on asynchronous advantage actor-critic systems (A3C) and reduces the state space that agents need to consider by hierarchically expressing a 2D grid space and determining actions. Specifically, the state space is expressed in the upper and lower layers. Based on the learning results using A3C in the lower layer, the upper layer makes decisions without additional learning, and accordingly, the total learning time can be reduced. Our method was verified experimentally using a virtual autonomous surface vehicle simulator. It reduced the learning time required to reach a 90% goal achievement rate by 7.1% compared to the conventional double layered A3C. In addition, the goal achievement by the proposed method was 18.86% higher than that of the traditional double layered A3C over 20,000 learning episodes.

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

This paper presents a framework to alleviate the Deep Reinforcement Learning (DRL) training data sparsity problem that is present in challenging domains by creating a DRL agent training and vehicle integration methodology. The methodology leverages accessible domains to train an agent to solve navigational problems such as obstacle avoidance and allows the agent to generalize to challenging and inaccessible domains such as those present in marine environments with minimal further training. This is done by integrating a DRL agent at a high level of vehicle control and leveraging existing path planning and proven low-level control methodologies that are utilized in multiple domains. An autonomy package with a tertiary multilevel controller is developed to enable the DRL agent to interface at the prescribed high control level and thus be separated from vehicle dynamics and environmental constraints. An example Deep Q Network (DQN) employing this methodology for obstacle avoidance is trained in a simulated ground environment, and then its ability to generalize across domains is experimentally validated. Experimental validation utilized a simulated water surface environment and real-world deployment of ground and water robotic platforms. This methodology, when used, shows that it is possible to leverage accessible and data rich domains, such as ground, to effectively develop marine DRL agents for use on Autonomous Surface Vehicle (ASV) navigation. This will allow rapid and iterative agent development without the risk of ASV loss, the cost and logistic overhead of marine deployment, and allow landlocked institutions to develop agents for marine applications.

Experiment of Path Planning Using Multi-way Points with A* Algorithm for Autonomous Surface Vehicle

Commonly, surveillance activities on lake waters is mostly carried out by using a surface vehicle as special-designed vehicle, especially to conduct water quality measurements, underwater surveys, and bathymetry mapping. However, conventional survey and monitoring still involves humans on the site. If a survey is conducted during strong wind conditions, it could jeopardize surveyor’s safety. Therefore, a vehicle must have several criteria, e.g., it must be pretty spacious and comfortable to carry surveyors, free from engine vibrations, stabilized and easy to maneuver, and the surveyor's safety can be guaranteed. This paper discusses preliminary research aiming to develop an Autonomous Raft Vehicle (ARV), a type of autonomous unmanned surface vehicle. The ARV is equipped with autonomous control based on multi-way-points with an A* algorithm. Thus, a user only requires giving a command once initially during path planning. A* algorithm over multi-way-point could improve ARV navigation when there are obstacles along the predetermined trajectory. Hence the predetermined trajectory will be maintained throughout the mission. It is a significant contribution to this paper.