GPS-Denied Three Dimensional Leader-Follower Formation Control Using Deep Reinforcement Learning

2022 ◽  
Author(s):  
Robert A. Selje ◽  
Amer Al-Radaideh ◽  
Rajdeep Dutta ◽  
Liang Sun
Keyword(s):  
Reinforcement Learning ◽  
Formation Control ◽  
Three Dimensional

scholarly journals Reinforcement-Learning-Based Asynchronous Formation Control Scheme for Multiple Unmanned Surface Vehicles

2021 ◽  
Vol 11 (2) ◽  
pp. 546
Author(s):  
Jiajia Xie ◽  
Rui Zhou ◽  
Yuan Liu ◽  
Jun Luo ◽  
Shaorong Xie ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Formation Control ◽  
Rapid Development ◽  
Gradient Algorithm ◽  
Robot System ◽  
Physical Relationship ◽  
Unmanned Surface Vehicles ◽  
Main Challenge ◽  
Control Scheme ◽  
Multi Robot

The high performance and efficiency of multiple unmanned surface vehicles (multi-USV) promote the further civilian and military applications of coordinated USV. As the basis of multiple USVs’ cooperative work, considerable attention has been spent on developing the decentralized formation control of the USV swarm. Formation control of multiple USV belongs to the geometric problems of a multi-robot system. The main challenge is the way to generate and maintain the formation of a multi-robot system. The rapid development of reinforcement learning provides us with a new solution to deal with these problems. In this paper, we introduce a decentralized structure of the multi-USV system and employ reinforcement learning to deal with the formation control of a multi-USV system in a leader–follower topology. Therefore, we propose an asynchronous decentralized formation control scheme based on reinforcement learning for multiple USVs. First, a simplified USV model is established. Simultaneously, the formation shape model is built to provide formation parameters and to describe the physical relationship between USVs. Second, the advantage deep deterministic policy gradient algorithm (ADDPG) is proposed. Third, formation generation policies and formation maintenance policies based on the ADDPG are proposed to form and maintain the given geometry structure of the team of USVs during movement. Moreover, three new reward functions are designed and utilized to promote policy learning. Finally, various experiments are conducted to validate the performance of the proposed formation control scheme. Simulation results and contrast experiments demonstrate the efficiency and stability of the formation control scheme.

scholarly journals UAV Formation Shape Control via Decentralized Markov Decision Processes

Algorithms ◽  
2021 ◽  
Vol 14 (3) ◽  
pp. 91
Author(s):  
Md Ali Azam ◽  
Hans D. Mittelmann ◽  
Shankarachary Ragi
Keyword(s):  
Control Problem ◽  
Shape Control ◽  
Formation Control ◽  
Three Dimensional ◽  
Geographical Region ◽  
Dynamic Programming Method ◽  
Theoretic Approach ◽  
Control Approach ◽  
Uav Swarm ◽  
Markov Decision

In this paper, we present a decentralized unmanned aerial vehicle (UAV) swarm formation control approach based on a decision theoretic approach. Specifically, we pose the UAV swarm motion control problem as a decentralized Markov decision process (Dec-MDP). Here, the goal is to drive the UAV swarm from an initial geographical region to another geographical region where the swarm must form a three-dimensional shape (e.g., surface of a sphere). As most decision-theoretic formulations suffer from the curse of dimensionality, we adapt an existing fast approximate dynamic programming method called nominal belief-state optimization (NBO) to approximately solve the formation control problem. We perform numerical studies in MATLAB to validate the performance of the above control algorithms.

scholarly journals 3D cephalometric landmark detection by multiple stage deep reinforcement learning

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Sung Ho Kang ◽  
Kiwan Jeon ◽  
Sang-Hoon Kang ◽  
Sang-Hwy Lee
Keyword(s):  
Reinforcement Learning ◽  
Three Dimensional ◽  
Detection Accuracy ◽  
Sequential Decision ◽  
Landmark Detection ◽  
Boundary Estimation ◽  
Multi Stage ◽  
Sequential Decision Process ◽  
Gradient Based ◽  
3D Cephalometry

AbstractThe lengthy time needed for manual landmarking has delayed the widespread adoption of three-dimensional (3D) cephalometry. We here propose an automatic 3D cephalometric annotation system based on multi-stage deep reinforcement learning (DRL) and volume-rendered imaging. This system considers geometrical characteristics of landmarks and simulates the sequential decision process underlying human professional landmarking patterns. It consists mainly of constructing an appropriate two-dimensional cutaway or 3D model view, then implementing single-stage DRL with gradient-based boundary estimation or multi-stage DRL to dictate the 3D coordinates of target landmarks. This system clearly shows sufficient detection accuracy and stability for direct clinical applications, with a low level of detection error and low inter-individual variation (1.96 ± 0.78 mm). Our system, moreover, requires no additional steps of segmentation and 3D mesh-object construction for landmark detection. We believe these system features will enable fast-track cephalometric analysis and planning and expect it to achieve greater accuracy as larger CT datasets become available for training and testing.

Sign in / Sign up

Export Citation Format

Share Document