Stochastic optimal enhancement of distributed formation control using Kalman smoothers

Robotica ◽  
2014 ◽  
Vol 32 (2) ◽  
pp. 305-324 ◽  
Author(s):  
Ross P. Anderson ◽  
Dejan Milutinović
Keyword(s):  
Formation Control ◽  
Optimal Controls ◽  
Control Approach ◽  
Path Integral Representation ◽  
Continuous State Space ◽  
Continuous State ◽  
Hamilton Jacobi Bellman Equation ◽  
Additive Correction ◽  
Hamilton Jacobi Bellman ◽  
Correction Terms

SUMMARYBeginning with a deterministic distributed feedback control for nonholonomic vehicle formations, we develop a stochastic optimal control approach for agents to enhance their non-optimal controls with additive correction terms based on the Hamilton–Jacobi–Bellman equation, making them optimal and robust to uncertainties. In order to avoid discretization of the high-dimensional cost-to-go function, we exploit the stochasticity of the distributed nature of the problem to develop an equivalent Kalman smoothing problem in a continuous state space using a path integral representation. Our approach is illustrated by numerical examples in which agents achieve a formation with their neighbors using only local observations.

Necessary and sufficient conditions for optimal controls in viscous flow problems

1994 ◽  
Vol 124 (2) ◽  
pp. 211-251 ◽  
Author(s):  
H. O. Fattorini ◽  
S. S. Sritharan
Keyword(s):  
Maximum Principle ◽  
Viscous Flow ◽  
Sufficient Conditions ◽  
Optimal Controls ◽  
The Maximum Principle ◽  
Flow Problems ◽  
Hamilton Jacobi Bellman Equation ◽  
Hamilton Jacobi Bellman ◽  
Necessary And Sufficient ◽  
The Pontryagin Maximum Principle

A class of optimal control problems in viscous flow is studied. Main results are the Pontryagin maximum principle and the verification theorem for the Hamilton–Jacobi–Bellman equation characterising the feedback problem. The maximum principle is established by two quite different methods.

scholarly journals A Hybrid Approach for Coordinated Formation Control of Multiple Surface Vessels

2013 ◽  
Vol 2013 ◽  
pp. 1-8 ◽  
Author(s):  
Mingyu Fu ◽  
Jianfang Jiao
Keyword(s):  
Formation Control ◽  
Control Method ◽  
Hybrid Control ◽  
Weighting Function ◽  
Discrete Event ◽  
Hybrid Approach ◽  
Path Following ◽  
Coordination Control ◽  
Control Approach ◽  
Continuous State

This paper investigates the coordination control of multiple marine vessels in different operational modes. Based on hybrid control theory, a novel coordinated formation control approach is proposed. The proposed method comprises several continuous state controllers and discrete event logics. Continuous controllers for coordinated formation, coordinated dynamic positioning and coordinated path following are designed, and an appropriate weighting function is given to switch between these controllers according to initiated commands. In order to ensure the security of coordination operations of vessels in arbitrary initial locations, the supervisory switching control method is employed in the integrated coordination control system where all the controllers are governed by a supervisor. The effectiveness of the proposed coordinated formation control approach is finally illustrated by simulations.

scholarly journals UAV Formation Shape Control via Decentralized Markov Decision Processes

Algorithms ◽  
2021 ◽  
Vol 14 (3) ◽  
pp. 91
Author(s):  
Md Ali Azam ◽  
Hans D. Mittelmann ◽  
Shankarachary Ragi
Keyword(s):  
Control Problem ◽  
Shape Control ◽  
Formation Control ◽  
Three Dimensional ◽  
Geographical Region ◽  
Dynamic Programming Method ◽  
Theoretic Approach ◽  
Control Approach ◽  
Uav Swarm ◽  
Markov Decision

In this paper, we present a decentralized unmanned aerial vehicle (UAV) swarm formation control approach based on a decision theoretic approach. Specifically, we pose the UAV swarm motion control problem as a decentralized Markov decision process (Dec-MDP). Here, the goal is to drive the UAV swarm from an initial geographical region to another geographical region where the swarm must form a three-dimensional shape (e.g., surface of a sphere). As most decision-theoretic formulations suffer from the curse of dimensionality, we adapt an existing fast approximate dynamic programming method called nominal belief-state optimization (NBO) to approximately solve the formation control problem. We perform numerical studies in MATLAB to validate the performance of the above control algorithms.

scholarly journals Reinforcement learning versus swarm intelligence for autonomous multi-HAPS coordination

2021 ◽  
Vol 3 (6) ◽  
Author(s):  
Ogbonnaya Anicho ◽  
Philip B. Charlesworth ◽  
Gurvinder S. Baicher ◽  
Atulya K. Nagar
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Swarm Intelligence ◽  
Performance Indicators ◽  
Convergence Rates ◽  
Tuning Parameters ◽  
Continuous State Space ◽  
Continuous State ◽  
User Coverage ◽  
Better Than

AbstractThis work analyses the performance of Reinforcement Learning (RL) versus Swarm Intelligence (SI) for coordinating multiple unmanned High Altitude Platform Stations (HAPS) for communications area coverage. It builds upon previous work which looked at various elements of both algorithms. The main aim of this paper is to address the continuous state-space challenge within this work by using partitioning to manage the high dimensionality problem. This enabled comparing the performance of the classical cases of both RL and SI establishing a baseline for future comparisons of improved versions. From previous work, SI was observed to perform better across various key performance indicators. However, after tuning parameters and empirically choosing suitable partitioning ratio for the RL state space, it was observed that the SI algorithm still maintained superior coordination capability by achieving higher mean overall user coverage (about 20% better than the RL algorithm), in addition to faster convergence rates. Though the RL technique showed better average peak user coverage, the unpredictable coverage dip was a key weakness, making SI a more suitable algorithm within the context of this work.

On coupling of random walks and renewal processes

1996 ◽  
Vol 33 (1) ◽  
pp. 122-126
Author(s):  
Torgny Lindvall ◽  
L. C. G. Rogers
Keyword(s):  
Random Walk ◽  
Random Walks ◽  
Simple Random Walk ◽  
Renewal Processes ◽  
Renewal Theorem ◽  
One Dimensional ◽  
Continuous State Space ◽  
Continuous State ◽  
Efficient Coupling ◽  
Blackwell's Renewal Theorem

The use of Mineka coupling is extended to a case with a continuous state space: an efficient coupling of random walks S and S' in can be made such that S' — S is virtually a one-dimensional simple random walk. This insight settles a zero-two law of ergodicity. One more proof of Blackwell's renewal theorem is also presented.

Interception of automated adversarial drone swarms in partially observed environments

2021 ◽  
pp. 1-14
Author(s):  
Daniel Saranovic ◽  
Martin Pavlovski ◽  
William Power ◽  
Ivan Stojkovic ◽  
Zoran Obradovic
Keyword(s):  
Bellman Equation ◽  
Pid Controllers ◽  
Partially Observed ◽  
Hamilton Jacobi Bellman Equation ◽  
Partial Feedback ◽  
Point Solution ◽  
Physical Limitations ◽  
Defensive Mechanisms ◽  
Hamilton Jacobi Bellman ◽  
New Framework

As the prevalence of drones increases, understanding and preparing for possible adversarial uses of drones and drone swarms is of paramount importance. Correspondingly, developing defensive mechanisms in which swarms can be used to protect against adversarial Unmanned Aerial Vehicles (UAVs) is a problem that requires further attention. Prior work on intercepting UAVs relies mostly on utilizing additional sensors or uses the Hamilton-Jacobi-Bellman equation, for which strong conditions need to be met to guarantee the existence of a saddle-point solution. To that end, this work proposes a novel interception method that utilizes the swarm’s onboard PID controllers for setting the drones’ states during interception. The drone’s states are constrained only by their physical limitations, and only partial feedback of the adversarial drone’s positions is assumed. The new framework is evaluated in a virtual environment under different environmental and model settings, using random simulations of more than 165,000 swarm flights. For certain environmental settings, our results indicate that the interception performance of larger swarms under partial observation is comparable to that of a one-drone swarm under full observation of the adversarial drone.

scholarly journals Instantaneous brain dynamics mapped to a continuous state space

NeuroImage ◽  
2017 ◽  
Vol 162 ◽  
pp. 344-352 ◽  
Author(s):  
Jacob C.W. Billings ◽  
Alessio Medda ◽  
Sadia Shakil ◽  
Xiaohong Shen ◽  
Amrit Kashyap ◽  
...  
Keyword(s):  
State Space ◽  
Brain Dynamics ◽  
Continuous State Space ◽  
Continuous State

scholarly journals Optimal Portfolios for Financial Markets with Wishart Volatility

2013 ◽  
Vol 50 (4) ◽  
pp. 1025-1043 ◽  
Author(s):  
Nicole Bäuerle ◽  
Zejing Li
Keyword(s):  
Value Function ◽  
Numerical Study ◽  
Heston Model ◽  
Hard Part ◽  
Control Approach ◽  
Portfolio Strategy ◽  
Hamilton Jacobi Bellman ◽  
The One ◽  
Hedging Demand ◽  
The Value Function

We consider a multi asset financial market with stochastic volatility modeled by a Wishart process. This is an extension of the one-dimensional Heston model. Within this framework we study the problem of maximizing the expected utility of terminal wealth for power and logarithmic utility. We apply the usual stochastic control approach and obtain, explicitly, the optimal portfolio strategy and the value function in some parameter settings. In particular, we do this when the drift of the assets is a linear function of the volatility matrix. In this case the affine structure of the model can be exploited. In some cases we obtain a Feynman-Kac representation of the candidate value function. Though the approach we use is quite standard, the hard part is to identify when the solution of the Hamilton-Jacobi-Bellman equation is finite. This involves a couple of matrix analytic arguments. In a numerical study we discuss the influence of the investors' risk aversion on the hedging demand.

Sign in / Sign up

Export Citation Format

Share Document