scholarly journals Optimal Stabilization of Linear Stochastic System with Statistically Uncertain Piecewise Constant Drift

Mathematics ◽  
2022 ◽  
Vol 10 (2) ◽  
pp. 184
Author(s):  
Andrey Borisov ◽  
Alexey Bosov ◽  
Gregory Miller
Keyword(s):  
Control Problem ◽  
Jump Process ◽  
Complete Information ◽  
Dynamic Programming Method ◽  
Markov Jump ◽  
Optimal Stabilization ◽  
Piecewise Constant ◽  
Equivalent Control ◽  
Partially Observable ◽  
Theoretical Results

The paper presents an optimal control problem for the partially observable stochastic differential system driven by an external Markov jump process. The available controlled observations are indirect and corrupted by some Wiener noise. The goal is to optimize a linear function of the state (output) given a general quadratic criterion. The separation principle, verified for the system at hand, allows examination of the control problem apart from the filter optimization. The solution to the latter problem is provided by the Wonham filter. The solution to the former control problem is obtained by formulating an equivalent control problem with a linear drift/nonlinear diffusion stochastic process and with complete information. This problem, in turn, is immediately solved by the application of the dynamic programming method. The applicability of the obtained theoretical results is illustrated by a numerical example, where an optimal amplification/stabilization problem is solved for an unstable externally controlled step-wise mechanical actuator.

scholarly journals UAV Formation Shape Control via Decentralized Markov Decision Processes

Algorithms ◽  
2021 ◽  
Vol 14 (3) ◽  
pp. 91
Author(s):  
Md Ali Azam ◽  
Hans D. Mittelmann ◽  
Shankarachary Ragi
Keyword(s):  
Control Problem ◽  
Shape Control ◽  
Formation Control ◽  
Three Dimensional ◽  
Geographical Region ◽  
Dynamic Programming Method ◽  
Theoretic Approach ◽  
Control Approach ◽  
Uav Swarm ◽  
Markov Decision

In this paper, we present a decentralized unmanned aerial vehicle (UAV) swarm formation control approach based on a decision theoretic approach. Specifically, we pose the UAV swarm motion control problem as a decentralized Markov decision process (Dec-MDP). Here, the goal is to drive the UAV swarm from an initial geographical region to another geographical region where the swarm must form a three-dimensional shape (e.g., surface of a sphere). As most decision-theoretic formulations suffer from the curse of dimensionality, we adapt an existing fast approximate dynamic programming method called nominal belief-state optimization (NBO) to approximately solve the formation control problem. We perform numerical studies in MATLAB to validate the performance of the above control algorithms.

Emergent Tangled Program Graphs in Partially Observable Recursive Forecasting and ViZDoom Navigation Tasks

2021 ◽  
Vol 1 (3) ◽  
pp. 1-41
Author(s):  
Stephen Kelly ◽  
Robert J. Smith ◽  
Malcolm I. Heywood ◽  
Wolfgang Banzhaf
Keyword(s):  
Empirical Evaluation ◽  
Complete Information ◽  
Visual Navigation ◽  
Sources Of Information ◽  
Variation Operators ◽  
Incremental Construction ◽  
Partially Observable ◽  
First Person Shooter ◽  
With Memory ◽  
Program Graph

Modularity represents a recurring theme in the attempt to scale evolution to the design of complex systems. However, modularity rarely forms the central theme of an artificial approach to evolution. In this work, we report on progress with the recently proposed Tangled Program Graph (TPG) framework in which programs are modules. The combination of the TPG representation and its variation operators enable both teams of programs and graphs of teams of programs to appear in an emergent process. The original development of TPG was limited to tasks with, for the most part, complete information. This work details two recent approaches for scaling TPG to tasks that are dominated by partially observable sources of information using different formulations of indexed memory. One formulation emphasizes the incremental construction of memory, again as an emergent process, resulting in a distributed view of state. The second formulation assumes a single global instance of memory and develops it as a communication medium, thus a single global view of state. The resulting empirical evaluation demonstrates that TPG equipped with memory is able to solve multi-task recursive time-series forecasting problems and visual navigation tasks expressed in two levels of a commercial first-person shooter environment.

scholarly journals Symbolic Algorithm of the Functional-Discrete Method for a Sturm–Liouville Problem with a Polynomial Potential

2018 ◽  
Vol 18 (4) ◽  
pp. 703-715 ◽  
Author(s):  
Volodymyr Makarov ◽  
Nataliia Romaniuk
Keyword(s):  
Finite Interval ◽  
General Scheme ◽  
Liouville Problem ◽  
Piecewise Constant Function ◽  
Polynomial Potential ◽  
Discrete Method ◽  
Piecewise Constant ◽  
Sturm Liouville ◽  
Symbolic Algorithm ◽  
Theoretical Results

AbstractA new symbolic algorithmic implementation of the general scheme of the exponentially convergent functional-discrete method is developed and justified for the Sturm–Liouville problem on a finite interval for the Schrödinger equation with a polynomial potential and the boundary conditions of Dirichlet type. The algorithm of the general scheme of our method is developed when the potential function is approximated by the piecewise-constant function. Our algorithm is symbolic and operates with the decomposition coefficients of the eigenfunction corrections in some basis. The number of summands in these decompositions depends on the degree of the potential polynomial and on the correction number. Our method uses the algebraic operations only and does not need solutions of any boundary value problems and computations of any integrals unlike the previous version. A numerical example illustrates the theoretical results.

scholarly journals A minimal time optimal control for a drone landing problem

2021 ◽  
Author(s):  
Filippo Gazzola ◽  
Elsa Maria Marchini
Keyword(s):  
Optimal Control ◽  
Initial Data ◽  
Optimal Control Problem ◽  
Control Problem ◽  
Aerospace Engineering ◽  
Time Optimal Control ◽  
Height Velocity ◽  
The Moon ◽  
Piecewise Constant ◽  
Time Optimal

We study a variant of the classical safe landing optimal control problem in aerospace engineering, introduced by Miele in 1962, where the target was to land a spacecraft on the moon by minimizing the consumption of fuel. A more modern model consists in replacing the spacecraft by a hybrid gas-electric drone. Assuming that the drone has a failure and that the thrust (representing the control) can act in both vertical directions, the new target is to land safely by minimizing time, no matter of what the consumption is. In dependence of the initial data (height, velocity, and fuel), we prove that the optimal control can be of four different kinds, all being piecewise constant. Our analysis covers all possible situations, including the nonexistence of a safe landing strategy due to the lack of fuel or for heights/velocities for which also a total braking is insufficient to stop the drone.

Nonlinear Stochastic Optimal Control of MDOF Partially Observable Linear Systems Excited by Combined Harmonic and Wide-Band Noises

2019 ◽  
Vol 19 (03) ◽  
pp. 1950019 ◽  
Author(s):  
R. C. Hu ◽  
X. F. Wang ◽  
X. D. Gu ◽  
R. H. Huan
Keyword(s):  
Optimal Control ◽  
Dynamic Programming ◽  
Control Problem ◽  
Linear Systems ◽  
Control Strategy ◽  
Stochastic Optimal Control ◽  
Wide Band ◽  
Dynamic Programming Principle ◽  
Stochastic Dynamic ◽  
Partially Observable

In this paper, nonlinear stochastic optimal control of multi-degree-of-freedom (MDOF) partially observable linear systems subjected to combined harmonic and wide-band random excitations is investigated. Based on the separation principle, the control problem of a partially observable system is converted into a completely observable one. The dynamic programming equation for the completely observable control problem is then set up based on the stochastic averaging method and stochastic dynamic programming principle, from which the nonlinear optimal control law is derived. To illustrate the feasibility and efficiency of the proposed control strategy, the responses of the uncontrolled and optimal controlled systems are respectively obtained by solving the associated Fokker–Planck–Kolmogorov (FPK) equation. Numerical results show the proposed control strategy can dramatically reduce the response of stochastic systems subjected to both harmonic and wide-band random excitations.

PRODUCT-FORM MARKOVIAN QUEUEING SYSTEMS WITH MULTIPLE RESOURCES

2019 ◽  
pp. 1-9 ◽  
Author(s):  
Valeriy Naumov ◽  
Konstantin Samouylov
Keyword(s):  
Queueing Systems ◽  
Sufficient Conditions ◽  
Jump Process ◽  
Product Form ◽  
Temporary Increase ◽  
Stationary Probability Distribution ◽  
Markov Jump ◽  
Multiple Resources ◽  
Resource Requirements ◽  
Necessary And Sufficient

In the paper, we study general Markovian models of loss systems with random resource requirements, in which customers at arrival occupy random quantities of various resources and release them at departure. Customers may request negative quantities of resources, but total amount of resources allocated to customers should be nonnegative and cannot exceed predefined maximum levels. Allocating a negative volume of a resource to a customer leads to a temporary increase in its volume in the system. We derive necessary and sufficient conditions for the product-form of the stationary probability distribution of the Markov jump process describing the system.

Sign in / Sign up

Export Citation Format

Share Document