General Lyapunov-Based Iterative Algorithm for Linear Quadratic Regulator Problem of Stochastic Systems with Markovian Jump

This paper investigates the linear quadratic regulator(LQR) problem of linear stochastic systems with Markovian jump. Firstly, two iterative algorithms are proposed for solving the corresponding coupled algebraic Riccati equa- tions (CAREs) based on the general-type Lyapunov equation derived from linear stochastic systems. It is verified that the second algorithm adding an adjustable factor converges faster than the first one without it. Secondly, a monotonic convergence theorem is established for the proposed iterative algorithms under certain initial conditions. In the end, a numerical example is given to verify the efficiency of the proposed algorithms.

General Lyapunov-Based Iterative Algorithm for Linear Quadratic Regulator Problem of Stochastic Systems with Markovian Jump

This paper investigates the linear quadratic regulator(LQR) problem of linear stochastic systems with Markovian jump. Firstly, two iterative algorithms are proposed for solving the corresponding coupled algebraic Riccati equa- tions (CAREs) based on the general-type Lyapunov equation derived from linear stochastic systems. It is verified that the second algorithm adding an adjustable factor converges faster than the first one without it. Secondly, a monotonic convergence theorem is established for the proposed iterative algorithms under certain initial conditions. In the end, a numerical example is given to verify the efficiency of the proposed algorithms.

Convergence results for an averaged LQR problem with applications to reinforcement learning

In this paper, we will deal with a linear quadratic optimal control problem with unknown dynamics. As a modeling assumption, we will suppose that the knowledge that an agent has on the current system is represented by a probability distribution $$\pi $$ π on the space of matrices. Furthermore, we will assume that such a probability measure is opportunely updated to take into account the increased experience that the agent obtains while exploring the environment, approximating with increasing accuracy the underlying dynamics. Under these assumptions, we will show that the optimal control obtained by solving the “average” linear quadratic optimal control problem with respect to a certain $$\pi $$ π converges to the optimal control driven related to the linear quadratic optimal control problem governed by the actual, underlying dynamics. This approach is closely related to model-based reinforcement learning algorithms where prior and posterior probability distributions describing the knowledge on the uncertain system are recursively updated. In the last section, we will show a numerical test that confirms the theoretical results.

Parallel Computing for Rolling Mill Process with a Numerical Treatment of the LQR Problem

The considerable increase in computation of the optimal control problems has in many cases overflowed the computing capacity available to handle complex systems in real time. For this reason, alternatives such as parallel computing are studied in this article, where the problem is worked out by distributing the tasks among several processors in order to accelerate the computation and to analyze and investigate the reduction of the total time of calculation the incremental gradually the processors used in it. We explore the use of these methods with a case study represented in a rolling mill process, and in turn making use of the strategy of updating the Phase Finals values for the construction of the final penalty matrix for the solution of the differential Riccati Equation. In addition, the order of the problem studied is increasing gradually for compare the improvements achieved in the models with major dimension. Parallel computing alternatives are also studied through multiple processing elements within a single machine or in a cluster via OpenMP, which is an application programming interface (API) that allows the creation of shared memory programs.

A New Method for Determining the Degree of Controllability of State Variables for the LQR Problem Using the Duality Theorem

The control performance of a dynamic system can be checked by the degree of controllability. In this work, we present a new method for determining the degree of observability of state variables for the linear quadratic optimal estimation (LQE) problem. We carried out the calculation of the degree of controllability for the linear quadratic optimal control (LQR) problem using a duality theorem. Compared with the traditional measures of controllability such as determinant, trace, and maximal eigenvalue of the inverse controllability Gramian, the proposed degree of controllability was developed for each state variable and takes into account both the controllability Gramian and the cost function. The new method is convenient to apply to LQR problem. In the numerical simulation, we determined the influence of the model parameters on the degree of controllability. Besides that, we analyzed the degree of controllability, which gives an insight into the relationship between the system model design and the control performance.

A periodic linear–quadratic controller for suppressing rotor-blade vibration

This paper presents an active control strategy, based on a time-varying linear–quadratic optimal control problem, to attenuate the tip vibration of a two-dimensional coupled rotor-blade system whose dynamics is periodic. First, a periodic full-state feedback controller based on the linear–quadratic regulator (LQR) problem is designed. If all the states are not available for feedback, then an optimal periodic time-varying estimator, using the Kalman–Bucy filter, is computed. Both the Kalman filter gain and the LQR gain are obtained as the solution of a periodic Riccati differential equation (PRDE). Together, these gains provide the observer-based linear–quadratic–Gaussian (LQG) controller. An algorithm to solve the PRDE is also presented. Both controller designs ensure closed-loop stability and performance for the linear time-varying rotor-blade equation of motion. Numerical simulations show that the LQR and the LQG controllers are able to significantly attenuate the rotor-blade tip vibration.

Uncertain DC-DC Zeta Converter Control in Convex Polytope Model Based on LMI Approach

<span>A dc-dc zeta converter is a switch mode dc-dc converter that can either step-up or step-down dc input voltage. In order to regulate the dc output voltage, a control subsystem needs to be deployed for the dc-dc zeta converter. This paper presents the dc-dc zeta converter control. Unlike conventional dc-dc zeta converter control which produces a controller based on the nominal value model, we propose a convex polytope model of the dc-dc zeta converter which takes into account parameter uncertainty. A linear matrix inequality (LMI) is formulated based on the linear quadratic regulator (LQR) problem to find the state-feedback controller for the convex polytope model. Simulation results are presented to compare the control performance between the conventional LQR and the proposed LMI based controller on the dc-dc zeta converter. Furthermore, the reduction technique of the convex polytope is proposed and its effect is investigated.</span>

Numerical solution of the infinite-dimensional LQR problem and the associated Riccati differential equations

The numerical analysis of linear quadratic regulator design problems for parabolic partial differential equations requires solving Riccati equations. In the finite time horizon case, the Riccati differential equation (RDE) arises. The coefficient matrices of the resulting RDE often have a given structure, e.g., sparse, or low-rank. The associated RDE usually is quite stiff, so that implicit schemes should be used in this situation. In this paper, we derive efficient numerical methods for solving RDEs capable of exploiting this structure, which are based on a matrix-valued implementation of the BDF and Rosenbrock methods. We show that these methods are suitable for large-scale problems by working only on approximate low-rank factors of the solutions. We also incorporate step size and order control in our numerical algorithms for solving RDEs. In addition, we show that within a Galerkin projection framework the solutions of the finite-dimensional RDEs converge in the strong operator topology to the solutions of the infinite-dimensional RDEs. Numerical experiments show the performance of the proposed methods.