scholarly journals On Stability of Perturbed Nonlinear Switched Systems with Adaptive Reinforcement Learning

Energies ◽  
2020 ◽  
Vol 13 (19) ◽  
pp. 5069
Author(s):  
Phuong Nam Dao ◽  
Hong Quang Nguyen ◽  
Minh-Duc Ngo ◽  
Seon-Ju Ahn
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Switched Systems ◽  
Tracking Control ◽  
Control Method ◽  
Learning Algorithm ◽  
Hjb Equation ◽  
Control Approach ◽  
Nonlinear Switched Systems ◽  
Hamilton Jacobi Bellman

In this paper, a tracking control approach is developed based on an adaptive reinforcement learning algorithm with a bounded cost function for perturbed nonlinear switched systems, which represent a useful framework for modelling these converters, such as DC–DC converter, multi-level converter, etc. An optimal control method is derived for nominal systems to solve the tracking control problem, which results in solving a Hamilton–Jacobi–Bellman (HJB) equation. It is shown that the optimal controller obtained by solving the HJB equation can stabilize the perturbed nonlinear switched systems. To develop a solution to the translated HJB equation, the proposed neural networks consider the training technique obtaining the minimization of square of Bellman residual error in critic term due to the description of Hamilton function. Theoretical analysis shows that all the closed-loop system signals are uniformly ultimately bounded (UUB) and the proposed controller converges to optimal control law. The simulation results of two situations demonstrate the effectiveness of the proposed controller.

scholarly journals Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Xiaoyi Long ◽  
Zheng He ◽  
Zhongyuan Wang
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Tracking Control ◽  
Learning Algorithm ◽  
Tracking Error ◽  
Adaptive Dynamic Programming ◽  
Robotic Systems ◽  
Control Synthesis ◽  
Optimal Tracking ◽  
Optimal Tracking Control

This paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network (NN)-based reinforcement learning (RL) method. To this end, we rewrite the robotic system model as a state-space form, which will facilitate the realization of optimal tracking control synthesis. To maintain the tracking response, a steady-state control is designed, and then an adaptive optimal tracking control is used to ensure that the tracking error can achieve convergence in an optimal sense. To solve the obtained optimal control via the framework of adaptive dynamic programming (ADP), the command trajectory to be tracked and the modified tracking Hamilton-Jacobi-Bellman (HJB) are all formulated. An online RL algorithm is the developed to address the HJB equation using a critic NN with online learning algorithm. Simulation results are given to verify the effectiveness of the proposed method.

scholarly journals Adaptive Asymptotic Tracking Control for a Class of Uncertain Switched Systems via Dynamic Surface Technique

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-9
Author(s):  
Xuemiao Chen ◽  
Qianjin Zhao ◽  
Chunsheng Zhang ◽  
Jian Wu
Keyword(s):  
Switched Systems ◽  
Tracking Control ◽  
Control Method ◽  
Reference Signal ◽  
Adaptive Controller ◽  
Dynamic Surface Control ◽  
Dynamic Surface ◽  
Adaptive Tracking Control ◽  
Nonlinear Switched Systems ◽  
Control Scheme

A novel adaptive tracking control scheme is proposed for a class of uncertain nonlinear switched systems with perturbations in this paper. The common Lyapunov function method is introduced to handle the switched system in the design process of the desired adaptive controller. In addition, a dynamic surface control method is proposed by employing a nonlinear filter such that the “explosion of complexity” problem existing in the conventional backstepping design can be overcome. Under the presented adaptive controller, all the closed-loop signals are semiglobally bounded, and especially the output signal of the controlled system can follow the given reference signal asymptotically. To show the availability of the presented control scheme, a simulation is given in this paper.

A Nonlinear Optimal Control Approach for the Vertical Take-off and Landing Aircraft

2021 ◽  
pp. 2150012
Author(s):  
G. Rigatos
Keyword(s):  
Optimal Control ◽  
Control Method ◽  
Nonlinear Optimal Control ◽  
The Body ◽  
Algebraic Riccati Equation ◽  
Control Input ◽  
Time Step ◽  
Control Approach ◽  
Vtol Aircraft ◽  
Optimal Control Approach

The paper proposes a nonlinear optimal control approach for the model of the vertical take-off and landing (VTOL) aircraft. This aerial drone receives as control input a directed thrust, as well as forces acting on its wing tips. The latter forces are not perpendicular to the body axis of the drone but are tilted by a small angle. The dynamic model of the VTOL undergoes approximate linearization with the use of Taylor series expansion around a temporary operating point which is recomputed at each iteration of the control method. For the approximately linearized model, an H-infinity feedback controller is designed. The linearization procedure relies on the computation of the Jacobian matrices of the state-space model of the VTOL aircraft. The proposed control method stands for the solution of the optimal control problem for the nonlinear and multivariable dynamics of the aerial drone, under model uncertainties and external perturbations. For the computation of the controller’s feedback gains, an algebraic Riccati equation is solved at each time-step of the control method. The new nonlinear optimal control approach achieves fast and accurate tracking for all state variables of the VTOL aircraft, under moderate variations of the control inputs. The stability properties of the control scheme are proven through Lyapunov analysis.

scholarly journals Decentralized Reinforcement Learning Robust Optimal Tracking Control for Time Varying Constrained Reconfigurable Modular Robot Based on ACI andQ-Function

2013 ◽  
Vol 2013 ◽  
pp. 1-16 ◽  
Author(s):  
Bo Dong ◽  
Yuanchun Li
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Control Theory ◽  
Continuous Time ◽  
Tracking Control ◽  
Control Policy ◽  
Time Varying ◽  
Optimal Tracking ◽  
Tracking Controller ◽  
Global Uncertainty

A novel decentralized reinforcement learning robust optimal tracking control theory for time varying constrained reconfigurable modular robots based on action-critic-identifier (ACI) and state-action value function (Q-function) has been presented to solve the problem of the continuous time nonlinear optimal control policy for strongly coupled uncertainty robotic system. The dynamics of time varying constrained reconfigurable modular robot is described as a synthesis of interconnected subsystem, and continuous time state equation andQ-function have been designed in this paper. Combining with ACI and RBF network, the global uncertainty of the subsystem and the HJB (Hamilton-Jacobi-Bellman) equation have been estimated, where critic-NN and action-NN are used to approximate the optimalQ-function and the optimal control policy, and the identifier is adopted to identify the global uncertainty as well as RBF-NN which is used to update the weights of ACI-NN. On this basis, a novel decentralized robust optimal tracking controller of the subsystem is proposed, so that the subsystem can track the desired trajectory and the tracking error can converge to zero in a finite time. The stability of ACI and the robust optimal tracking controller are confirmed by Lyapunov theory. Finally, comparative simulation examples are presented to illustrate the effectiveness of the proposed ACI and decentralized control theory.

scholarly journals Spatial-Temporal Flows-Adaptive Street Layout Control Using Reinforcement Learning

2021 ◽  
Vol 14 (1) ◽  
pp. 107
Author(s):  
Qiming Ye ◽  
Yuxiang Feng ◽  
Eduardo Candela ◽  
Jose Escribano Macias ◽  
Marc Stettler ◽  
...  
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Real Time ◽  
Control Method ◽  
Traffic Light ◽  
Traffic Efficiency ◽  
Road Management ◽  
Street Space ◽  
Road Space ◽  
Pedestrian Travel

Complete streets scheme makes seminal contributions to securing the basic public right-of-way (ROW), improving road safety, and maintaining high traffic efficiency for all modes of commute. However, such a popular street design paradigm also faces endogenous pressures like the appeal to a more balanced ROW for non-vehicular users. In addition, the deployment of Autonomous Vehicle (AV) mobility is likely to challenge the conventional use of the street space as well as this scheme. Previous studies have invented automated control techniques for specific road management issues, such as traffic light control and lane management. Whereas models and algorithms that dynamically calibrate the ROW of road space corresponding to travel demands and place-making requirements still represent a research gap. This study proposes a novel optimal control method that decides the ROW of road space assigned to driveways and sidewalks in real-time. To solve this optimal control task, a reinforcement learning method is introduced that employs a microscopic traffic simulator, namely SUMO, as its environment. The model was trained for 150 episodes using a four-legged intersection and joint AVs-pedestrian travel demands of a day. Results evidenced the effectiveness of the model in both symmetric and asymmetric road settings. After being trained by 150 episodes, our proposed model significantly increased its comprehensive reward of both pedestrians and vehicular traffic efficiency and sidewalk ratio by 10.39%. Decisions on the balanced ROW are optimised as 90.16% of the edges decrease the driveways supply and raise sidewalk shares by approximately 9%. Moreover, during 18.22% of the tested time slots, a lane-width equivalent space is shifted from driveways to sidewalks, minimising the travel costs for both an AV fleet and pedestrians. Our study primarily contributes to the modelling architecture and algorithms concerning centralised and real-time ROW management. Prospective applications out of this method are likely to facilitate AV mobility-oriented road management and pedestrian-friendly street space design in the near future.

scholarly journals Trajectory Tracking Control in Real-Time of Dual-Motor-Driven Driverless Racing Car Based on Optimal Control Theory and Fuzzy Logic Method

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Gang Li ◽  
Sucai Zhang ◽  
Lei Liu ◽  
Xubin Zhang ◽  
Yuming Yin
Keyword(s):  
Optimal Control ◽  
Fuzzy Logic ◽  
Optimal Control Theory ◽  
Tracking Control ◽  
Control Method ◽  
Steering Control ◽  
Trajectory Tracking Control ◽  
Differential Drive ◽  
Active Steering ◽  
Fuzzy Logic Method

To improve the accuracy and timeliness of the trajectory tracking control of the driverless racing car during the race, this paper proposes a track tracking control method that integrates the rear wheel differential drive and the front wheel active steering based on optimal control theory and fuzzy logic method. The model of the lateral track tracking error of the racing car is established. The model is linearized and discretized, and the quadratic optimal steering control problem is constructed. Taking advantage of the differential drive of dual-motor-driven racing car, the dual motors differential drive fuzzy controller is designed and integrated driving with active steering control. Simulation analysis and actual car verification show that this integrated control method can ensure that the car tracks different race tracks well and improve the track tracking control accuracy by nearly 30%.

Optimal Tracking Control for Discrete-time Systems with Time-delay Based on the Preview Control Method

2019 ◽  
Vol 7 (5) ◽  
pp. 452-461
Author(s):  
Haishan Xu ◽  
Fucheng Liao
Keyword(s):  
Optimal Control ◽  
Time Delay ◽  
Discrete Time ◽  
Tracking Control ◽  
Control Method ◽  
Reference Signal ◽  
Control Law ◽  
Preview Control ◽  
Optimal Tracking ◽  
Delayed System

Abstract In this paper, the optimal tracking control problem for discrete-time with state and input delays is studied based on the preview control method. First, a transformation is introduced. Thus, the system is transformed into a non-delayed system and the tracking problem of the time-delay system is transformed into the regulation problem of a non-delayed system via processing of the reference signal. Then, by applying the preview control theory, an augmented system for the non-delayed system is derived, and a controller with preview function is designed, assuming that the reference signal is previewable. Finally, the optimal control law of the augmented error system and the optimal control law of the original system are obtained by letting the preview length of the reference signal go to zero.

Sign in / Sign up

Export Citation Format

Share Document