Bridging Reinforcement Learning and Iterative Learning Control: Autonomous Reference Tracking for Unknown, Nonlinear Dynamics

<div>This work addresses the problem of reference tracking in autonomously learning agents with unknown, nonlinear dynamics. Existing solutions require model information or extensive parameter tuning, and have rarely been validated in real-world experiments. We propose a learning control scheme that learns to approximate the unknown dynamics by a Gaussian Process (GP), which is used to optimize and apply a feedforward control input on each trial. Unlike existing approaches, the proposed method neither requires knowledge of the system states and their dynamics nor knowledge of an effective feedback control structure. All algorithm parameters are chosen automatically, i.e. the learning method works plug and play. The proposed method is validated in extensive simulations and real-world experiments. In contrast to most existing work, we study learning dynamics for more than one motion task as well as the robustness of performance across a large range of learning parameters. The method’s plug and play applicability is demonstrated by experiments with a balancing robot, in which the proposed method rapidly learns to track the desired output. Due to its model-agnostic and plug and play properties, the proposed method is expected to have high potential for application to a large class of reference tracking problems in systems with unknown, nonlinear dynamics.</div>

Bridging Reinforcement Learning and Iterative Learning Control: Autonomous Reference Tracking for Unknown, Nonlinear Dynamics

<div>This work addresses the problem of reference tracking in autonomously learning agents with unknown, nonlinear dynamics. Existing solutions require model information or extensive parameter tuning, and have rarely been validated in real-world experiments. We propose a learning control scheme that learns to approximate the unknown dynamics by a Gaussian Process (GP), which is used to optimize and apply a feedforward control input on each trial. Unlike existing approaches, the proposed method neither requires knowledge of the system states and their dynamics nor knowledge of an effective feedback control structure. All algorithm parameters are chosen automatically, i.e. the learning method works plug and play. The proposed method is validated in extensive simulations and real-world experiments. In contrast to most existing work, we study learning dynamics for more than one motion task as well as the robustness of performance across a large range of learning parameters. The method’s plug and play applicability is demonstrated by experiments with a balancing robot, in which the proposed method rapidly learns to track the desired output. Due to its model-agnostic and plug and play properties, the proposed method is expected to have high potential for application to a large class of reference tracking problems in systems with unknown, nonlinear dynamics.</div>

Formation tracking of multiple amphibious robots with unknown nonlinear dynamics

Cooperative tracking control problem of multiple water–land amphibious robots is discussed in this article with consideration of unknown nonlinear dynamics. Firstly, the amphibious robot dynamic model is formulated as an uncoupled nonlinear one in horizontal plane through eliminating relatively small sway velocity of the platform. Then cooperative tracking control algorithm is proposed with a two-stage strategy including dynamic control stage and kinematic control stage. In dynamic control stage, adaptive consensus control algorithm is obtained with estimating nonlinear properties of amphibious robots and velocities of the leader by neural network with unreliable communication links which is always the case in underwater applications. After that, kinematic cooperative controller is presented to guarantee formation stability of multiple water–land amphibious robots system in kinematic control stage. As a result, with the implementation of graph theory and Lyapunov theory, the stability of the formation tracking of multiple water–land amphibious robots system is proved with consideration of jointly connected communication graph. At last, simulations are carried out to prove the effectiveness of the proposed approaches.

Finite-time consensus of heterogeneous unknown nonlinear multi-agent systems with external disturbances via event-triggered control

This article investigates the practical finite-time consensus for a class of heterogeneous multi-agent systems composed of first-order and second-order agents with heterogeneous unknown nonlinear dynamics and external disturbances in an undirected communication topology. To reduce the system updates, we propose an event-triggered approach. By defining auxiliary states, an adaptive distributed event-triggered control is designed to achieve practical finite-time consensus. Unknown nonlinear dynamics for each agent are estimated using radial basis function neural network. The stability of the overall closed-loop system is studied through the Lyapunov criterion. It is proven that by applying the proposed control scheme, the local neighbor position error and the velocity error between any two agents converge to a small region in finite time. Furthermore, it is shown that the Zeno behavior is ruled out. Finally, applicability and effectiveness of the proposed control scheme is verified and validated by two examples.

Elegant anti-disturbance control for stochastic systems with multiple heterogeneous disturbances based on fuzzy logic systems

There are a large number of non-harmonic disturbances generated by nonlinear exogenous systems in realistic engineering. The current disturbance observer is not applicable for estimating the non-harmonic disturbance with unknown nonlinear dynamics, thus greatly reducing the accuracy of the controller. This paper addresses a class of stochastic systems with multiple heterogeneous disturbances including white noise and non-harmonic disturbance with unknown smooth nonlinear function, which can be approximated by fuzzy logic systems. Based on the approximation of the unknown nonlinear function, an adaptive disturbance observer (ADO) is constructed to estimate non-harmonic disturbance. Combining disturbance observer-based control with fuzzy control, an elegant anti-disturbance control (EADC) scheme is proposed such that the composite system achieves asymptotically bounded in mean square. Simulation examples show that the state responses of the system gradually approache [Formula: see text] from divergence, indicating that the effectiveness of the controller is satisfactory. In addition, the anti-disturbance control accuracy of EADC approximately improves [Formula: see text] times compared with [Formula: see text] control. The simulation results demonstrate the feasibility and effectiveness of the proposed scheme.

Active disturbance rejection consensus control of uncertain high-order nonlinear multi-agent systems

In this paper, we consider the consensus control problem for uncertain high-order nonlinear multi-agent systems in a leader-follower scheme. Each follower node is modeled by a high-order integrator incorporating with unmeasurable states and unknown nonlinear dynamics. First, the total uncertainty that lumps the unknown nonlinear dynamics and the mismatch of control is viewed as an extended state of the agent. By using local information from neighborhood set, a distributed extended state observer (ESO) is designed to estimate not only the unmeasurable agent states but also its total uncertainty. Then, based on the output of the ESO, a novel consensus control law is proposed, in which the total uncertainty is canceled out in the feedback loop in real time. We show that, with the application of the proposed approach, the ESO estimation errors and the disagreement error vectors between the leader and the followers can be made arbitrarily small. A simulation example is given to illustrate the effectiveness of the proposed consensus control method.