An Iterative Scheme of Safe Reinforcement Learning for Nonlinear Systems via Barrier Certificate Generation

In this paper, we propose a safe reinforcement learning approach to synthesize deep neural network (DNN) controllers for nonlinear systems subject to safety constraints. The proposed approach employs an iterative scheme where a learner and a verifier interact to synthesize safe DNN controllers. The learner trains a DNN controller via deep reinforcement learning, and the verifier certifies the learned controller through computing a maximal safe initial region and its corresponding barrier certificate, based on polynomial abstraction and bilinear matrix inequalities solving. Compared with the existing verification-in-the-loop synthesis methods, our iterative framework is a sequential synthesis scheme of controllers and barrier certificates, which can learn safe controllers with adaptive barrier certificates rather than user-defined ones. We implement the tool SRLBC and evaluate its performance over a set of benchmark examples. The experimental results demonstrate that our approach efficiently synthesizes safe DNN controllers even for a nonlinear system with dimension up to 12.

The control of drilling vibrations: A coupled PDE-ODE modeling approach

The main purpose of this contribution is the control of both torsional and axial vibrations occurring along a rotary oilwell drilling system. The model considered consists of a wave equation coupled to an ordinary differential equation (ODE) through a nonlinear function describing the rock-bit interaction. We propose a systematic method to design feedback controllers guaranteeing ultimate boundedness of the system trajectories and leading consequently to the suppression of harmful dynamics. The proposal of a Lyapunov-Krasovskii functional provides stability conditions stated in terms of the solution of a set of linear and bilinear matrix inequalities (LMIs, BMIs). Numerical simulations illustrate the efficiency of the obtained control laws.