scholarly journals Self-Optimizing Path Tracking Controller for Intelligent Vehicles Based on Reinforcement Learning

Symmetry ◽  
2021 ◽  
Vol 14 (1) ◽  
pp. 31
Author(s):  
Jichang Ma ◽  
Hui Xie ◽  
Kang Song ◽  
Hao Liu
Keyword(s):  
Reinforcement Learning ◽  
Pid Control ◽  
Tracking Control ◽  
Tracking Error ◽  
Path Tracking ◽  
Pid Controllers ◽  
Model Parameters ◽  
Control Parameters ◽  
Tracking Controller ◽  
Complex Trajectories

The path tracking control system is a crucial component for autonomous vehicles; it is challenging to realize accurate tracking control when approaching a wide range of uncertain situations and dynamic environments, particularly when such control must perform as well as, or better than, human drivers. While many methods provide state-of-the-art tracking performance, they tend to emphasize constant PID control parameters, calibrated by human experience, to improve tracking accuracy. A detailed analysis shows that PID controllers inefficiently reduce the lateral error under various conditions, such as complex trajectories and variable speed. In addition, intelligent driving vehicles are highly non-linear objects, and high-fidelity models are unavailable in most autonomous systems. As for the model-based controller (MPC or LQR), the complex modeling process may increase the computational burden. With that in mind, a self-optimizing, path tracking controller structure, based on reinforcement learning, is proposed. For the lateral control of the vehicle, a steering method based on the fusion of the reinforcement learning and traditional PID controllers is designed to adapt to various tracking scenarios. According to the pre-defined path geometry and the real-time status of the vehicle, the interactive learning mechanism, based on an RL framework (actor–critic—a symmetric network structure), can realize the online optimization of PID control parameters in order to better deal with the tracking error under complex trajectories and dynamic changes of vehicle model parameters. The adaptive performance of velocity changes was also considered in the tracking process. The proposed controlling approach was tested in different path tracking scenarios, both the driving simulator platforms and on-site vehicle experiments have verified the effects of our proposed self-optimizing controller. The results show that the approach can adaptively change the weights of PID to maintain a tracking error (simulation: within ±0.071 m; realistic vehicle: within ±0.272 m) and steering wheel vibration standard deviations (simulation: within ±0.04°; realistic vehicle: within ±80.69°); additionally, it can adapt to high-speed simulation scenarios (the maximum speed is above 100 km/h and the average speed through curves is 63–76 km/h).

scholarly journals A learning-based optimal tracking controller for continuous linear systems with unknown dynamics: Theory and case study

2020 ◽  
Vol 53 (5-6) ◽  
pp. 778-787
Author(s):  
Jingren Zhang ◽  
Qingfeng Wang ◽  
Tao Wang
Keyword(s):  
Reinforcement Learning ◽  
Control Problem ◽  
Tracking Control ◽  
Tracking Error ◽  
Infinite Horizon ◽  
Learning Method ◽  
Optimal Tracking ◽  
Optimal Tracking Control ◽  
Tracking Controller

In this article, a novel continuous-time optimal tracking controller is proposed for the single-input-single-output linear system with completely unknown dynamics. Unlike those existing solutions to the optimal tracking control problem, the proposed controller introduces an integral compensation to reduce the steady-state error and regulates the feedforward part simultaneously with the feedback part. An augmented system composed of the integral compensation, error dynamics, and desired trajectory is established to formulate the optimal tracking control problem. The input energy and tracking error of the optimal controller are minimized according to the objective function in the infinite horizon. With the application of reinforcement learning techniques, the proposed controller does not require any prior knowledge of the system drift or input dynamics. The integral reinforcement learning method is employed to approximate the Q-function and update the critic network on-line. And the actor network is updated with the deterministic learning method. The Lyapunov stability is proved under the persistence of excitation condition. A case study on a hydraulic loading system has shown the effectiveness of the proposed controller by simulation and experiment.

scholarly journals Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Xiaoyi Long ◽  
Zheng He ◽  
Zhongyuan Wang
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Tracking Control ◽  
Learning Algorithm ◽  
Tracking Error ◽  
Adaptive Dynamic Programming ◽  
Robotic Systems ◽  
Control Synthesis ◽  
Optimal Tracking ◽  
Optimal Tracking Control

This paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network (NN)-based reinforcement learning (RL) method. To this end, we rewrite the robotic system model as a state-space form, which will facilitate the realization of optimal tracking control synthesis. To maintain the tracking response, a steady-state control is designed, and then an adaptive optimal tracking control is used to ensure that the tracking error can achieve convergence in an optimal sense. To solve the obtained optimal control via the framework of adaptive dynamic programming (ADP), the command trajectory to be tracked and the modified tracking Hamilton-Jacobi-Bellman (HJB) are all formulated. An online RL algorithm is the developed to address the HJB equation using a critic NN with online learning algorithm. Simulation results are given to verify the effectiveness of the proposed method.

scholarly journals A model reference adaptive variable impedance control method for robot

2021 ◽  
Vol 336 ◽  
pp. 03005
Author(s):  
Xinchao Sun ◽  
Lianyu Zhao ◽  
Zhenzhong Liu
Keyword(s):  
Real Time ◽  
Tracking Control ◽  
Control Method ◽  
Impedance Control ◽  
Tracking Error ◽  
Reference Trajectory ◽  
Control Parameters ◽  
Model Reference ◽  
Force Tracking ◽  
Model Reference Adaptive

As a simple and effective force tracking control method, impedance control is widely used in robot contact operations. The internal control parameters of traditional impedance control are constant and cannot be corrected in real time, which will lead to instability of control system or large force tracking error. Therefore, it is difficult to be applied to the occasions requiring higher force accuracy, such as robotic medical surgery, robotic space operation and so on. To solve this problem, this paper proposes a model reference adaptive variable impedance control method, which can realize force tracking control by adjusting internal impedance control parameters in real time and generating a reference trajectory at the same time. The simulation experiment proves that compared with the traditional impedance control method, this method has faster force tracking speed and smaller force tracking error. It is a better force tracking control method.

Design of Optimum Controller of APT Fine Tracking Control System

2013 ◽  
Vol 712-715 ◽  
pp. 2738-2741 ◽  
Author(s):  
Ming Qiu Li ◽  
Shu Hua Jiang
Keyword(s):  
Pid Control ◽  
Tracking Control ◽  
Control Algorithm ◽  
Tracking System ◽  
Dynamic Performance ◽  
Tracking Error ◽  
Stable State ◽  
Response Speed ◽  
Optimum Control ◽  
Performance Requirements

APT (Acquisition, Pointing, and Tracking) system of space laser communication adopts compound axis structure; it consists of coarse tracking and fine tracking system. Its response speed and tracking precision mainly rests with the fine tracking system. Traditional PID control algorithm often is used in APT fine tracking system. In order to improve the dynamic performance of the system and decrease the tracking error, optimum control technology was adopted in this paper. On the basis of considering the system dynamic performance requirements and tracking precision requirement, optimum controller was designed. The simulation result shows that the bandwidth of APT fine tracking system is up to 1310 Hz, and the stable state error is less than 0.002. Compared with PID control, optimum control can improve the tracking performance of system.

scholarly journals Decentralized Reinforcement Learning Robust Optimal Tracking Control for Time Varying Constrained Reconfigurable Modular Robot Based on ACI andQ-Function

2013 ◽  
Vol 2013 ◽  
pp. 1-16 ◽  
Author(s):  
Bo Dong ◽  
Yuanchun Li
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Control Theory ◽  
Continuous Time ◽  
Tracking Control ◽  
Control Policy ◽  
Time Varying ◽  
Optimal Tracking ◽  
Tracking Controller ◽  
Global Uncertainty

A novel decentralized reinforcement learning robust optimal tracking control theory for time varying constrained reconfigurable modular robots based on action-critic-identifier (ACI) and state-action value function (Q-function) has been presented to solve the problem of the continuous time nonlinear optimal control policy for strongly coupled uncertainty robotic system. The dynamics of time varying constrained reconfigurable modular robot is described as a synthesis of interconnected subsystem, and continuous time state equation andQ-function have been designed in this paper. Combining with ACI and RBF network, the global uncertainty of the subsystem and the HJB (Hamilton-Jacobi-Bellman) equation have been estimated, where critic-NN and action-NN are used to approximate the optimalQ-function and the optimal control policy, and the identifier is adopted to identify the global uncertainty as well as RBF-NN which is used to update the weights of ACI-NN. On this basis, a novel decentralized robust optimal tracking controller of the subsystem is proposed, so that the subsystem can track the desired trajectory and the tracking error can converge to zero in a finite time. The stability of ACI and the robust optimal tracking controller are confirmed by Lyapunov theory. Finally, comparative simulation examples are presented to illustrate the effectiveness of the proposed ACI and decentralized control theory.

scholarly journals Preview Tracking Control for Continuous-Time Singular Interconnected Systems

2019 ◽  
Vol 2019 ◽  
pp. 1-13
Author(s):  
Hao Xie ◽  
Fucheng Liao ◽  
Jiamei Deng
Keyword(s):  
Numerical Simulation ◽  
Continuous Time ◽  
Tracking Control ◽  
Singular Systems ◽  
Tracking Error ◽  
Original System ◽  
Interconnected Systems ◽  
Interconnected System ◽  
Tracking Controller ◽  
Error System

This paper proposes and investigates a problem of preview tracking control for a class of continuous-time singular interconnected systems. Firstly, the related items are deleted to obtain several isolated subsystems with low dimensions. An error system is constructed for each isolated subsystem so that the tracking error is included in the state vector of the error system; then, the tracking problem is transformed into a regulation problem. Secondly, the preview tracking controller is designed for each error system and obtained controllers are combined as the controller of the error system of the singular interconnected system. Thirdly, the Lyapunov function method is utilized to determine the constraints of the related terms so that the closed-loop system of the error system of the singular interconnected system is stable under the action of the controller obtained. Finally, the preview tracking controller of the singular interconnected system is obtained from the relationship between the error system and the original system. A numerical simulation algorithm for continuous-time singular systems is also proposed in this paper. The numerical simulation illustrates the effectiveness of the theoretical results.

Path tracking control of electromechanical micro-positioner by considering control effort of the system

2020 ◽  
pp. 095965182095327
Author(s):  
Mohammad Reza Gharib ◽  
Ali Koochi ◽  
Mojtaba Ghorbani
Keyword(s):  
Tracking Control ◽  
Tracking Error ◽  
Path Tracking ◽  
Lumped Parameter Model ◽  
Proportional Integral Derivative ◽  
Control Effort ◽  
Proportional Integral Derivative Controller ◽  
Proportional Integral ◽  
Micro Actuators ◽  
And Control

Position controlling with less overshoot and control effort is a fundamental issue in the design and application of micro-actuators such as micro-positioner. Also, tracking a considered path is very crucial for some particular applications of micro-actuators such as surgeon robots. Herein, a proportional–integral–derivative controller is designed using a feedback linearization technique for path tracking control of a cantilever electromechanical micro-positioner. The micro-positioner is simulated based on a 1-degree-of-freedom lumped-parameter model. Three different paths are considered, and the capability of the designed controller on the path tracking with lower error and control effort is investigated. The obtained results demonstrate the efficiency of the designed proportional–integral–derivative controller not only for reducing the tracking error but also for decreasing the control effort.

Sign in / Sign up

Export Citation Format

Share Document