Balance control of two-wheeled robot based on reinforcement learning

Single-track two-wheeled robots have become an important research topic in recent years, owing to their simple structure, energy savings and ability to run on narrow roads. However, the ramp jump remains a challenging task. In this study, we propose to realize a single-track two-wheeled robot ramp jump. We present a control method that employs continuous action reinforcement learning techniques for single-track two-wheeled robot control. We design a novel reward function for reinforcement learning, optimize the dimensions of the action space, and enable training under the deep deterministic policy gradient algorithm. Finally, we validate the control method through simulation experiments and successfully realize the single-track two-wheeled robot ramp jump task. Simulation results validate that the control method is effective and has several advantages over high-dimension action space control, reinforcement learning control of sparse reward function and discrete action reinforcement learning control.

Download Full-text

The balance control of two-wheeled robot based on bionic learning algorithm

The 26th Chinese Control and Decision Conference (2014 CCDC) ◽

10.1109/ccdc.2014.6852911 ◽

2014 ◽

Cited By ~ 3

Author(s):

Ren Hongge ◽

Wang Zhilong ◽

Li Fujin ◽

Huo Meijie

Keyword(s):

Learning Algorithm ◽

Balance Control ◽

Wheeled Robot

Download Full-text

Self-Balance Control of Two-Wheeled Robot Based on Skinner's Operant Conditioning

2009 International Workshop on Intelligent Systems and Applications ◽

10.1109/iwisa.2009.5073106 ◽

2009 ◽

Author(s):

Xiao-Gang Ruan ◽

Hong-Ge Ren ◽

Qi-Yuan Wang

Keyword(s):

Operant Conditioning ◽

Balance Control ◽

Wheeled Robot

Download Full-text

Advanced Proportional-Integral-Derivative Control Compensation Based on a Grey Estimated Model in Dynamic Balance of Single-Wheeled Robot

Axioms ◽

10.3390/axioms10040326 ◽

2021 ◽

Vol 10 (4) ◽

pp. 326

Author(s):

Mao-Lin Chen ◽

Chun-Yen Chen ◽

Chien-Hung Wen ◽

Pin-Hao Liao ◽

Kai-Jung Chen

Keyword(s):

Mathematical Modeling ◽

Control Method ◽

Reference Model ◽

Balance Control ◽

Lagrange Equation ◽

Linear Quadratic Regulator ◽

Dynamic State ◽

Linear Quadratic ◽

Wheeled Robot ◽

The One

This paper aims to design a one-wheeled robot as regards its pitch freedom and balance control on the one hand and to assess the application feasibility of the GM (1,1) swing estimation controller on the other. System control focuses mainly on one-wheeled robot stability, body swings in position, and speed control. Mathematical modeling and GM (1,1) prediction control are under investigation. The mathematical modeling is firstly conducted through referencing to the Newtonian mechanics and the Lagrange equation, from which the robot transfer function and state-space differential equation are derived. Next, the linear quadratic regulator is applied as the control rule at the balance point. Applying GM (1,1) to assess the robot gyro signal at a dynamic state is a discussion. Next, model reference estimation control is processed, and a mathematical model of the balance control method is completed. Finally, a simulation is conducted to verify the feasibility of the GM (1,1) estimation reference model. The linear quadratic regulator, which is credited with tenacity, can provide pitch swing and balance control of the one-wheeled robot.

Download Full-text

Dynamic model and balance control of two-wheeled robot with non-holonomic constraints

Proceeding of the 11th World Congress on Intelligent Control and Automation ◽

10.1109/wcica.2014.7052764 ◽

2014 ◽

Cited By ~ 1

Author(s):

Ling Li ◽

Shigong Jiang ◽

Fuquan Dai ◽

Xueshan Gao

Keyword(s):

Dynamic Model ◽

Balance Control ◽

Holonomic Constraints ◽

Wheeled Robot

Download Full-text

Adaptive Quadruped Balance Control for Dynamic Environments Using Maximum-Entropy Reinforcement Learning

Sensors ◽

10.3390/s21175907 ◽

2021 ◽

Vol 21 (17) ◽

pp. 5907

Author(s):

Haoran Sun ◽

Tingting Fu ◽

Yuanhuai Ling ◽

Chaoming He

Keyword(s):

Neural Network ◽

Reinforcement Learning ◽

Maximum Entropy ◽

Maximum Entropy Method ◽

Balance Control ◽

External Disturbance ◽

Control Policy ◽

Dynamic Environments ◽

Entropy Method ◽

Analytical Models

External disturbance poses the primary threat to robot balance in dynamic environments. This paper provides a learning-based control architecture for quadrupedal self-balancing, which is adaptable to multiple unpredictable scenes of external continuous disturbance. Different from conventional methods which construct analytical models which explicitly reason the balancing process, our work utilized reinforcement learning and artificial neural network to avoid incomprehensible mathematical modeling. The control policy is composed of a neural network and a Tanh Gaussian policy, which implicitly establishes the fuzzy mapping from proprioceptive signals to action commands. During the training process, the maximum-entropy method (soft actor-critic algorithm) is employed to endow the policy with powerful exploration and generalization ability. The trained policy is validated in both simulations and realistic experiments with a customized quadruped robot. The results demonstrate that the policy can be easily transferred to the real world without elaborate configurations. Moreover, although this policy is trained in merely one specific vibration condition, it demonstrates robustness under conditions that were never encountered during training.

Download Full-text