Hybrid Dynamic Control Algorithm for Humanoid Robots Based on Reinforcement Learning

2007 ◽  
Vol 51 (1) ◽  
pp. 3-30 ◽  
Author(s):  
Duśko M. Katić ◽  
Aleksandar D. Rodić ◽  
Miomir K. Vukobratović
Keyword(s):  
Reinforcement Learning ◽  
Control Algorithm ◽  
Dynamic Control ◽  
Humanoid Robots

An Energy Dynamic Control Algorithm Based on Reinforcement Learning for Data Centers

2019 ◽  
Vol 33 (13) ◽  
pp. 1951009
Author(s):  
Yao Xiang ◽  
Jingling Yuan ◽  
Ruiqi Luo ◽  
Xian Zhong ◽  
Tao Li
Keyword(s):  
Renewable Energy ◽  
Reinforcement Learning ◽  
Energy Cost ◽  
Control Algorithm ◽  
Dynamic Control ◽  
Optimal Solution ◽  
Operation Process ◽  
Actual Operation ◽  
Internet Data Center ◽  
Future Information

In recent years, how to use renewable energy to reduce the energy cost of internet data center (IDC) has been an urgent problem to be solved. More and more solutions are beginning to consider machine learning, but many of the existing methods need to take advantage of some future information, which is difficult to obtain in the actual operation process. In this paper, we focus on reducing the energy cost of IDC by controlling the energy flow of renewable energy without any future information. we propose an efficient energy dynamic control algorithm based on the theory of reinforcement learning, which approximates the optimal solution by learning the feedback of historical control decisions. For the purpose of avoiding overestimation, improving the convergence ability of the algorithm, we use the double [Formula: see text]-method to further optimize. The extensive experimental results show that our algorithm can on average save the energy cost by 18.3% and reduce the rate of grid intervention by 26.2% compared with other algorithms, and thus has good application prospects.

scholarly journals Intelligent Routing Control for MANET Based on Reinforcement Learning

2018 ◽  
Vol 232 ◽  
pp. 04002
Author(s):  
Fang Dong ◽  
Ou Li ◽  
Min Tong
Keyword(s):  
Reinforcement Learning ◽  
Control Algorithm ◽  
Rapid Development ◽  
Selection Strategy ◽  
Performance Simulation ◽  
Multiple Parameters ◽  
Intelligent Routing ◽  
Routing Control ◽  
The Cost

With the rapid development and wide use of MANET, the quality of service for various businesses is much higher than before. Aiming at the adaptive routing control with multiple parameters for universal scenes, we propose an intelligent routing control algorithm for MANET based on reinforcement learning, which can constantly optimize the node selection strategy through the interaction with the environment and converge to the optimal transmission paths gradually. There is no need to update the network state frequently, which can save the cost of routing maintenance while improving the transmission performance. Simulation results show that, compared with other algorithms, the proposed approach can choose appropriate paths under constraint conditions, and can obtain better optimization objective.

scholarly journals On path following control of nonholonomic mobile manipulators

2009 ◽  
Vol 19 (4) ◽  
pp. 561-574 ◽  
Author(s):  
Alicja Mazur ◽  
Dawid Szakiel
Keyword(s):  
Control Algorithm ◽  
Dynamic Control ◽  
Path Following ◽  
Mobile Manipulator ◽  
Mobile Manipulators ◽  
Control Laws ◽  
Kinematic Control ◽  
Motion Constraints ◽  
Path Following Control ◽  
Cascade Structure

On path following control of nonholonomic mobile manipulatorsThis paper describes the problem of designing control laws for path following robots, including two types of nonholonomic mobile manipulators. Due to a cascade structure of the motion equation, a backstepping procedure is used to achieve motion along a desired path. The control algorithm consists of two simultaneously working controllers: the kinematic controller, solving motion constraints, and the dynamic controller, preserving an appropriate coordination between both subsystems of a mobile manipulator, i.e. the mobile platform and the manipulating arm. A description of the nonholonomic subsystem relative to the desired path using the Frenet parametrization is the basis for formulating the path following problem and designing a kinematic control algorithm. In turn, the dynamic control algorithm is a modification of a passivity-based controller. Theoretical deliberations are illustrated with simulations.

scholarly journals Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators

2019 ◽  
Vol 35 (1) ◽  
pp. 124-134 ◽  
Author(s):  
Thomas George Thuruthel ◽  
Egidio Falotico ◽  
Federico Renda ◽  
Cecilia Laschi
Keyword(s):  
Reinforcement Learning ◽  
Closed Loop ◽  
Dynamic Control ◽  
Robotic Manipulators ◽  
Model Based

scholarly journals Formation tracking of multiple amphibious robots with unknown nonlinear dynamics

2020 ◽  
Vol 17 (5) ◽  
pp. 172988142093854
Author(s):  
Di Wu ◽  
Lichao Hao ◽  
Xiujun Xu ◽  
Hongjian Wang ◽  
Jiajia Zhou
Keyword(s):  
Nonlinear Dynamics ◽  
Tracking Control ◽  
Control Algorithm ◽  
Dynamic Control ◽  
Lyapunov Theory ◽  
Kinematic Control ◽  
Formation Tracking ◽  
Control Stage ◽  
Cooperative Tracking ◽  
Unknown Nonlinear Dynamics

Cooperative tracking control problem of multiple water–land amphibious robots is discussed in this article with consideration of unknown nonlinear dynamics. Firstly, the amphibious robot dynamic model is formulated as an uncoupled nonlinear one in horizontal plane through eliminating relatively small sway velocity of the platform. Then cooperative tracking control algorithm is proposed with a two-stage strategy including dynamic control stage and kinematic control stage. In dynamic control stage, adaptive consensus control algorithm is obtained with estimating nonlinear properties of amphibious robots and velocities of the leader by neural network with unreliable communication links which is always the case in underwater applications. After that, kinematic cooperative controller is presented to guarantee formation stability of multiple water–land amphibious robots system in kinematic control stage. As a result, with the implementation of graph theory and Lyapunov theory, the stability of the formation tracking of multiple water–land amphibious robots system is proved with consideration of jointly connected communication graph. At last, simulations are carried out to prove the effectiveness of the proposed approaches.

Research on Dynamic Torque Control Strategy for Parallel Hybrid Electric Vehicle

2011 ◽  
Vol 228-229 ◽  
pp. 951-956 ◽  
Author(s):  
Yun Bing Yan ◽  
Fu Wu Yan ◽  
Chang Qing Du
Keyword(s):  
Electric Vehicle ◽  
Hybrid Electric Vehicle ◽  
Control Algorithm ◽  
Dynamic Control ◽  
Torque Control ◽  
Engine Torque ◽  
Parallel Hybrid Electric Vehicle ◽  
Torque Control Strategy ◽  
Parallel Hybrid ◽  
Hybrid Electric

It is necessary for Parallel Hybrid Electric Vehicle (PHEV) to distribute energy between engine and motor and to control state-switch during work. Aimed at keeping the total torque unchanging under state-switch, the dynamic torque control algorithm is put forward, which can be expressed as motor torque compensation for engine after torque pre-distribution, engine speed regulation and dynamic engine torque estimation. Taking Matlab as the platform, the vehicle control simulation model is built, based on which the fundamental control algorithm is verified by simulation testing. The results demonstrate that the dynamic control algorithm can effectively dampen torque fluctuations and ensures power transfer smoothly under various state-switches.

Efficient Actor-Critic Reinforcement Learning With Embodiment of Muscle Tone for Posture Stabilization of the Human Arm

2021 ◽  
Vol 33 (1) ◽  
pp. 129-156
Author(s):  
Masami Iwamoto ◽  
Daichi Kato
Keyword(s):  
Reinforcement Learning ◽  
Muscle Tone ◽  
Muscle Tension ◽  
Experimental Studies ◽  
Control Policy ◽  
Humanoid Robots ◽  
Pedunculopontine Tegmental Nucleus ◽  
Learning Efficiency ◽  
Human Arm ◽  
Posture Stabilization

This letter proposes a new idea to improve learning efficiency in reinforcement learning (RL) with the actor-critic method used as a muscle controller for posture stabilization of the human arm. Actor-critic RL (ACRL) is used for simulations to realize posture controls in humans or robots using muscle tension control. However, it requires very high computational costs to acquire a better muscle control policy for desirable postures. For efficient ACRL, we focused on embodiment that is supposed to potentially achieve efficient controls in research fields of artificial intelligence or robotics. According to the neurophysiology of motion control obtained from experimental studies using animals or humans, the pedunculopontine tegmental nucleus (PPTn) induces muscle tone suppression, and the midbrain locomotor region (MLR) induces muscle tone promotion. PPTn and MLR modulate the activation levels of mutually antagonizing muscles such as flexors and extensors in a process through which control signals are translated from the substantia nigra reticulata to the brain stem. Therefore, we hypothesized that the PPTn and MLR could control muscle tone, that is, the maximum values of activation levels of mutually antagonizing muscles using different sigmoidal functions for each muscle; then we introduced antagonism function models (AFMs) of PPTn and MLR for individual muscles, incorporating the hypothesis into the process to determine the activation level of each muscle based on the output of the actor in ACRL. ACRL with AFMs representing the embodiment of muscle tone successfully achieved posture stabilization in five joint motions of the right arm of a human adult male under gravity in predetermined target angles at an earlier period of learning than the learning methods without AFMs. The results obtained from this study suggest that the introduction of embodiment of muscle tone can enhance learning efficiency in posture stabilization disorders of humans or humanoid robots.

Sign in / Sign up

Export Citation Format

Share Document