Bayesian Optimization for Efficient Tuning of Visual Servo and Computed Torque Controllers in a Reinforcement Learning Scenario

Similar to their counterparts in nature, the flexible bodies of snake-like robots enhance their movement capability and adaptability in diverse environments. However, this flexibility corresponds to a complex control task involving highly redundant degrees of freedom, where traditional model-based methods usually fail to propel the robots energy-efficiently. In this work, we present a novel approach for designing an energy-efficient slithering gait for a snake-like robot using a model-free reinforcement learning (RL) algorithm. Specifically, we present an RL-based controller for generating locomotion gaits at a wide range of velocities, which is trained using the proximal policy optimization (PPO) algorithm. Meanwhile, a traditional parameterized gait controller is presented and the parameter sets are optimized using the grid search and Bayesian optimization algorithms for the purposes of reasonable comparisons. Based on the analysis of the simulation results, we demonstrate that this RL-based controller exhibits very natural and adaptive movements, which are also substantially more energy-efficient than the gaits generated by the parameterized controller. Videos are shown at https://videoviewsite.wixsite.com/rlsnake .

Download Full-text

Fog Computing Resource Management for Video Processing Using Evolutionary Bayesian Optimization and Nondeterministic Deep Reinforcement Learning

Military Operations Research ◽

10.5711/1082598326441 ◽

2021 ◽

Vol 26 (4) ◽

pp. 41-66

Author(s):

Buddy Aswin

Keyword(s):

Reinforcement Learning ◽

Resource Management ◽

Video Processing ◽

Fog Computing ◽

Bayesian Optimization ◽

Computing Resource

Download Full-text

Distance-directed Target Searching for a Deep Visual Servo SMA Driven Soft Robot Using Reinforcement Learning

Journal of Bionic Engineering ◽

10.1007/s42235-020-0102-8 ◽

2020 ◽

Vol 17 (6) ◽

pp. 1126-1138

Author(s):

Wuji Liu ◽

Zhongliang Jing ◽

Han Pan ◽

Lingfeng Qiao ◽

Henry Leung ◽

...

Keyword(s):

Reinforcement Learning ◽

Soft Robot ◽

Visual Servo ◽

Target Searching

Download Full-text

Improving the surface quality of friction stir welds using reinforcement learning and Bayesian optimization

The International Journal of Advanced Manufacturing Technology ◽

10.1007/s00170-020-05696-x ◽

2020 ◽

Vol 110 (11-12) ◽

pp. 3145-3167

Author(s):

R. Hartl ◽

J. Hansjakob ◽

M. F. Zaeh

Keyword(s):

Reinforcement Learning ◽

Surface Topography ◽

Welding Speed ◽

Rotational Speed ◽

Optimization Problem ◽

The Other ◽

Bayesian Optimization ◽

Welding Parameters ◽

Value Iteration ◽

Friction Stir

Abstract Friction stir welding is an advanced joining technology that is particularly suitable for aluminum alloys. Various studies have shown a significant dependence of the welding quality on the welding speed and the rotational speed of the tool. Frequently, an inappropriate setting of these parameters can be detected through an examination of the resulting surface defects, such as increased flash formation or surface galling. In this work, two different learning-based algorithms were applied to improve the surface topography of friction stir welds. For this purpose, the surface topographies of 262 welds, which were performed as part of ten studies, were evaluated offline. The aim was to use reinforcement learning and Bayesian optimization approaches to determine the most appropriate settings for the welding speed and the rotational speed of the tool. The optimization problem was solved using reinforcement learning, specifically value iteration. However, the value iteration algorithm was not efficient, since all actions and states had to be iterated over, i.e., each possible parameter combination had to be evaluated, to find the best policy. Instead, it was better to solve the optimization problem directly using the Bayesian optimization. Two approaches were applied: both an approach in which the information from the other studies was not used and an approach in which the information from the other studies was used. On average, both the Bayesian optimization approaches found suitable welding parameters significantly faster than a random search algorithm, and the latter approach improved the result even further compared with the former approach. Future research will aim to show that optimization of the surface topography also leads to an increase in the ultimate tensile strength.

Download Full-text