A Proposal For Jerk-Suppressed Action Planning Using Deep Reinforcement Learning

Kota SHIMADA; Takumi MATSUDA; Yoji KURODA

doi:10.1299/jsmermd.2021.1a1-g12

A Proposal For Jerk-Suppressed Action Planning Using Deep Reinforcement Learning

The Proceedings of JSME annual Conference on Robotics and Mechatronics (Robomec) ◽

10.1299/jsmermd.2021.1a1-g12 ◽

2021 ◽

Vol 2021 (0) ◽

pp. 1A1-G12

Author(s):

Kota SHIMADA ◽

Takumi MATSUDA ◽

Yoji KURODA

Keyword(s):

Reinforcement Learning ◽

Action Planning

Download Full-text

A Reinforcement Learning Based Algorithm for Robot Action Planning

Advances in Service and Industrial Robotics - Mechanisms and Machine Science ◽

10.1007/978-3-030-00232-9_52 ◽

2018 ◽

pp. 493-503 ◽

Cited By ~ 5

Author(s):

Marko Švaco ◽

Bojan Jerbić ◽

Mateo Polančec ◽

Filip Šuligoj

Keyword(s):

Reinforcement Learning ◽

Action Planning

Download Full-text

Robust Reinforcement Learning

Neural Computation ◽

10.1162/0899766053011528 ◽

2005 ◽

Vol 17 (2) ◽

pp. 335-359 ◽

Cited By ~ 45

Author(s):

Jun Morimoto ◽

Kenji Doya

Keyword(s):

Reinforcement Learning ◽

Value Function ◽

Learning Algorithm ◽

Action Planning ◽

Control Agent ◽

Control Input ◽

Environmental Models ◽

Online Learning Algorithms ◽

The Difference ◽

The Value Function

This letter proposes a new reinforcement learning (RL) paradigm that explicitly takes into account input disturbance as well as modeling errors. The use of environmental models in RL is quite popular for both off-line learning using simulations and for online action planning. However, the difference between the model and the real environment can lead to unpredictable, and often unwanted, results. Based on the theory of H∞ control, we consider a differential game in which a “disturbing” agent tries to make the worst possible disturbance while a “control” agent tries to make the best control input. The problem is formulated as finding a min-max solution of a value function that takes into account the amount of the reward and the norm of the disturbance. We derive online learning algorithms for estimating the value function and for calculating the worst disturbance and the best control in reference to the value function. We tested the paradigm, which we call robust reinforcement learning (RRL), on the control task of an inverted pendulum. In the linear domain, the policy and the value function learned by online algorithms coincided with those derived analytically by the linear H∞ control theory. For a fully nonlinear swing-up task, RRL achieved robust performance with changes in the pendulum weight and friction, while a standard reinforcement learning algorithm could not deal with these changes. We also applied RRL to the cart-pole swing-up task, and a robust swing-up policy was acquired.

Download Full-text

Task Transfer by Preference-Based Cost Learning

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33012471 ◽

2019 ◽

Vol 33 ◽

pp. 2471-2478

Author(s):

Mingxuan Jing ◽

Xiaojian Ma ◽

Wenbing Huang ◽

Fuchun Sun ◽

Huaping Liu

Keyword(s):

Reinforcement Learning ◽

Theoretical Analysis ◽

Cost Function ◽

Action Planning ◽

Target Task ◽

Target Distribution ◽

Target Cost ◽

Task Transfer ◽

Action Policy

The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task. Given their successes on robotic action planning, current methods mostly rely on two requirements: exactlyrelevant expert demonstrations or the explicitly-coded cost function on target task, both of which, however, are inconvenient to obtain in practice. In this paper, we relax these two strong conditions by developing a novel task transfer framework where the expert preference is applied as a guidance. In particular, we alternate the following two steps: Firstly, letting experts apply pre-defined preference rules to select related expert demonstrates for the target task. Secondly, based on the selection result, we learn the target cost function and trajectory distribution simultaneously via enhanced Adversarial MaxEnt IRL and generate more trajectories by the learned target distribution for the next preference selection. The theoretical analysis on the distribution learning and convergence of the proposed algorithm are provided. Extensive simulations on several benchmarks have been conducted for further verifying the effectiveness of the proposed method.

Download Full-text