scholarly journals An Introduction to Intertask Transfer for Reinforcement Learning

AI Magazine ◽  
2011 ◽  
Vol 32 (1) ◽  
pp. 15 ◽  
Author(s):  
Matthew E. Taylor ◽  
Peter Stone
Keyword(s):  
Reinforcement Learning ◽  
Transfer Learning ◽  
Learning Problem ◽  
Open Problems ◽  
Learning Framework ◽  
Learning Domains ◽  
Multiple Tasks ◽  
Exciting Area ◽  
Generalize Information ◽  
Selection Of

Transfer learning has recently gained popularity due to the development of algorithms that can successfully generalize information across multiple tasks. This article focuses on transfer in the context of reinforcement learning domains, a general learning framework where an agent acts in an environment to maximize a reward signal. The goals of this article are to (1) familiarize readers with the transfer learning problem in reinforcement learning domains, (2) explain why the problem is both interesting and difficult, (3) present a selection of existing techniques that demonstrate different solutions, and (4) provide representative open problems in the hope of encouraging additional research in this exciting area.

scholarly journals An Algorithm of Reinforcement Learning for Maneuvering Parameter Self-Tuning Applying in Satellite Cluster

2020 ◽  
Vol 2020 ◽  
pp. 1-17
Author(s):  
Xiao Wang ◽  
Peng Shi ◽  
Changxuan Wen ◽  
Yushan Zhao
Keyword(s):  
Reinforcement Learning ◽  
Traditional Method ◽  
Parameter Method ◽  
Empirical Parameter ◽  
Learning Framework ◽  
Self Tuning ◽  
Flight Parameters ◽  
Cluster A ◽  
Error Ellipsoid ◽  
Selection Of

Satellite cluster is a type of artificial cluster, which is attracting wide attention at present. Although the traditional empirical parameter method (TEPM) has the potential to deal with the mission of satellite flocking, it is difficult to select the proper parameters. In order to improve the flight effect in the problem of satellite cluster, as well as to make the selection of flight parameters more reasonable, the traditional sensing zones are improved. A 3σ position error ellipsoid and an induction ellipsoid are applied for substituting the traditional repulsing zone and attracting zone, respectively. Besides, we propose an algorithm of reinforcement learning for parameter self-tuning (RLPST), which is based on the actor-critic framework, to automatically learn the suitable flight parameters. To obtain the parameters in the repulsing zone, orientating zone, and attracting zone of each member in the cluster, a three-channel learning framework is designed. The learning process makes the framework finally find the suitable parameters. Numerical experimental results have shown the superiorities compared to the traditional method, which include trajectory deviation and sensing rate or terminal matching rate, as well as the improvement of the flight paths under the learning framework.

scholarly journals Mastering Complex Control in MOBA Games with Deep Reinforcement Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 6672-6679 ◽  
Author(s):  
Deheng Ye ◽  
Zhao Liu ◽  
Mingfei Sun ◽  
Bei Shi ◽  
Peilin Zhao ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Action Control ◽  
Learning Problem ◽  
Complex Action ◽  
Complex Control ◽  
Learning Framework ◽  
High Scalability ◽  
Action Spaces ◽  
Level Performance

We study the reinforcement learning problem of complex action control in the Multi-player Online Battle Arena (MOBA) 1v1 games. This problem involves far more complicated state and action spaces than those of traditional 1v1 games, such as Go and Atari series, which makes it very difficult to search any policies with human-level performance. In this paper, we present a deep reinforcement learning framework to tackle this problem from the perspectives of both system and algorithm. Our system is of low coupling and high scalability, which enables efficient explorations at large scale. Our algorithm includes several novel strategies, including control dependency decoupling, action mask, target attention, and dual-clip PPO, with which our proposed actor-critic network can be effectively trained in our system. Tested on the MOBA game Honor of Kings, the trained AI agents can defeat top professional human players in full 1v1 games.

scholarly journals Reinforcement Learning - A Technical Introduction

2019 ◽  
Vol 2 (2) ◽  
pp. 25
Author(s):  
Elmar Diederichs
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Sequential Decision Making ◽  
Sequential Decision ◽  
Learning Problem ◽  
Open Problems ◽  
Recent Developments ◽  
Directed Learning ◽  
Abstract Class ◽  
Algorithmic Approaches

Reinforcement learning provides a cognitive science perspective to behavior and sequential decision making provided that RL-algorithms introduce a computational concept of agency to the learning problem. Hence it addresses an abstract class of problems that can be characterized as follows: An algorithm confronted with information from an unknown environment is supposed to find stepwise an optimal way to behave based only on some sparse, delayed or noisy feedback from some environment, that changes according to the algorithm's behavior. Hence reinforcement learning offers an abstraction to the problem of goal-directed learning from interaction. The paper offers an opintionated introduction in the algorithmic advantages and drawbacks of several algorithmic approaches such that one can understand recent developments and open problems in reinforcement learning.

Transfer Learning

2010 ◽  
pp. 242-264 ◽  
Author(s):  
Lisa Torrey ◽  
Jude Shavlik
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Learning Community ◽  
Transfer Learning ◽  
Inductive Learning ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Task Mapping ◽  
Transfer Of Knowledge ◽  
Open Problems

Transfer learning is the improvement of learning in a new task through the transfer of knowledge from a related task that has already been learned. While most machine learning algorithms are designed to address single tasks, the development of algorithms that facilitate transfer learning is a topic of ongoing interest in the machine-learning community. This chapter provides an introduction to the goals, settings, and challenges of transfer learning. It surveys current research in this area, giving an overview of the state of the art and outlining the open problems. The survey covers transfer in both inductive learning and reinforcement learning, and discusses the issues of negative transfer and task mapping.

scholarly journals Determining the Possibility of Transfer Learning in Deep Reinforcement Learning Using Grad-CAM (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13831-13832
Author(s):  
Ho-Taek Joo ◽  
Kyung-Joong Kim
Keyword(s):  
Reinforcement Learning ◽  
Transfer Learning ◽  
Source Model ◽  
Experimental Results ◽  
Training Time ◽  
Testing Method ◽  
Multiple Tasks ◽  
Speed Up ◽  
Multiple Games

Humans are usually good at guessing whether the two games are similar to each other and easily estimate how much time to master new games based on the similarity. Although Deep Reinforcement Learning (DRL) has been successful in various domains, it takes much training time to get a successful controller for a single game. Therefore, there has been much demand for the use of transfer learning to speed up reinforcement learning across multiple tasks. If we can automatically determine the possibility of transfer learning in DRL domain before training, it could efficiently transfer knowledge across multiple games. In this work, we propose a simple testing method, Determining the Possibility of Transfer Learning (DPTL), to determine the transferability of models based on Grad-CAM visualization of the CNN layer from the source model. Experimental results on Atari games show that the transferability measure is successfully suggesting the possibility of transfer learning.

Sign in / Sign up

Export Citation Format

Share Document