scholarly journals Methods and Algorithms for Knowledge Reuse in Multiagent Reinforcement Learning

2020 ◽  
Author(s):  
Felipe Leno Da Silva ◽  
Anna Helena Reali Costa
Keyword(s):  
Reinforcement Learning ◽  
Transfer Learning ◽  
Learning Process ◽  
Trial And Error ◽  
Knowledge Reuse ◽  
Previous Knowledge ◽  
Learning Methods ◽  
Types Of Knowledge ◽  
Learning Agent ◽  
Multiagent Reinforcement Learning

Reinforcement Learning (RL) is a powerful tool that has been used to solve increasingly complex tasks. RL operates through repeated interactions of the learning agent with the environment, via trial and error. However, this learning process is extremely slow, requiring many interactions. In this thesis, we leverage previous knowledge so as to accelerate learning in multiagent RL problems. We propose knowledge reuse both from previous tasks and from other agents. Several flexible methods are introduced so that each of these two types of knowledge reuse is possible. This thesis adds important steps towards more flexible and broadly applicable multiagent transfer learning methods.

Hierarchical Neuro-Fuzzy Systems Part II

2011 ◽  
pp. 817-824
Author(s):  
Marley Vellasco ◽  
Marco Pacheco ◽  
Karla Figueiredo ◽  
Flavio Souza
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Learning Process ◽  
Fuzzy Systems ◽  
Space Partitioning ◽  
Learning Methods ◽  
New Class ◽  
Neuro Fuzzy ◽  
Binary Space Partitioning ◽  
Priori Information

This paper describes a new class of neuro-fuzzy models, called Reinforcement Learning Hierarchical Neuro- Fuzzy Systems (RL-HNF). These models employ the BSP (Binary Space Partitioning) and Politree partitioning of the input space [Chrysanthou,1992] and have been developed in order to bypass traditional drawbacks of neuro-fuzzy systems: the reduced number of allowed inputs and the poor capacity to create their own structure and rules (ANFIS [Jang,1997], NEFCLASS [Kruse,1995] and FSOM [Vuorimaa,1994]). These new models, named Reinforcement Learning Hierarchical Neuro-Fuzzy BSP (RL-HNFB) and Reinforcement Learning Hierarchical Neuro-Fuzzy Politree (RL-HNFP), descend from the original HNFB that uses Binary Space Partitioning (see Hierarchical Neuro-Fuzzy Systems Part I). By using hierarchical partitioning, together with the Reinforcement Learning (RL) methodology, a new class of Neuro-Fuzzy Systems (SNF) was obtained, which executes, in addition to automatically learning its structure, the autonomous learning of the actions to be taken by an agent, dismissing a priori information (number of rules, fuzzy rules and sets) relative to the learning process. These characteristics represent an important differential when compared with existing intelligent agents learning systems, because in applications involving continuous environments and/or environments considered to be highly dimensional, the use of traditional Reinforcement Learning methods based on lookup tables (a table that stores value functions for a small or discrete state space) is no longer possible, since the state space becomes too large. This second part of hierarchical neuro-fuzzy systems focus on the use of reinforcement learning process. The first part presented HNFB models based on supervised learning methods. The RL-HNFB and RL-HNFP models were evaluated in a benchmark control application and a simulated Khepera robot environment with multiple obstacles.

scholarly journals Combined Reinforcement Learning via Abstract Representations

2019 ◽  
Vol 33 ◽  
pp. 3582-3589 ◽  
Author(s):  
Vincent Francois-Lavet ◽  
Yoshua Bengio ◽  
Doina Precup ◽  
Joelle Pineau
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Transfer Learning ◽  
Computationally Efficient ◽  
Dimensional Representation ◽  
Learning Methods ◽  
Model Free ◽  
Abstract Representations ◽  
Low Dimensional ◽  
New Strategies

In the quest for efficient and robust reinforcement learning methods, both model-free and model-based approaches offer advantages. In this paper we propose a new way of explicitly bridging both approaches via a shared low-dimensional learned encoding of the environment, meant to capture summarizing abstractions. We show that the modularity brought by this approach leads to good generalization while being computationally efficient, with planning happening in a smaller latent state space. In addition, this approach recovers a sufficient low-dimensional representation of the environment, which opens up new strategies for interpretable AI, exploration and transfer learning.

scholarly journals A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

2019 ◽  
Vol 64 ◽  
pp. 645-703 ◽  
Author(s):  
Felipe Leno Da Silva ◽  
Anna Helena Reali Costa
Keyword(s):  
Reinforcement Learning ◽  
Scale Up ◽  
Shared Environment ◽  
Complex Task ◽  
General Knowledge ◽  
Sample Complexity ◽  
Knowledge Reuse ◽  
Open Questions ◽  
Multiagent Reinforcement Learning ◽  
Current Lines

Multiagent Reinforcement Learning (RL) solves complex tasks that require coordination with other agents through autonomous exploration of the environment. However, learning a complex task from scratch is impractical due to the huge sample complexity of RL algorithms. For this reason, reusing knowledge that can come from previous experience or other agents is indispensable to scale up multiagent RL algorithms. This survey provides a unifying view of the literature on knowledge reuse in multiagent RL. We define a taxonomy of solutions for the general knowledge reuse problem, providing a comprehensive discussion of recent progress on knowledge reuse in Multiagent Systems (MAS) and of techniques for knowledge reuse across agents (that may be actuating in a shared environment or not). We aim at encouraging the community to work towards reusing all the knowledge sources available in a MAS. For that, we provide an in-depth discussion of current lines of research and open questions.

scholarly journals Reinforcement Learning for Systematic FX Trading

2021 ◽  
Author(s):  
Gabriel Borrageiro ◽  
Nick Firoozye ◽  
Paolo Barucca
Keyword(s):  
Reinforcement Learning ◽  
Transfer Learning ◽  
Radial Basis Function Network ◽  
Gaussian Mixture ◽  
Feature Representation ◽  
Currency Markets ◽  
Learning Context ◽  
Information Ratio ◽  
Learning Agent ◽  
Inductive Transfer Learning

We explore online inductive transfer learning, with a feature representation transfer from a radial basis function network formed of Gaussian mixture model hidden processing units to a direct, recurrent reinforcement learning agent. This agent is put to work in an experiment, trading the major spot market currency pairs, where we accurately account for transaction and funding costs. These sources of profit and loss, including the price trends that occur in the currency markets, are made available to the agent via a quadratic utility, who learns to target a position directly. We improve upon earlier work by learning to target a risk position in an online transfer learning context. Our agent achieves an annualised portfolio information ratio of 0.52 with a compound return of 9.3%, net of execution and funding cost, over a 7-year test set; this is despite forcing the model to trade at the close of the trading day at 5 pm EST when trading costs are statistically the most expensive.<br>

scholarly journals Autonomously Reusing Knowledge in Multiagent Reinforcement Learning

2018 ◽  
Author(s):  
Felipe Leno Da Silva ◽  
Matthew E. Taylor ◽  
Anna Helena Reali Costa
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Agents ◽  
Knowledge Reuse ◽  
Challenging Problem ◽  
Multiple Sources ◽  
Relevant Research ◽  
Core Technology ◽  
Multiagent Reinforcement Learning ◽  
Important Challenge ◽  
Research Challenge

Autonomous agents are increasingly required to solve complex tasks; hard-coding behaviors has become infeasible. Hence, agents must learn how to solve tasks via interactions with the environment. In many cases, knowledge reuse will be a core technology to keep training times reasonable, and for that, agents must be able to autonomously and consistently reuse knowledge from multiple sources, including both their own previous internal knowledge and from other agents. In this paper, we provide a literature review of methods for knowledge reuse in Multiagent Reinforcement Learning. We define an important challenge problem for the AI community, survey the existent methods, and discuss how they can all contribute to this challenging problem. Moreover, we highlight gaps in the current literature, motivating "low-hanging fruit'' for those interested in the area. Our ambition is that this paper will encourage the community to work on this difficult and relevant research challenge.

scholarly journals Coordinated Learning by Model Difference Identification in Multiagent Systems with Sparse Interactions

2016 ◽  
Vol 2016 ◽  
pp. 1-17
Author(s):  
Qi Zhang ◽  
Peng Jiao ◽  
Quanjun Yin ◽  
Lin Sun
Keyword(s):  
Reinforcement Learning ◽  
Multiagent Systems ◽  
Learning Process ◽  
Independent Learning ◽  
Promising Technique ◽  
Joint Learning ◽  
State Action ◽  
Multiagent Reinforcement Learning ◽  
General Mass ◽  
Coordinated Learning

Multiagent Reinforcement Learning (MARL) is a promising technique for agents learning effective coordinated policy in Multiagent Systems (MASs). In many MASs, interactions between agents are usually sparse, and then a lot of MARL methods were devised for them. These methods divide learning process into independent learning and joint learning in coordinated states to improve traditional joint state-action space learning. However, most of those methods identify coordinated states based on assumptions about domain structure (e.g., dependencies) or agent (e.g., prior individual optimal policy and agent homogeneity). Moreover, situations that current methods cannot deal with still exist. In this paper, a modified approach is proposed to learn where and how to coordinate agents’ behaviors in more general MASs with sparse interactions. Our approach introduces sample grouping and a more accurate metric of model difference degree to identify which states of other agents should be considered in coordinated states, without strong additional assumptions. Experimental results show that the proposed approach outperforms its competitors by improving the average agent reward per step and works well in some broader scenarios.

scholarly journals Attention Guided Imitation Learning and Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 9906-9907
Author(s):  
Ruohan Zhang
Keyword(s):  
Reinforcement Learning ◽  
Learning Process ◽  
Human Action ◽  
Imitation Learning ◽  
Tracking Data ◽  
Experimental Setting ◽  
High Quality ◽  
Attention Model ◽  
Learning Agent ◽  
Human Visual Attention

We propose a framework that uses learned human visual attention model to guide the learning process of an imitation learning or reinforcement learning agent. We have collected high-quality human action and eye-tracking data while playing Atari games in a carefully controlled experimental setting. We have shown that incorporating a learned human gaze model into deep imitation learning yields promising results.

Sign in / Sign up

Export Citation Format

Share Document