scholarly journals Novel Algorithm for Agent Navigation Based on Intrinsic Motivation Due to Boredom

2021 ◽  
Vol 50 (3) ◽  
pp. 485-494
Author(s):  
Oscar Loyola ◽  
John Kern ◽  
Claudio Urrea
Keyword(s):  
Reinforcement Learning ◽  
Intrinsic Motivation ◽  
Navigation Algorithm ◽  
Agent Navigation ◽  
The Creation ◽  
Novel Algorithm

We propose a novel algorithm for the navigation of agents based on reinforcement learning, using boredomas an element of intrinsic motivation. Improvements obtained with the inclusion of this element over classicstrategies are shown through simulations. Boredom is modeled through a chaotic element that generates conditionsfor the creation of routes when the environment does not offer any reward, allowing prompting the robotto navigate. Our proposal seeks to avoid what classical algorithms suffer in scenarios without rewards, generatinglosses of time in the resolution. We demonstrate experimentally that by adding the element of boredomit is possible to generate routes in scenarios in which rewards do not exist, allowing the use of these strategiesin real circumstances and facilitating the robot's navigation towards its objective. The most important contributionsustained by this work corresponds to the fact that it is possible to improve navigation in completelyadverse scenarios for a navigation algorithm based on rewards.

scholarly journals A study on handling intrinsic motivation for devising sample efficient actor-critic agents

2021 ◽  
Author(s):  
André Quadros ◽  
Roberto Xavier Junior ◽  
Kleber Souza ◽  
Bruno Gomes ◽  
Filipe Saraiva ◽  
...  
Keyword(s):  
Machine Learning ◽  
Experimental Study ◽  
Reinforcement Learning ◽  
Intrinsic Motivation ◽  
Sampling Efficiency ◽  
Practical Guidelines

Reinforcement learning has evolved in recent years,and overcoming challenges found in this field. This area, unlikeconventional machine learning, does not learn through a setof observational instances, but through interaction with anenvironment. The sampling efficiency of a reinforcement learningagent is a challenge. That is, how to make an agent learn withinan environment with as little interaction as possible. In this workwe perform an experimental study on the difficulties to integratea strategy of intrinsic motivation to an actor-critic agent toimprove the sampling efficiency. We found results that point to theeffectiveness of the intrinsic motivation as a approach to improvethe agent’s sampling efficiency, as well as its performance. Weshare practical guidelines to assist in the implementation of actor-critic agents to deal with sparse reward environments whilemaking use of intrinsic motivation feedback.

scholarly journals End-to-End Autonomous Exploration with Deep Reinforcement Learning and Intrinsic Motivation

2021 ◽  
Vol 2021 ◽  
pp. 1-15
Author(s):  
Xiaogang Ruan ◽  
Peng Li ◽  
Xiaoqing Zhu ◽  
Hejie Yu ◽  
Naigong Yu
Keyword(s):  
Reinforcement Learning ◽  
Intrinsic Motivation ◽  
Driving Forces ◽  
Temporal Distance ◽  
Training Methods ◽  
Complex Environments ◽  
Learning Problem ◽  
Autonomous Exploration ◽  
Exploration Behavior ◽  
Efficient Exploration

Developing artificial intelligence (AI) agents is challenging for efficient exploration in visually rich and complex environments. In this study, we formulate the exploration question as a reinforcement learning problem and rely on intrinsic motivation to guide exploration behavior. Such intrinsic motivation is driven by curiosity and is calculated based on episode memory. To distribute the intrinsic motivation, we use a count-based method and temporal distance to generate it synchronously. We tested our approach in 3D maze-like environments and validated its performance in exploration tasks through extensive experiments. The experimental results show that our agent can learn exploration ability from raw sensory input and accomplish autonomous exploration across different mazes. In addition, the learned policy is not biased by stochastic objects. We also analyze the effects of different training methods and driving forces on exploration policy.

scholarly journals Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning

2019 ◽  
Vol 30 (11) ◽  
pp. 3409-3418 ◽  
Author(s):  
Nat Dilokthanakul ◽  
Christos Kaplanis ◽  
Nick Pawlowski ◽  
Murray Shanahan
Keyword(s):  
Reinforcement Learning ◽  
Intrinsic Motivation ◽  
Hierarchical Reinforcement Learning

scholarly journals Learning Representations in Model-Free Hierarchical Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 10009-10010
Author(s):  
Jacob Rafati ◽  
David C. Noelle
Keyword(s):  
Reinforcement Learning ◽  
Intrinsic Motivation ◽  
Large Scale ◽  
Temporal Abstraction ◽  
State Spaces ◽  
Hierarchical Reinforcement Learning ◽  
Model Free ◽  
Small Set ◽  
Multiple Levels ◽  
Novel Model

Common approaches to Reinforcement Learning (RL) are seriously challenged by large-scale applications involving huge state spaces and sparse delayed reward feedback. Hierarchical Reinforcement Learning (HRL) methods attempt to address this scalability issue by learning action selection policies at multiple levels of temporal abstraction. Abstraction can be had by identifying a relatively small set of states that are likely to be useful as subgoals, in concert with the learning of corresponding skill policies to achieve those subgoals. Many approaches to subgoal discovery in HRL depend on the analysis of a model of the environment, but the need to learn such a model introduces its own problems of scale. Once subgoals are identified, skills may be learned through intrinsic motivation, introducing an internal reward signal marking subgoal attainment. We present a novel model-free method for subgoal discovery using incremental unsupervised learning over a small memory of the most recent experiences of the agent. When combined with an intrinsic motivation learning mechanism, this method learns subgoals and skills together, based on experiences in the environment. Thus, we offer an original approach to HRL that does not require the acquisition of a model of the environment, suitable for large-scale applications. We demonstrate the efficiency of our method on a variant of the rooms environment.

scholarly journals A Review of the Relationship between Novelty, Intrinsic Motivation and Reinforcement Learning

2017 ◽  
Vol 8 (1) ◽  
pp. 58-69 ◽  
Author(s):  
Nazmul Siddique ◽  
Paresh Dhakan ◽  
Inaki Rano ◽  
Kathryn Merrick
Keyword(s):  
Reinforcement Learning ◽  
Intrinsic Motivation ◽  
Computational Models ◽  
Functional Relationship ◽  
Novelty Detection ◽  
Literature Survey ◽  
Reward Function ◽  
Hedonic Value ◽  
Survey Results ◽  
The Relationship

Abstract This paper presents a review on the tri-partite relationship between novelty, intrinsic motivation and reinforcement learning. The paper first presents a literature survey on novelty and the different computational models of novelty detection, with a specific focus on the features of stimuli that trigger a Hedonic value for generating a novelty signal. It then presents an overview of intrinsic motivation and investigations into different models with the aim of exploring deeper co-relationships between specific features of a novelty signal and its effect on intrinsic motivation in producing a reward function. Finally, it presents survey results on reinforcement learning, different models and their functional relationship with intrinsic motivation.

Diversity-Augmented Intrinsic Motivation for Deep Reinforcement Learning

2021 ◽  
Author(s):  
Tianhong Dai ◽  
Yali Du ◽  
Meng Fang ◽  
Anil Anthony Bharath
Keyword(s):  
Reinforcement Learning ◽  
Intrinsic Motivation
Sign in / Sign up

Export Citation Format

Share Document