scholarly journals Exploration Entropy for Reinforcement Learning

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Bo Xin ◽  
Haixu Yu ◽  
You Qin ◽  
Qing Tang ◽  
Zhangqing Zhu
Keyword(s):  
Reinforcement Learning ◽  
Theoretical Analysis ◽  
Learning Process ◽  
Process Analysis ◽  
Action Selection ◽  
Training Process ◽  
New Approach ◽  
State Entropy ◽  
Key Issues ◽  
New Criterion

The training process analysis and termination condition of the training process of a Reinforcement Learning (RL) system have always been the key issues to train an RL agent. In this paper, a new approach based on State Entropy and Exploration Entropy is proposed to analyse the training process. The concept of State Entropy is used to denote the uncertainty for an RL agent to select the action at every state that the agent will traverse, while the Exploration Entropy denotes the action selection uncertainty of the whole system. Actually, the action selection uncertainty of a certain state or the whole system reflects the degree of exploration and the stage of the learning process for an agent. The Exploration Entropy is a new criterion to analyse and manage the training process of RL. The theoretical analysis and experiment results illustrate that the curve of Exploration Entropy contains more information than the existing analytical methods.

scholarly journals Deep Reinforcement Learning via Past-Success Directed Exploration

2019 ◽  
Vol 33 ◽  
pp. 9979-9980
Author(s):  
Xiaoming Liu ◽  
Zhixiong Xu ◽  
Lei Cao ◽  
Xiliang Chen ◽  
Kai Kang
Keyword(s):  
Online Learning ◽  
Adaptive Control ◽  
Reinforcement Learning ◽  
Learning Process ◽  
Control Method ◽  
Learning Algorithms ◽  
Action Selection ◽  
Experimental Results ◽  
Continuous Control ◽  
Exploration And Exploitation

The balance between exploration and exploitation has always been a core challenge in reinforcement learning. This paper proposes “past-success exploration strategy combined with Softmax action selection”(PSE-Softmax) as an adaptive control method for taking advantage of the characteristics of the online learning process of the agent to adapt exploration parameters dynamically. The proposed strategy is tested on OpenAI Gym with discrete and continuous control tasks, and the experimental results show that PSE-Softmax strategy delivers better performance than deep reinforcement learning algorithms with basic exploration strategies.

scholarly journals Applying and Augmenting Deep Reinforcement Learning in Serious Games through Interaction

2017 ◽  
Vol 61 (2) ◽  
pp. 198 ◽  
Author(s):  
Aline Dobrovsky ◽  
Uwe M. Borghoff ◽  
Marko Hofmann
Keyword(s):  
Reinforcement Learning ◽  
Learning Process ◽  
Serious Games ◽  
Adaptive Behaviour ◽  
Cognitive Systems ◽  
New Approach ◽  
Domain Experts ◽  
E Learning ◽  
Short Horizon ◽  
And Training

Serious games belong to the most important future e-learning trends and are frequently used in recruitment and training. Their development, however, is still a demanding and tedious process, especially when regarding reasonable non-player character behaviour. Serious games can generally profit from diverse, adaptive behaviour to increase learning effectiveness. Deep reinforcement learning has already shown considerable results in automatically generating successful AI behaviour, but its past applications were mainly focused on optimization and short-horizon games. To expand the underlying ideas to serious games, we introduce a new approach of augmenting the application of deep reinforcement learning methods by interactively making use of domain experts’ knowledge to guide the learning process. Thereby, we aim to establish a synergistic combination of experts and emergent cognitive systems to create adaptive and more human behaviour. We call this approach interactive deep reinforcement learning and point out important aspects regarding realization within a novel framework.

scholarly journals Forced ε-Greedy, an Expansion to the ε-Greedy Action Selection Method

2021 ◽  
Author(s):  
George Angelopoulos ◽  
Dimitris Metafas
Keyword(s):  
Reinforcement Learning ◽  
Action Selection ◽  
Selection Method ◽  
Board Game ◽  
Selection Methods ◽  
Training Process ◽  
Greedy Method ◽  
Learning Methods ◽  
Q Learning ◽  
Time Required

Reinforcement Learning methods such as Q Learning, make use of action selection methods, in order to train an agent to perform a task. As the complexity of the task grows, so does the time required to train the agent. In this paper Q Learning is applied onto the board game Dominion, and Forced ε-greedy, an expansion to the ε-greedy action selection method is introduced. As shown in this paper the Forced ε-greedy method achieves to accelerate the training process and optimize its results, especially as the complexity of the task grows.

scholarly journals Goal-driven active learning

2021 ◽  
Vol 35 (2) ◽  
Author(s):  
Nicolas Bougie ◽  
Ryutaro Ichise
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Learning Process ◽  
Real World ◽  
Imitation Learning ◽  
Learning Approaches ◽  
Wide Range ◽  
Fixed Set ◽  
Complex Decision Making ◽  
Complex Decision

AbstractDeep reinforcement learning methods have achieved significant successes in complex decision-making problems. In fact, they traditionally rely on well-designed extrinsic rewards, which limits their applicability to many real-world tasks where rewards are naturally sparse. While cloning behaviors provided by an expert is a promising approach to the exploration problem, learning from a fixed set of demonstrations may be impracticable due to lack of state coverage or distribution mismatch—when the learner’s goal deviates from the demonstrated behaviors. Besides, we are interested in learning how to reach a wide range of goals from the same set of demonstrations. In this work we propose a novel goal-conditioned method that leverages very small sets of goal-driven demonstrations to massively accelerate the learning process. Crucially, we introduce the concept of active goal-driven demonstrations to query the demonstrator only in hard-to-learn and uncertain regions of the state space. We further present a strategy for prioritizing sampling of goals where the disagreement between the expert and the policy is maximized. We evaluate our method on a variety of benchmark environments from the Mujoco domain. Experimental results show that our method outperforms prior imitation learning approaches in most of the tasks in terms of exploration efficiency and average scores.

scholarly journals Diversity oriented Deep Reinforcement Learning for targeted molecule generation

2021 ◽  
Vol 13 (1) ◽  
Author(s):  
Tiago Pereira ◽  
Maryam Abbasi ◽  
Bernardete Ribeiro ◽  
Joel P. Arrais
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Reinforcement Learning ◽  
Deep Neural Networks ◽  
Chemical Space ◽  
Biological Properties ◽  
Training Process ◽  
Training Strategy ◽  
Inhibitory Power ◽  
Exploratory Strategy

AbstractIn this work, we explore the potential of deep learning to streamline the process of identifying new potential drugs through the computational generation of molecules with interesting biological properties. Two deep neural networks compose our targeted generation framework: the Generator, which is trained to learn the building rules of valid molecules employing SMILES strings notation, and the Predictor which evaluates the newly generated compounds by predicting their affinity for the desired target. Then, the Generator is optimized through Reinforcement Learning to produce molecules with bespoken properties. The innovation of this approach is the exploratory strategy applied during the reinforcement training process that seeks to add novelty to the generated compounds. This training strategy employs two Generators interchangeably to sample new SMILES: the initially trained model that will remain fixed and a copy of the previous one that will be updated during the training to uncover the most promising molecules. The evolution of the reward assigned by the Predictor determines how often each one is employed to select the next token of the molecule. This strategy establishes a compromise between the need to acquire more information about the chemical space and the need to sample new molecules, with the experience gained so far. To demonstrate the effectiveness of the method, the Generator is trained to design molecules with an optimized coefficient of partition and also high inhibitory power against the Adenosine $$A_{2A}$$ A 2 A and $$\kappa$$ κ opioid receptors. The results reveal that the model can effectively adjust the newly generated molecules towards the wanted direction. More importantly, it was possible to find promising sets of unique and diverse molecules, which was the main purpose of the newly implemented strategy.

scholarly journals A strategy learning model for autonomous agents based on classification

2015 ◽  
Vol 25 (3) ◽  
pp. 471-482 ◽  
Author(s):  
Bartłomiej Śnieżyński
Keyword(s):  
Reinforcement Learning ◽  
Supervised Learning ◽  
Learning Process ◽  
Autonomous Agents ◽  
Good Alternative ◽  
Learning Model ◽  
Learning Method ◽  
Complex Environments ◽  
Agent Based ◽  
Proposed Model

AbstractIn this paper we propose a strategy learning model for autonomous agents based on classification. In the literature, the most commonly used learning method in agent-based systems is reinforcement learning. In our opinion, classification can be considered a good alternative. This type of supervised learning can be used to generate a classifier that allows the agent to choose an appropriate action for execution. Experimental results show that this model can be successfully applied for strategy generation even if rewards are delayed. We compare the efficiency of the proposed model and reinforcement learning using the farmer-pest domain and configurations of various complexity. In complex environments, supervised learning can improve the performance of agents much faster that reinforcement learning. If an appropriate knowledge representation is used, the learned knowledge may be analyzed by humans, which allows tracking the learning process

Simulation and Research of Dynamic Voltage Detection and Regulator Based on MATLAB

2014 ◽  
Vol 536-537 ◽  
pp. 1527-1531
Author(s):  
Ya Feng Li ◽  
Zi Wei Zheng
Keyword(s):  
Theoretical Analysis ◽  
Reactive Power ◽  
Voltage Regulator ◽  
New Approach ◽  
Dynamic Voltage ◽  
Harmonic Voltage ◽  
Simulation Results ◽  
Harmonic Source ◽  
Detecting Method ◽  
Dynamic Voltage Regulator

The Series Dynamic Voltage Regulator can compensate the harmonics distortion caused by voltage type harmonic source This paper presents a new approach of detecting harmonic voltage in dq0 coordinates, based on the generalized instantaneous reactive power ,and used in the series dynamic voltage regulator successfully. It is demonstrated by theoretical analysis and simulation results that the proposed detecting method of harmonic voltage is correct and valid.

Sign in / Sign up

Export Citation Format

Share Document