Designing Self-Organizing Systems With Deep Multi-Agent Reinforcement Learning

2019 ◽  
Author(s):  
Hao Ji ◽  
Yan Jin
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Task Complexity ◽  
A Priori ◽  
Scale Up ◽  
Simple Task ◽  
Rule Based ◽  
Social Rule ◽  
Multi Agent ◽  
Self Organizing

Abstract Self-organizing systems (SOS) are able to perform complex tasks in unforeseen situations with adaptability. Previous work has introduced field-based approaches and rule-based social structuring for individual agents to not only comprehend the task situations but also take advantage of the social rule-based agent relations in order to accomplish their overall tasks without a centralized controller. Although the task fields and social rules can be predefined for relatively simple task situations, when the task complexity increases and task environment changes, having a priori knowledge about these fields and the rules may not be feasible. In this paper, we propose a multi-agent reinforcement learning based model as a design approach to solving the rule generation problem with complex SOS tasks. A deep multi-agent reinforcement learning algorithm was devised as a mechanism to train SOS agents for acquisition of the task field and social rule knowledge, and the scalability property of this learning approach was investigated with respect to the changing team sizes and environmental noises. Through a set of simulation studies on a box-pushing problem, the results have shown that the SOS design based on deep multi-agent reinforcement learning can be generalizable with different individual settings when the training starts with larger number of agents, but if a SOS is trained with smaller team sizes, the learned neural network does not scale up to larger teams. Design of SOS with a deep reinforcement learning model should keep this in mind and training should be carried out with larger team sizes.

Knowledge Acquisition of Self-organizing Systems with Deep Multiagent Reinforcement Learning

2021 ◽  
pp. 1-47
Author(s):  
Hao Ji ◽  
Yan Jin
Keyword(s):  
Reinforcement Learning ◽  
Knowledge Acquisition ◽  
Learning Algorithm ◽  
Task Complexity ◽  
Functional Differentiation ◽  
Simple Task ◽  
Social Rules ◽  
Rule Based ◽  
Multi Agent ◽  
Self Organizing

Abstract Self-organizing systems (SOS) can perform complex tasks in unforeseen situations with adaptability. Previous work has introduced field-based approaches and rule-based social structuring for individual agents to not only comprehend the task situations but also take advantage of the social rule-based agent relations to accomplish their tasks without a centralized controller. Although the task fields and social rules can be predefined for relatively simple task situations, when the task complexity increases and the task environment changes, having a priori knowledge about these fields and the rules may not be feasible. In this paper, a multi-agent reinforcement learning based model is proposed as a design approach to solving the rule generation problem with complex SOS tasks. A deep multi-agent reinforcement learning algorithm was devised as a mechanism to train SOS agents for knowledge acquisition of the task field and social rules. Learning stability, functional differentiation and robustness properties of this learning approach were investigated with respect to the changing team sizes and task variations. Through computer simulation studies of a box-pushing problem, the results have shown that there is an optimal range of number of agents that achieves good learning stability; agents in a team learn to differentiate from other agents with changing team sizes and box dimensions; and the robustness of the learned knowledge shows to be stronger to the external noises than with changing task constraints.

Improvement on Supporting Machine Learning Algorithm for Solving Problem in Immediate Decision Making

2012 ◽  
Vol 566 ◽  
pp. 572-579
Author(s):  
Abdolkarim Niazi ◽  
Norizah Redzuan ◽  
Raja Ishak Raja Hamzah ◽  
Sara Esfandiari
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Multi Agent Systems ◽  
Combined Model ◽  
Q Learning ◽  
Agent Systems ◽  
Multi Agent ◽  
Case Base ◽  
Case Base Reasoning ◽  
Robotic Tool

In this paper, a new algorithm based on case base reasoning and reinforcement learning (RL) is proposed to increase the convergence rate of the reinforcement learning algorithms. RL algorithms are very useful for solving wide variety decision problems when their models are not available and they must make decision correctly in every state of system, such as multi agent systems, artificial control systems, robotic, tool condition monitoring and etc. In the propose method, we investigate how making improved action selection in reinforcement learning (RL) algorithm. In the proposed method, the new combined model using case base reasoning systems and a new optimized function is proposed to select the action, which led to an increase in algorithms based on Q-learning. The algorithm mentioned was used for solving the problem of cooperative Markov’s games as one of the models of Markov based multi-agent systems. The results of experiments Indicated that the proposed algorithms perform better than the existing algorithms in terms of speed and accuracy of reaching the optimal policy.

A FILTER PROPOSAL FOR INCLUDING FEATURE CONSTRUCTION IN A GENETIC LEARNING ALGORITHM

2012 ◽  
Vol 20 (supp02) ◽  
pp. 31-49 ◽  
Author(s):  
DAVID GARCIA ◽  
ANTONIO GONZALEZ ◽  
RAUL PEREZ
Keyword(s):  
Learning Algorithm ◽  
A Priori ◽  
Fuzzy Rule ◽  
Feature Construction ◽  
Rule Based ◽  
Filter Model ◽  
Information Measures ◽  
Construction Techniques ◽  
Basic Learning ◽  
Current Proposal

In system identification process often a predetermined set of features is used. However, in many cases it is difficult to know a priori whether the selected features were really the more appropriate ones. This is the reason why the feature construction techniques have been very interesting in many applications. Thus, the current proposal introduces the use of these techniques in order to improve the description of fuzzy rule-based systems. In particular, the idea is to include feature construction in a genetic learning algorithm. The construction of attributes in this study will be restricted to the inclusion of functions defined on the initial attributes of the system. Since the number of functions and the number of attributes can be very large, a filter model, based on the use of information measures, is introduced. In this way, the genetic algorithm only needs to explore the particular new features that may be of greater interest to the final identification of the system. In order to manage the knowledge provided by the new attributes based on the use of functions we propose a new model of rule by extending a basic learning fuzzy rule-based model. Finally, we show the experimental study associated with this work.

scholarly journals Coordinated Multi-Agent Deep Reinforcement Learning for Energy-Aware UAV-Based Big-Data Platforms

Electronics ◽  
2021 ◽  
Vol 10 (5) ◽  
pp. 543
Author(s):  
Soyi Jung ◽  
Won Joon Yun ◽  
Joongheon Kim ◽  
Jae-Hyun Kim
Keyword(s):  
Big Data ◽  
Reinforcement Learning ◽  
Resource Sharing ◽  
Learning Algorithm ◽  
Energy Resource ◽  
Energy Aware ◽  
Aerial Surveillance ◽  
Performance Improvements ◽  
Multi Agent ◽  
Energy Sharing

This paper proposes a novel coordinated multi-agent deep reinforcement learning (MADRL) algorithm for energy sharing among multiple unmanned aerial vehicles (UAVs) in order to conduct big-data processing in a distributed manner. For realizing UAV-assisted aerial surveillance or flexible mobile cellular services, robust wireless charging mechanisms are essential for delivering energy sources from charging towers (i.e., charging infrastructure) to their associated UAVs for seamless operations of autonomous UAVs in the sky. In order to actively and intelligently manage the energy resources in charging towers, a MADRL-based coordinated energy management system is desired and proposed for energy resource sharing among charging towers. When the required energy for charging UAVs is not enough in charging towers, the energy purchase from utility company (i.e., energy source provider in local energy market) is desired, which takes high costs. Therefore, the main objective of our proposed coordinated MADRL-based energy sharing learning algorithm is minimizing energy purchase from external utility companies to minimize system-operational costs. Finally, our performance evaluation results verify that the proposed coordinated MADRL-based algorithm achieves desired performance improvements.

Sign in / Sign up

Export Citation Format

Share Document