scholarly journals Neighborhood Cognition Consistent Multi-Agent Reinforcement Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 7219-7226
Author(s):  
Hangyu Mao ◽  
Wulong Liu ◽  
Jianye Hao ◽  
Jun Luo ◽  
Dong Li ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Football Player ◽  
Superior Performance ◽  
Human Society ◽  
Packet Routing ◽  
Q Learning ◽  
Cognitive Consistency ◽  
Challenging Tasks ◽  
Multi Agent

Social psychology and real experiences show that cognitive consistency plays an important role to keep human society in order: if people have a more consistent cognition about their environments, they are more likely to achieve better cooperation. Meanwhile, only cognitive consistency within a neighborhood matters because humans only interact directly with their neighbors. Inspired by these observations, we take the first step to introduce neighborhood cognitive consistency (NCC) into multi-agent reinforcement learning (MARL). Our NCC design is quite general and can be easily combined with existing MARL methods. As examples, we propose neighborhood cognition consistent deep Q-learning and Actor-Critic to facilitate large-scale multi-agent cooperations. Extensive experiments on several challenging tasks (i.e., packet routing, wifi configuration and Google football player control) justify the superior performance of our methods compared with state-of-the-art MARL approaches.

scholarly journals Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces

2019 ◽  
Author(s):  
Haotian Fu ◽  
Hongyao Tang ◽  
Jianye Hao ◽  
Zihan Lei ◽  
Yingfeng Chen ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Continuous Action ◽  
Q Learning ◽  
Challenging Tasks ◽  
Discrete Action ◽  
Multi Agent ◽  
Decentralized Execution ◽  
Novel Algorithms ◽  
Action Spaces ◽  
Different Levels

Deep Reinforcement Learning (DRL) has been applied to address a variety of cooperative multi-agent problems with either discrete action spaces or continuous action spaces. However, to the best of our knowledge, no previous work has ever succeeded in applying DRL to multi-agent problems with discrete-continuous hybrid (or parameterized) action spaces which is very common in practice. Our work fills this gap by proposing two novel algorithms: Deep Multi-Agent Parameterized Q-Networks (Deep MAPQN) and Deep Multi-Agent Hierarchical Hybrid Q-Networks (Deep MAHHQN). We follow the centralized training but decentralized execution paradigm: different levels of communication between different agents are used to facilitate the training process, while each agent executes its policy independently based on local observations during execution. Our empirical results on several challenging tasks (simulated RoboCup Soccer and game Ghost Story) show that both Deep MAPQN and Deep MAHHQN are effective and significantly outperform existing independent deep parameterized Q-learning method.

scholarly journals A Confrontation Decision-Making Method with Deep Reinforcement Learning and Knowledge Transfer for Multi-Agent System

Symmetry ◽  
2020 ◽  
Vol 12 (4) ◽  
pp. 631
Author(s):  
Chunyang Hu
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Knowledge Transfer ◽  
Large Scale ◽  
Effective Control ◽  
Small Scale ◽  
Learning Agent ◽  
Multi Agent ◽  
Transfer Method ◽  
Parameter Sharing

In this paper, deep reinforcement learning (DRL) and knowledge transfer are used to achieve the effective control of the learning agent for the confrontation in the multi-agent systems. Firstly, a multi-agent Deep Deterministic Policy Gradient (DDPG) algorithm with parameter sharing is proposed to achieve confrontation decision-making of multi-agent. In the process of training, the information of other agents is introduced to the critic network to improve the strategy of confrontation. The parameter sharing mechanism can reduce the loss of experience storage. In the DDPG algorithm, we use four neural networks to generate real-time action and Q-value function respectively and use a momentum mechanism to optimize the training process to accelerate the convergence rate for the neural network. Secondly, this paper introduces an auxiliary controller using a policy-based reinforcement learning (RL) method to achieve the assistant decision-making for the game agent. In addition, an effective reward function is used to help agents balance losses of enemies and our side. Furthermore, this paper also uses the knowledge transfer method to extend the learning model to more complex scenes and improve the generalization of the proposed confrontation model. Two confrontation decision-making experiments are designed to verify the effectiveness of the proposed method. In a small-scale task scenario, the trained agent can successfully learn to fight with the competitors and achieve a good winning rate. For large-scale confrontation scenarios, the knowledge transfer method can gradually improve the decision-making level of the learning agent.

Improvement on Supporting Machine Learning Algorithm for Solving Problem in Immediate Decision Making

2012 ◽  
Vol 566 ◽  
pp. 572-579
Author(s):  
Abdolkarim Niazi ◽  
Norizah Redzuan ◽  
Raja Ishak Raja Hamzah ◽  
Sara Esfandiari
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Multi Agent Systems ◽  
Combined Model ◽  
Q Learning ◽  
Agent Systems ◽  
Multi Agent ◽  
Case Base ◽  
Case Base Reasoning ◽  
Robotic Tool

In this paper, a new algorithm based on case base reasoning and reinforcement learning (RL) is proposed to increase the convergence rate of the reinforcement learning algorithms. RL algorithms are very useful for solving wide variety decision problems when their models are not available and they must make decision correctly in every state of system, such as multi agent systems, artificial control systems, robotic, tool condition monitoring and etc. In the propose method, we investigate how making improved action selection in reinforcement learning (RL) algorithm. In the proposed method, the new combined model using case base reasoning systems and a new optimized function is proposed to select the action, which led to an increase in algorithms based on Q-learning. The algorithm mentioned was used for solving the problem of cooperative Markov’s games as one of the models of Markov based multi-agent systems. The results of experiments Indicated that the proposed algorithms perform better than the existing algorithms in terms of speed and accuracy of reaching the optimal policy.

Energy Optimization of Solar Micro-Grid Using Multi Agent Reinforcement Learning

2015 ◽  
Vol 787 ◽  
pp. 843-847
Author(s):  
Leo Raju ◽  
R.S. Milton ◽  
S. Sakthiyanandan
Keyword(s):  
Reinforcement Learning ◽  
Energy Savings ◽  
Learning Method ◽  
Solar Pv ◽  
Q Learning ◽  
Pv Systems ◽  
Model Free ◽  
Individual Unit ◽  
Multi Agent ◽  
Micro Grid

In this paper, two solar Photovoltaic (PV) systems are considered; one in the department with capacity of 100 kW and the other in the hostel with capacity of 200 kW. Each one has battery and load. The capital cost and energy savings by conventional methods are compared and it is proved that the energy dependency from grid is reduced in solar micro-grid element, operating in distributed environment. In the smart grid frame work, the grid energy consumption is further reduced by optimal scheduling of the battery, using Reinforcement Learning. Individual unit optimization is done by a model free reinforcement learning method, called Q-Learning and it is compared with distributed operations of solar micro-grid using a Multi Agent Reinforcement Learning method, called Joint Q-Learning. The energy planning is designed according to the prediction of solar PV energy production and observed load pattern of department and the hostel. A simulation model was developed using Python programming.

A Conflict-Free Routing Method for Automated Guided Vehicles Using Reinforcement Learning

2020 ◽  
Author(s):  
Taichi Chujo ◽  
Kosei Nishida ◽  
Tatsushi Nishi
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Learning Algorithm ◽  
Automated Guided Vehicles ◽  
Transport Systems ◽  
Deadlock Avoidance ◽  
Q Learning ◽  
Traffic Conditions ◽  
Bidirectional Transport ◽  
Routing Method

Abstract In a modern large-scale fabrication, hundreds of vehicles are used for transportation. Since traffic conditions are changing rapidly, the routing of automated guided vehicles (AGV) needs to be changed according to the change in traffic conditions. We propose a conflict-free routing method for AGVs using reinforcement learning in dynamic transportation. An advantage of the proposed method is that a change in the state can be obtained as an evaluation function. Therefore, the action can be selected according to the states. A deadlock avoidance method in bidirectional transport systems is developed using reinforcement learning. The effectiveness of the proposed method is demonstrated by comparing the performance with the conventional Q learning algorithm from computational results.

Sign in / Sign up

Export Citation Format

Share Document