A Scalable Privacy-Preserving Multi-agent Deep Reinforcement Learning Approach for Large-Scale Peer-to-Peer Transactive Energy Trading

2021 ◽  
pp. 1-1
Author(s):  
Yujian Ye ◽  
Yi Tang ◽  
Huiyu Wang ◽  
Xiao-Ping Zhang ◽  
Goran Strbac
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Privacy Preserving ◽  
Peer To Peer ◽  
Learning Approach ◽  
Energy Trading ◽  
Multi Agent ◽  
Transactive Energy

scholarly journals Multi-Agent Reinforcement Learning for Automated Peer-to-Peer Energy Trading in Double-Side Auction Market

2021 ◽  
Author(s):  
Dawei Qiu ◽  
Jianhong Wang ◽  
Junkai Wang ◽  
Goran Strbac
Keyword(s):  
Reinforcement Learning ◽  
Electricity Market ◽  
Smart Grids ◽  
Public Information ◽  
Economic Benefits ◽  
Peer To Peer ◽  
The Other ◽  
Energy Trading ◽  
Utility Company ◽  
Multi Agent

With increasing prosumers employed with distributed energy resources (DER), advanced energy management has become increasingly important. To this end, integrating demand-side DER into electricity market is a trend for future smart grids. The double-side auction (DA) market is viewed as a promising peer-to-peer (P2P) energy trading mechanism that enables interactions among prosumers in a distributed manner. To achieve the maximum profit in a dynamic electricity market, prosumers act as price makers to simultaneously optimize their operations and trading strategies. However, the traditional DA market is difficult to be explicitly modelled due to its complex clearing algorithm and the stochastic bidding behaviors of the participants. For this reason, in this paper we model this task as a multi-agent reinforcement learning (MARL) problem and propose an algorithm called DA-MADDPG that is modified based on MADDPG by abstracting the other agents’ observations and actions through the DA market public information for each agent’s critic. The experiments show that 1) prosumers obtain more economic benefits in P2P energy trading w.r.t. the conventional electricity market independently trading with the utility company; and 2) DA-MADDPG performs better than the traditional Zero Intelligence (ZI) strategy and the other MARL algorithms, e.g., IQL, IDDPG, IPPO and MADDPG.

Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach

2021 ◽  
Vol 292 ◽  
pp. 116940
Author(s):  
Dawei Qiu ◽  
Yujian Ye ◽  
Dimitrios Papadaskalopoulos ◽  
Goran Strbac
Keyword(s):  
Reinforcement Learning ◽  
Peer To Peer ◽  
Learning Approach ◽  
Energy Trading

scholarly journals A Confrontation Decision-Making Method with Deep Reinforcement Learning and Knowledge Transfer for Multi-Agent System

Symmetry ◽  
2020 ◽  
Vol 12 (4) ◽  
pp. 631
Author(s):  
Chunyang Hu
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Knowledge Transfer ◽  
Large Scale ◽  
Effective Control ◽  
Small Scale ◽  
Learning Agent ◽  
Multi Agent ◽  
Transfer Method ◽  
Parameter Sharing

In this paper, deep reinforcement learning (DRL) and knowledge transfer are used to achieve the effective control of the learning agent for the confrontation in the multi-agent systems. Firstly, a multi-agent Deep Deterministic Policy Gradient (DDPG) algorithm with parameter sharing is proposed to achieve confrontation decision-making of multi-agent. In the process of training, the information of other agents is introduced to the critic network to improve the strategy of confrontation. The parameter sharing mechanism can reduce the loss of experience storage. In the DDPG algorithm, we use four neural networks to generate real-time action and Q-value function respectively and use a momentum mechanism to optimize the training process to accelerate the convergence rate for the neural network. Secondly, this paper introduces an auxiliary controller using a policy-based reinforcement learning (RL) method to achieve the assistant decision-making for the game agent. In addition, an effective reward function is used to help agents balance losses of enemies and our side. Furthermore, this paper also uses the knowledge transfer method to extend the learning model to more complex scenes and improve the generalization of the proposed confrontation model. Two confrontation decision-making experiments are designed to verify the effectiveness of the proposed method. In a small-scale task scenario, the trained agent can successfully learn to fight with the competitors and achieve a good winning rate. For large-scale confrontation scenarios, the knowledge transfer method can gradually improve the decision-making level of the learning agent.

Sign in / Sign up

Export Citation Format

Share Document