A reinforcement learning approach to autonomous decision-making in smart electricity markets

The existence of high proportional distributed energy resources in energy Internet (EI) scenarios has a strong impact on the power supply-demand balance of the EI system. Decision-making optimization research that focuses on the transient voltage stability is of great significance for maintaining effective and safe operation of the EI. Within a typical EI scenario, this paper conducts a study of transient voltage stability analysis based on convolutional neural networks. Based on the judgment of transient voltage stability, a reactive power compensation decision optimization algorithm via deep reinforcement learning approach is proposed. In this sense, the following targets are achieved: the efficiency of decision-making is greatly improved, risks are identified in advance, and decisions are made in time. Simulations show the effectiveness of our proposed method.

Download Full-text

Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles

IEEE Access ◽

10.1109/access.2020.3034225 ◽

2020 ◽

Vol 8 ◽

pp. 209776-209789

Author(s):

Yang Lu ◽

Xin Xu ◽

Xinglong Zhang ◽

Lilin Qian ◽

Xing Zhou

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Motion Planning ◽

Intelligent Vehicles ◽

Hierarchical Reinforcement Learning ◽

Autonomous Decision

Download Full-text

A Reinforcement Learning Approach with Spline-Fit Object Tracking for AIBO Robot’s High Level Decision Making

Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing 2011 - Studies in Computational Intelligence ◽

10.1007/978-3-642-22288-7_14 ◽

2011 ◽

pp. 169-183

Author(s):

Subhasis Mukherjee ◽

Shamsul Huda ◽

John Yearwood

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Object Tracking ◽

Learning Approach ◽

High Level

Download Full-text

Actor–critic-based decision-making method for the artificial intelligence commander in tactical wargames

The Journal of Defense Modeling and Simulation Applications Methodology Technology ◽

10.1177/1548512920954542 ◽

2020 ◽

pp. 154851292095454

Author(s):

Junfeng Zhang ◽

Qing Xue

Keyword(s):

Neural Network ◽

Artificial Intelligence ◽

Decision Making ◽

Reinforcement Learning ◽

Convolutional Neural Network ◽

Difficult Problem ◽

Learning Method ◽

Rule Based ◽

Autonomous Decision ◽

Decision Making Problem

In a tactical wargame, the decisions of the artificial intelligence (AI) commander are critical to the final combat result. Due to the existence of fog-of-war, AI commanders are faced with unknown and invisible information on the battlefield and lack of understanding of the situation, and it is difficult to make appropriate tactical strategies. The traditional knowledge rule-based decision-making method lacks flexibility and autonomy. How to make flexible and autonomous decision-making when facing complex battlefield situations is a difficult problem. This paper aims to solve the decision-making problem of the AI commander by using the deep reinforcement learning (DRL) method. We develop a tactical wargame as the research environment, which contains built-in script AI and supports the machine–machine combat mode. On this basis, an end-to-end actor–critic framework for commander decision making based on the convolutional neural network is designed to represent the battlefield situation and the reinforcement learning method is used to try different tactical strategies. Finally, we carry out a combat experiment between a DRL-based agent and a rule-based agent in a jungle terrain scenario. The result shows that the AI commander who adopts the actor–critic method successfully learns how to get a higher score in the tactical wargame, and the DRL-based agent has a higher winning ratio than the rule-based agent.

Download Full-text

Multi-objective Longitudinal Decision-making for Autonomous Electric Vehicle: A Entropy-constrained Reinforcement Learning Approach

2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC) ◽

10.1109/itsc45102.2020.9294736 ◽

2020 ◽

Author(s):

Xiangkun He ◽

Cong Fei ◽

Yulong Liu ◽

Kaiming Yang ◽

Xuewu Ji

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Electric Vehicle ◽

Learning Approach ◽

Multi Objective

Download Full-text

UAV Swarm Confrontation Using Hierarchical Multiagent Reinforcement Learning

International Journal of Aerospace Engineering ◽

10.1155/2021/3360116 ◽

2021 ◽

Vol 2021 ◽

pp. 1-12

Author(s):

Baolai Wang ◽

Shengang Li ◽

Xianzhong Gao ◽

Tao Xie

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

State Space ◽

Optimal Solution ◽

Difficult Problem ◽

Action Space ◽

Sequential Decision ◽

Autonomous Decision ◽

Multiagent Reinforcement Learning ◽

Uav Swarm

With the development of unmanned aerial vehicle (UAV) technology, UAV swarm confrontation has attracted many researchers’ attention. However, the situation faced by the UAV swarm has substantial uncertainty and dynamic variability. The state space and action space increase exponentially with the number of UAVs, so that autonomous decision-making becomes a difficult problem in the confrontation environment. In this paper, a multiagent reinforcement learning method with macro action and human expertise is proposed for autonomous decision-making of UAVs. In the proposed approach, UAV swarm is modeled as a large multiagent system (MAS) with an individual UAV as an agent, and the sequential decision-making problem in swarm confrontation is modeled as a Markov decision process. Agents in the proposed method are trained based on the macro actions, where sparse and delayed rewards, large state space, and action space are effectively overcome. The key to the success of this method is the generation of the macro actions that allow the high-level policy to find a near-optimal solution. In this paper, we further leverage human expertise to design a set of good macro actions. Extensive empirical experiments in our constructed swarm confrontation environment show that our method performs better than the other algorithms.

Download Full-text

Modeling the Strategic Bidding of the Producers in Competitive Electricity Markets with the Watkins' Q (lambda) Reinforcement Learning

International Journal of Emerging Electric Power Systems ◽

10.2202/1553-779x.1258 ◽

2006 ◽

Vol 6 (2) ◽

Cited By ~ 1

Author(s):

Yuchao Ma ◽

Ettore F. Bompard ◽

Roberto Napoli ◽

Jiang Chuanwen

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Electricity Markets ◽

Complex Problem ◽

Test System ◽

Market Outcomes ◽

Strategic Bidding ◽

Optimal Bidding ◽

Competitive Electricity Markets ◽

Multi Agents

Competition has been introduced in the last decade into the electricity markets and is presently underway in many countries. A centralized approach for the dispatching of the generation units has been substituted by a market approach based on the biddings submitted by the supply side and, eventually, by the demand side. Each producer is a player in the market acting to maximize its utility. The decision making process of the producers and their interactions in the market are a typical complex problem that is difficult to be modeled explicitly, and can be studied with a multi agents approach. This paper proposes a model able to capture the decision making approach of the producers in submitting strategic biddings to the market and simulate the market outcomes resulting from those interactions. The model is based on the Watkins' Q (lambda) Reinforcement Learning and takes into account the network constraints that may pose considerable limitations to the electricity markets. The model can be used to define the optimal bidding strategy for each producer and, as well, to find the market equilibrium and assessing the market performances. The model proposed is applied to a standard IEEE 14-bus test system to illustrate its effectiveness.

Download Full-text