scholarly journals Assessing the Use of Reinforcement Learning for Integrated Voltage/Frequency Control in AC Microgrids

Energies ◽  
2020 ◽  
Vol 13 (5) ◽  
pp. 1250
Author(s):  
Abdollah Younesi ◽  
Hossein Shayeghi ◽  
Pierluigi Siano
Keyword(s):  
Reinforcement Learning ◽  
Power Systems ◽  
Control Strategy ◽  
Distribution System ◽  
Frequency Control ◽  
Dynamic Performance ◽  
Iterative Solution ◽  
Pid Controllers ◽  
Model Free ◽  
Markov Decision

The main purpose of this paper is to present a novel algorithmic reinforcement learning (RL) method for damping the voltage and frequency oscillations in a micro-grid (MG) with penetration of wind turbine generators (WTG). First, the continuous-time environment of the system is discretized to a definite number of states to form the Markov decision process (MDP). To solve the modeled discrete RL-based problem, Q-learning method, which is a model-free and simple iterative solution mechanism is used. Therefore, the presented control strategy is adaptive and it is suitable for the realistic power systems with high nonlinearities. The proposed adaptive RL controller has a supervisory nature that can improve the performance of any kind of controllers by adding an offset signal to the output control signal of them. Here, a part of Denmark distribution system is considered and the dynamic performance of the suggested control mechanism is evaluated and compared with fuzzy-proportional integral derivative (PID) and classical PID controllers. Simulations are carried out in two realistic and challenging scenarios considering system parameters changing. Results indicate that the proposed control strategy has an excellent dynamic response compared to fuzzy-PID and traditional PID controllers for damping the voltage and frequency oscillations.

scholarly journals Model-Free Reinforcement Learning for Branching Markov Decision Processes

2021 ◽  
pp. 651-673
Author(s):  
Ernst Moritz Hahn ◽  
Mateo Perez ◽  
Sven Schewe ◽  
Fabio Somenzi ◽  
Ashutosh Trivedi ◽  
...  
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Markov Decision Processes ◽  
Control Strategy ◽  
Natural Extension ◽  
Decision Processes ◽  
Optimal Control Strategy ◽  
Model Free ◽  
Learning Techniques ◽  
Markov Decision

AbstractWe study reinforcement learning for the optimal control of Branching Markov Decision Processes (BMDPs), a natural extension of (multitype) Branching Markov Chains (BMCs). The state of a (discrete-time) BMCs is a collection of entities of various types that, while spawning other entities, generate a payoff. In comparison with BMCs, where the evolution of a each entity of the same type follows the same probabilistic pattern, BMDPs allow an external controller to pick from a range of options. This permits us to study the best/worst behaviour of the system. We generalise model-free reinforcement learning techniques to compute an optimal control strategy of an unknown BMDP in the limit. We present results of an implementation that demonstrate the practicality of the approach.

scholarly journals Hybrid Wind–PV Frequency Control Strategy under Variable Weather Conditions in Isolated Power Systems

2020 ◽  
Vol 12 (18) ◽  
pp. 7750 ◽  
Author(s):  
Ana Fernández-Guillamón ◽  
Guillermo Martínez-Lucas ◽  
Ángel Molina-García ◽  
Jose-Ignacio Sarasua
Keyword(s):  
Renewable Energy ◽  
Power Systems ◽  
Control Strategy ◽  
Rotational Speed ◽  
Power Plants ◽  
Mean Squared Error ◽  
Frequency Control ◽  
Weather Conditions ◽  
Squared Error ◽  
Variable Weather

Over the last two decades, variable renewable energy technologies (i.e., variable-speed wind turbines (VSWTs) and photovoltaic (PV) power plants) have gradually replaced conventional generation units. However, these renewable generators are connected to the grid through power converters decoupled from the grid and do not provide any rotational inertia, subsequently decreasing the overall power system’s inertia. Moreover, the variable and stochastic nature of wind speed and solar irradiation may lead to large frequency deviations, especially in isolated power systems. This paper proposes a hybrid wind–PV frequency control strategy for isolated power systems with high renewable energy source integration under variable weather conditions. A new PV controller monitoring the VSWTs’ rotational speed deviation is presented in order to modify the PV-generated power accordingly and improve the rotational speed deviations of VSWTs. The power systems modeled include thermal, hydro-power, VSWT, and PV power plants, with generation mixes in line with future European scenarios. The hybrid wind–PV strategy is compared to three other frequency strategies already presented in the specific literature, and gets better results in terms of frequency deviation (reducing the mean squared error between 20% and 95%). Additionally, the rotational speed deviation of VSWTs is also reduced with the proposed approach, providing the same mean squared error as the case in which VSWTs do not participate in frequency control. However, this hybrid strategy requires up to a 30% reduction in the PV-generated energy. Extensive detailing of results and discussion can be also found in the paper.

scholarly journals An Autonomous Power-Frequency Control Strategy Based on Load Virtual Synchronous Generator

Processes ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. 433 ◽  
Author(s):  
Jiangbei Han ◽  
Zhijian Liu ◽  
Ning Liang ◽  
Qi Song ◽  
Pengcheng Li
Keyword(s):  
Power Systems ◽  
Dynamic Stability ◽  
Control Strategy ◽  
Load Capacity ◽  
Frequency Control ◽  
Synchronous Generator ◽  
Active Power ◽  
Power Oscillation ◽  
Power Frequency ◽  
Virtual Synchronous Generator

With the increasing penetration of the hybrid AC/DC microgrid in power systems, an inertia decrease of the microgrid is caused. Many scholars have put forward the concept of a virtual synchronous generator, which enables the converters of the microgrid to possess the characteristics of a synchronous generator, thus providing inertia support for the microgrid. Nevertheless, the problems of active power oscillation and unbalance would be serious when multiple virtual synchronous generators (VSGs) operate in the microgrid. To conquer these problems, a VSG-based autonomous power-frequency control strategy is proposed, which not only independently allocates the power grid capacity according to the load capacity, but also effectively suppresses the active power oscillation. In addition, by establishing a dynamic small-signal model of the microgrid, the dynamic stability of the proposed control strategy in the microgrid is verified, and further reveals the leading role of the VSG and filter in the dynamic stability of microgrids. Finally, the feasibility and effectiveness of the proposed control strategy are validated by the simulation results.

scholarly journals Comprehensive Power Flow Analyses and Novel Feedforward Coordination Control Strategy for MMC-Based UPFC

Energies ◽  
2019 ◽  
Vol 12 (5) ◽  
pp. 824 ◽  
Author(s):  
Jinlian Liu ◽  
Zheng Xu ◽  
Liang Xiao
Keyword(s):  
Power Systems ◽  
Control Strategy ◽  
Computer Aided Design ◽  
Power Flow ◽  
Feedforward Control ◽  
Control Strategies ◽  
Dynamic Performance ◽  
Unified Power Flow Controller ◽  
3 Dimensional ◽  
Steady State Operation

This paper aims to discover the general steady-state operation characteristics, as well as improving the dynamic performance, of the modular multilevel converter (MMC)-based unified power flow controller (UPFC). To achieve this, first, we established a detailed power flow model for MMC-based UPFC containing each critical part and made qualitative and graphical analyses combining 2-dimensional operation planes and 3-dimensional spatial curve surfaces comprehensively to derive general power flow principles and offer necessary references for regulating UPFC. Furthermore, to achieve better performance, we designed a feedforward control strategy for the shunt and series converters of UPFC, both comprising two feedforward control blocks with the introduction of necessary compensating branches, and analyzed the performance in complex and time domain, respectively. The proposed power flow principles and control strategies were validated by a (power systems computer aided design) PSCAD model of 220 kV double-end system; the results reveal the MMC-based UPFC can realize the power flow principles and improve the control speed, stability, and precision of the power flow regulations under various conditions.

scholarly journals Risk-Sensitive Reinforcement Learning Applied to Control under Constraints

2005 ◽  
Vol 24 ◽  
pp. 81-108 ◽  
Author(s):  
P. Geibel ◽  
F. Wysotzki
Keyword(s):  
Reinforcement Learning ◽  
Value Function ◽  
Learning Algorithm ◽  
Optimal Solution ◽  
Feed Tank ◽  
Model Free ◽  
Constrained Problem ◽  
Risk Sensitive ◽  
Markov Decision ◽  
The Value Function

In this paper, we consider Markov Decision Processes (MDPs) with error states. Error states are those states entering which is undesirable or dangerous. We define the risk with respect to a policy as the probability of entering such a state when the policy is pursued. We consider the problem of finding good policies whose risk is smaller than some user-specified threshold, and formalize it as a constrained MDP with two criteria. The first criterion corresponds to the value function originally given. We will show that the risk can be formulated as a second criterion function based on a cumulative return, whose definition is independent of the original value function. We present a model free, heuristic reinforcement learning algorithm that aims at finding good deterministic policies. It is based on weighting the original value function and the risk. The weight parameter is adapted in order to find a feasible solution for the constrained problem that has a good performance with respect to the value function. The algorithm was successfully applied to the control of a feed tank with stochastic inflows that lies upstream of a distillation column. This control task was originally formulated as an optimal control problem with chance constraints, and it was solved under certain assumptions on the model to obtain an optimal solution. The power of our learning algorithm is that it can be used even when some of these restrictive assumptions are relaxed.

Sign in / Sign up

Export Citation Format

Share Document