scholarly journals Active Exploration by Chance-Constrained Optimization for Voltage Regulation with Reinforcement Learning

Energies ◽  
2022 ◽  
Vol 15 (2) ◽  
pp. 614
Author(s):  
Zhenhuan Ding ◽  
Xiaoge Huang ◽  
Zhao Liu
Keyword(s):  
Reinforcement Learning ◽  
Constrained Optimization ◽  
Distribution Networks ◽  
Voltage Regulation ◽  
Load Flow ◽  
Engineering Knowledge ◽  
Q Learning ◽  
Battery Energy ◽  
Active Exploration ◽  
Training Efficiency

Voltage regulation in distribution networks encounters a challenge of handling uncertainties caused by the high penetration of photovoltaics (PV). This research proposes an active exploration (AE) method based on reinforcement learning (RL) to respond to the uncertainties by regulating the voltage of a distribution network with battery energy storage systems (BESS). The proposed method integrates engineering knowledge to accelerate the training process of RL. The engineering knowledge is the chance-constrained optimization. We formulate the problem in a chance-constrained optimization with a linear load flow approximation. The optimization results are used to guide the action selection of the exploration for improving training efficiency and reducing the conserveness characteristic. The comparison of methods focuses on how BESSs are used, training efficiency, and robustness under varying uncertainties and BESS sizes. We implement the proposed algorithm, a chance-constrained optimization, and a traditional Q-learning in the IEEE 13 Node Test Feeder. Our evaluation shows that the proposed AE method has a better response to the training efficiency compared to traditional Q-learning. Meanwhile, the proposed method has advantages in BESS usage in conserveness compared to the chance-constrained optimization.

scholarly journals Deep-Reinforcement-Learning-Based Two-Timescale Voltage Control for Distribution Systems

Energies ◽  
2021 ◽  
Vol 14 (12) ◽  
pp. 3540
Author(s):  
Jing Zhang ◽  
Yiqi Li ◽  
Zhi Wu ◽  
Chunyan Rong ◽  
Tao Wang ◽  
...  
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Distribution Systems ◽  
Rapid Development ◽  
Voltage Control ◽  
Distribution Networks ◽  
Voltage Regulation ◽  
Continuous Variables ◽  
Online Application ◽  
Optimal Control Scheme

Because of the high penetration of renewable energies and the installation of new control devices, modern distribution networks are faced with voltage regulation challenges. Recently, the rapid development of artificial intelligence technology has introduced new solutions for optimal control problems with high dimensions and dynamics. In this paper, a deep reinforcement learning method is proposed to solve the two-timescale optimal voltage control problem. All control variables are assigned to different agents, and discrete variables are solved by a deep Q network (DQN) agent while the continuous variables are solved by a deep deterministic policy gradient (DDPG) agent. All agents are trained simultaneously with specially designed reward aiming at minimizing long-term average voltage deviation. Case study is executed on a modified IEEE-123 bus system, and the results demonstrate that the proposed algorithm has similar or even better performance than the model-based optimal control scheme and has high computational efficiency and competitive potential for online application.

scholarly journals A Multi-Stage Coordinated Volt-Var Optimization for Integrated and Unbalanced Radial Distribution Networks

Energies ◽  
2020 ◽  
Vol 13 (18) ◽  
pp. 4877
Author(s):  
Xiangjing Su ◽  
Jining Liu ◽  
Shuxin Tian ◽  
Ping Ling ◽  
Yang Fu ◽  
...  
Keyword(s):  
Radial Distribution ◽  
Low Voltage ◽  
Distribution Networks ◽  
Voltage Regulation ◽  
Load Flow ◽  
Medium Voltage ◽  
Modified Particle Swarm Optimization ◽  
Multi Stage ◽  
Network Operation ◽  
Radial Distribution Networks

The growing penetrations of rooftop photovoltaics (PVs) into low-voltage (LV) distribution networks are challenging voltage regulation. Developing an effective volt-var (VV) control has been the focus of many researchers with various approaches proposed so far. However, assuming a single voltage level and balanced network model, widely adopted in existing literatures, tends to cause inaccurate and even infeasible control solutions. Besides, existing distribution VV control studies are usually based on the day-ahead predictions of PV generations and loads, introducing inevitable and non-negligible errors. To address the challenges above, this paper proposes a VV co-optimization across unbalanced medium-voltage (MV) and LV networks, by traditional and emerging techniques, to ensure the network operation with the required power quality. Specifically, the operation of MV delta-connected switched capacitors and LV distributed PV inverters is coordinated, under a three-stage strategy that suits integrated and unbalanced radial distribution networks. The proposal aims to simultaneously improve voltage magnitude and balance profiles while reducing network power loss, at the least controlling cost. To effectively solve the proposed VV optimization problem, a joint solver of the modified particle swarm optimization and the improved direct load flow is employed. Finally, the proposal is evaluated by simulations on real Australian distribution networks over 24 h.

scholarly journals Linear Sensitivity Modelling Useful for Voltage Control Analysis Using Power Injections from DER

Energies ◽  
2021 ◽  
Vol 14 (16) ◽  
pp. 4749
Author(s):  
Ulises D. Lubo-Matallana ◽  
Miguel Ángel Zorrozua ◽  
José Félix Miñambres
Keyword(s):  
Distribution System ◽  
Reactive Power ◽  
Distribution Networks ◽  
Voltage Regulation ◽  
Load Flow ◽  
System Response ◽  
Control Analysis ◽  
Voltage Profile ◽  
Reactive Compensation ◽  
And Control

The injection of apparent power to self-consumption buses generates voltage variations during network operation, which, when properly monitored, could support voltage regulation and control. In this paper, we propose a linear sensitivity modelling, quite useful for studies of voltage regulation with distributed energy resources (DER). This modelling consists of two analytical improvement steps: first, a full formulation for the voltage deviations, and second, the influence of line capacitance as Q-injections at the points of common couplings (PCCs). Our proposal is based on the linear topological sensitivity of an existing network (as a function of the line parameters X, R, and Bc), branch power flow (active–reactive power (P-Q)), and power injections at the PCCs. Here, the linear sensitivity algorithm is applied to a modified IEEE 33-bus distribution system to demonstrate its extended effectiveness to voltage monitoring and control scenarios. Its application estimates and compensates in a better way the voltage deviations with regard to the operating desired voltage (|V|op) constraints, using distributed power injections at the PCCs. Numerical results always showed a curtailment of the relative error against common simplifications of the electrical modelling in steady-state, thus simulating two critical scenarios of operation production–consumption for the existing system response. The proposed algorithm was validated considering as reference the voltage profile outputs of the load flow analysis, using the Newton–Raphson method via DIgSILENT, in terms of its accuracy, silhouette shape along the feeder and with regard to the application of reactive compensation as an analytical case study for voltage improvement in active distribution networks.

Using Dueling Double Q-learning for Voltage Regulation in PV-Rich Distribution Networks

2021 ◽  
Author(s):  
Guilherme Custodio ◽  
Luis F. Ochoa ◽  
Tansu Alpcan ◽  
Fernanda C. L. Trindade ◽  
Rafael Augusto de Godoy Rosolen
Keyword(s):  
Distribution Networks ◽  
Voltage Regulation ◽  
Q Learning

scholarly journals Coordinated and optimal voltage control for voltage regulation using firefly algorithm

2019 ◽  
Vol 16 (2) ◽  
pp. 568
Author(s):  
Muhamad Najib Kamarudin ◽  
Tengku Juhana Tengku Hashim
Keyword(s):  
Distribution Systems ◽  
Firefly Algorithm ◽  
Renewable Energy Sources ◽  
Optimization Technique ◽  
Voltage Control ◽  
Distribution Networks ◽  
Voltage Regulation ◽  
Load Flow ◽  
Distributed Generations ◽  
Voltage Deviation

The operation and control of electricity in distribution networks has faced great challenges as a large number of distributed generations (DGs) are integrated. Connection of distributed generations (DGs) in the distribution system offers advantages in terms of reducing distribution and transmission costs as well as encouraging the use of renewable energy sources. The power flow in the distribution systems is no longer moving in a single direction and this resulted the system to become as active distribution networks (ADN). One of the main problems in ADN is the voltage regulation issue which is to maintain the voltage to be within its permissible limits. Several methods of voltage control methods are available and focus is given in finding the optimal voltage control using artificial intelligence techniques. This paper presents an optimal and coordinated voltage control method while minimizing losses and voltage deviation of the network. The optimal and coordinated voltage control scheme is implemented on an IEEE 13 bus distribution network for loss and voltage deviation minimization in the networks. Firefly Algorithm (FA) which is a known heuristic optimization technique for finding the optimal solution is used in this work. The results are compared with another optimization method known as Backtracking Search Algorithm (BSA) for identifying the best setting for solving the voltage regulation problem. In order to solve the multi-objective optimization issue, the MATPOWER load flow simulation is integrated in the MATLAB environment with the optimization algorithm.

Sign in / Sign up

Export Citation Format

Share Document