Designing Optimal Arbitrage Policies for Distributed Energy Systems in Building Clusters Using Reinforcement Learning

2019 ◽  
Author(s):  
Philip Odonkor ◽  
Kemper Lewis
Keyword(s):  
Renewable Energy ◽  
Reinforcement Learning ◽  
Electricity Markets ◽  
Energy Cost ◽  
Cost Minimization ◽  
Control Strategies ◽  
Cost Savings ◽  
Price Volatility ◽  
Maximization Problem ◽  
Distributed Energy

Abstract In the wake of increasing proliferation of renewable energy and distributed energy resources (DERs), grid designers and operators alike are faced with several emerging challenges in curbing allocative grid inefficiencies and maintaining operational stability. One such challenge relates to the increased price volatility within real-time electricity markets, a result of the inherent intermittency of renewable energy. With this challenge, however, comes heightened economic interest in exploiting the arbitrage potential of price volatility towards demand-side energy cost savings. To this end, this paper aims to maximize the arbitrage value of electricity through the optimal design of control strategies for DERs. Formulated as an arbitrage maximization problem using design optimization, and solved using reinforcement learning, the proposed approach is applied towards shared DERs within multi-building residential clusters. We demonstrate its feasibility across three unique building cluster demand profiles, observing notable energy cost reductions over baseline values. This highlights a capability for generalized learning across multiple building clusters and the ability to design efficient arbitrage policies towards energy cost minimization. Finally, the approach is shown to be computationally tractable, designing efficient strategies in approximately 5 hours of training over a simulation time horizon of 1 month.

Data-Driven Design of Control Strategies for Distributed Energy Systems

2019 ◽  
Vol 141 (11) ◽  
Author(s):  
Philip Odonkor ◽  
Kemper Lewis
Keyword(s):  
Energy Cost ◽  
Cost Minimization ◽  
Computational Cost ◽  
Cost Savings ◽  
Price Volatility ◽  
Energy Resources ◽  
Data Driven ◽  
Distributed Energy Resources ◽  
Maximization Problem ◽  
Distributed Energy

Abstract The flexibility afforded by distributed energy resources in terms of energy generation and storage has the potential to disrupt the way we currently access and manage electricity. But as the energy grid moves to fully embrace this technology, grid designers and operators are having to come to terms with managing its adverse effects, exhibited through electricity price volatility, caused in part by the intermittency of renewable energy. With this concern however comes interest in exploiting this price volatility using arbitrage—the buying and selling of electricity to profit from a price imbalance—for energy cost savings for consumers. To this end, this paper aims to maximize arbitrage value through the data-driven design of optimal operational strategies for distributed energy resources (DERs). Formulated as an arbitrage maximization problem using design optimization principles and solved using reinforcement learning, the proposed approach is applied toward shared DERs within multi-building residential clusters. We demonstrate its feasibility across three unique building clusters, observing notable energy cost reductions over baseline values. This highlights a capability for generalized learning across multiple building clusters and the ability to design efficient arbitrage policies for energy cost minimization. The scalability of this approach is studied using two test cases, with results demonstrating an ability to scale with relatively minimal additional computational cost, and an ability to leverage system flexibility toward cost savings.

scholarly journals Integrated control of dynamic facades and distributed energy resources for energy cost minimization in commercial buildings

Solar Energy ◽  
2015 ◽  
Vol 122 ◽  
pp. 1384-1397 ◽  
Author(s):  
Eleanor S. Lee ◽  
Christoph Gehbauer ◽  
Brian E. Coffey ◽  
Andrew McNeil ◽  
Michael Stadler ◽  
...  
Keyword(s):  
Energy Cost ◽  
Cost Minimization ◽  
Integrated Control ◽  
Commercial Buildings ◽  
Energy Resources ◽  
Distributed Energy Resources ◽  
Distributed Energy

scholarly journals Renewable energy assisted traffic aware cellular base station energy cooperation

2021 ◽  
Author(s):  
Faran Ahmed ◽  
Muhammad Naeem ◽  
Waleed Ejaz ◽  
Muhammad Iqbal ◽  
Alagan Anpalagan ◽  
...  
Keyword(s):  
Renewable Energy ◽  
Energy Cost ◽  
Cost Minimization ◽  
Renewable Energy Sources ◽  
Northern Region ◽  
Base Station ◽  
Base Stations ◽  
Solar Irradiation ◽  
Energy Sources ◽  
Energy Cooperation

With global concern for climate change, and for cutting down the energy cost, especially in off grid areas, use of renewable energy has been gaining widespread attention in many areas including cellular communication. The base station (BS) has emerged as a strong candidate for the integration of renewable energy sources (RES), particularly solar and wind. The incorporation of renewable energy opens many possibilities for energy conservation through strategies such as energy cooperation between BSs during the off-peak hours, when the energy harvested from renewable energy sources may become surplus. In this paper, we present the case for cellular BSs enabled with renewable energy sources (RES) to have an arrangement in which the BS provide surplus energy to a neighboring BS, thus minimizing the use of conventional energy. A realistic objective is developed for northern region of Pakistan, which entails modeling of solar panels and wind-turbine according to the average solar irradiation and wind speed of the region. We also model the dynamic load of the BS, which depicts temporal fluctuations with traffic variations. Based on these models we initiate an energy cooperation scheme between the BS in which an energy cost minimization framework is mathematically modeled and solved through the interior point method algorithm. Results are obtained for different times of the year for different number of base stations showing respective energy cost savings. Keywords: Base station Cooperation; alternate energy; cellular networks

Evaluation of Reinforcement Learning for Optimal Control of Building Active and Passive Thermal Storage Inventory

2006 ◽  
Vol 129 (2) ◽  
pp. 215-225 ◽  
Author(s):  
Simeng Liu ◽  
Gregor P. Henze
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Storage Capacity ◽  
Control Strategies ◽  
Cost Savings ◽  
Thermal Storage ◽  
Learning Control ◽  
Learning Performance ◽  
Optimal Control Strategy ◽  
Model Free

This paper describes an investigation of machine learning for supervisory control of active and passive thermal storage capacity in buildings. Previous studies show that the utilization of active or passive thermal storage, or both, can yield significant peak cooling load reduction and associated electrical demand and operational cost savings. In this study, a model-free learning control is investigated for the operation of electrically driven chilled water systems in heavy-mass commercial buildings. The reinforcement learning controller learns to operate the building and cooling plant based on the reinforcement feedback (monetary cost of each action, in this study) it receives for past control actions. The learning agent interacts with its environment by commanding the global zone temperature setpoints and thermal energy storage charging∕discharging rate. The controller extracts information about the environment based solely on the reinforcement signal; the controller does not contain a predictive or system model. Over time and by exploring the environment, the reinforcement learning controller establishes a statistical summary of plant operation, which is continuously updated as operation continues. The present analysis shows that learning control is a feasible methodology to find a near-optimal control strategy for exploiting the active and passive building thermal storage capacity, and also shows that the learning performance is affected by the dimensionality of the action and state space, the learning rate and several other factors. It is found that it takes a long time to learn control strategies for tasks associated with large state and action spaces.

scholarly journals Renewable energy assisted traffic aware cellular base station energy cooperation

2021 ◽  
Author(s):  
Faran Ahmed ◽  
Muhammad Naeem ◽  
Waleed Ejaz ◽  
Muhammad Iqbal ◽  
Alagan Anpalagan ◽  
...  
Keyword(s):  
Renewable Energy ◽  
Energy Cost ◽  
Cost Minimization ◽  
Renewable Energy Sources ◽  
Northern Region ◽  
Base Station ◽  
Base Stations ◽  
Solar Irradiation ◽  
Energy Sources ◽  
Energy Cooperation

With global concern for climate change, and for cutting down the energy cost, especially in off grid areas, use of renewable energy has been gaining widespread attention in many areas including cellular communication. The base station (BS) has emerged as a strong candidate for the integration of renewable energy sources (RES), particularly solar and wind. The incorporation of renewable energy opens many possibilities for energy conservation through strategies such as energy cooperation between BSs during the off-peak hours, when the energy harvested from renewable energy sources may become surplus. In this paper, we present the case for cellular BSs enabled with renewable energy sources (RES) to have an arrangement in which the BS provide surplus energy to a neighboring BS, thus minimizing the use of conventional energy. A realistic objective is developed for northern region of Pakistan, which entails modeling of solar panels and wind-turbine according to the average solar irradiation and wind speed of the region. We also model the dynamic load of the BS, which depicts temporal fluctuations with traffic variations. Based on these models we initiate an energy cooperation scheme between the BS in which an energy cost minimization framework is mathematically modeled and solved through the interior point method algorithm. Results are obtained for different times of the year for different number of base stations showing respective energy cost savings. Keywords: Base station Cooperation; alternate energy; cellular networks

scholarly journals A Stochastic Approach to Energy Cost Minimization in Smart-Grid-Enabled Data Center Network

2019 ◽  
Vol 2019 ◽  
pp. 1-11 ◽  
Author(s):  
Abolfazl Ghassemi ◽  
Pejman Goudarzi ◽  
Mohammad R. Mirsarraf ◽  
T. Aaron Gulliver
Keyword(s):  
Smart Grid ◽  
Data Center ◽  
Energy Cost ◽  
Cost Minimization ◽  
Average Power ◽  
Cost Savings ◽  
Stochastic Approach ◽  
Green Energy ◽  
Data Center Network ◽  
Power Cost

We propose a Lyapunov drift-plus-penalty- (LDPP-) based algorithm to optimize the average power cost for a data center network. In particular, we develop an algorithm to minimize the operational cost using real-time electricity pricing with the integration of green energy resources from the smart grid. The LDPP technique can achieve significant energy cost savings under quality of service (QoS) constraints. Numerical results are presented to evaluate and validate our solution. These results illustrate significant operational/energy cost reductions for a data center network over the conventional approach which optimizes the predicted values of stochastic parameters under a fixed QoS constraint.

Sign in / Sign up

Export Citation Format

Share Document