scholarly journals Thermal and Energy Management Based on Bimodal Airflow-Temperature Sensing and Reinforcement Learning

Energies ◽  
2018 ◽  
Vol 11 (10) ◽  
pp. 2575 ◽  
Author(s):  
Zhen Zhang ◽  
Cheng Ma ◽  
Rong Zhu
Keyword(s):  
Reinforcement Learning ◽  
Energy Management ◽  
High Efficiency ◽  
Intelligent System ◽  
Integrated System ◽  
Temperature Sensing ◽  
Real Time Control ◽  
Exploration Process ◽  
Energy Devices ◽  
Self Learning

Multi-physical field sensing and machine learning have drawn great attention in various fields such as sensor networks, robotics, energy devices, smart buildings, intelligent system and so on. In this paper, we present a novel efficient method for thermal and energy management based on bimodal airflow-temperature sensing and reinforcement learning, which expedites an exploration process by self-learning and adjusts action policy only through actuators interacting with the environment, being free of the controlled object model and priori experiences. In general, training of reinforcement learning requires a large amount of data iterations, which takes a long time and is not suitable for real-time control. Here, we propose an approach to speed up the learning process by indicating the action adjustment direction. We adopt tailor-designed bimodal sensors to simultaneously detect airflow and temperature field, which provides comprehensive information for reinforcement learning. The proposed thermal and energy management incorporates bimodal parametric sensing with an improved actor-critic algorithm to realize self-learning control. Experiments of thermal and energy management in a multi-module integrated system validate the effectiveness of the proposed methodology, which demonstrate high efficiency, fast response, and good robustness in various control scenarios. The proposed methodology can be widely applied to thermal and energy management of diverse integrated systems.

scholarly journals Model-Free Real-Time Autonomous Energy Management for a Residential Multi-Carrier Energy System: A Deep Reinforcement Learning Approach

2020 ◽  
Author(s):  
Yujian Ye ◽  
Dawei Qiu ◽  
Jonathan Ward ◽  
Marcin Abram
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Energy Management ◽  
Energy System ◽  
Management Problem ◽  
Energy Carriers ◽  
Energy Devices ◽  
Continuous State ◽  
Carrier Energy ◽  
Time Autonomous

The problem of real-time autonomous energy management is an application area that is receiving unprecedented attention from consumers, governments, academia, and industry. This paper showcases the first application of deep reinforcement learning (DRL) to real-time autonomous energy management for a multi-carrier energy system. The proposed approach is tailored to align with the nature of the energy management problem by posing it in multi-dimensional continuous state and action spaces, in order to coordinate power flows between different energy devices, and to adequately capture the synergistic effect of couplings between different energy carriers. This fundamental contribution is a significant step forward from earlier approaches that only sought to control the power output of a single device and neglected the demand-supply coupling of different energy carriers. Case studies on a real-world scenario demonstrate that the proposed method significantly outperforms existing DRL methods as well as model-based control approaches in achieving the lowest energy cost and yielding a representation of energy management policies that adapt to system uncertainties.

scholarly journals IOT and Data Research In Industrial Power Management

2019 ◽  
Vol 8 (6S) ◽  
pp. 57-60
Keyword(s):  
Machine Learning ◽  
Energy Management ◽  
High Efficiency ◽  
Weather Conditions ◽  
Integrated System ◽  
Machine Learning Algorithms ◽  
Sensor Data ◽  
Current Time ◽  
Solar Battery ◽  
Hourly Variation

IOT plays an important role in collecting data and machine learning for prediction in variety of applications like homecare, healthcare and energy management. In energy management there are various variables such as future power demands, generation status weather conditions and current battery status hard to expect high efficiency. Here, in this proposed idea, for higher efficiency of renewable energy, an IOT system is needed to monitor and collect these Statuses and provide energy management services. Energy will be consumed of passive operation according to hourly variation in price and battery status will be predicted by using machine learning algorithms like Logistic regression, SVM, and k-NN. We trained the system by considering five random variables in datasheet such as Current time, Current cost, predicted time, predicted cost and Solar battery status from the device. This integrated system is used for uploading power related details of Grid and Solar to IBM cloud. Depending on previous datasheet, analytics will be done by resulting which source has to be triggered to drive the load either Solar or Grid. APIs and NodeRed Tool were used for wiring sensor data and Model predicted output. In future power demands, this design will help to predict the price according to hourly variation based on the units and to trigger the source

scholarly journals Intelligent Energy Management for Plug-in Hybrid Electric Bus with Limited State Space

Processes ◽  
2019 ◽  
Vol 7 (10) ◽  
pp. 672 ◽  
Author(s):  
Hongqiang Guo ◽  
Shangye Du ◽  
Fengrui Zhao ◽  
Qinghu Cui ◽  
Weilong Ren
Keyword(s):  
State Space ◽  
Energy Management ◽  
Minimum Principle ◽  
Management Control ◽  
Real Time Control ◽  
Pontryagin Minimum Principle ◽  
Hybrid Electric Bus ◽  
Electric Bus ◽  
Hybrid Electric ◽  
Self Learning

Tabular Q-learning (QL) can be easily implemented into a controller to realize self-learning energy management control of a plug-in hybrid electric bus (PHEB). However, the “curse of dimensionality” problem is difficult to avoid, as the design space is huge. This paper proposes a QL-PMP algorithm (QL and Pontryagin minimum principle (PMP)) to address the problem. The main novelty is that the difference between the feedback SOC (state of charge) and the reference SOC is exclusively designed as state, and then a limited state space with 50 rows and 25 columns is proposed. The off-line training process shows that the limited state space is reasonable and adequate for the self-learning; the Hardware-in-Loop (HIL) simulation results show that the QL-PMP strategy can be implemented into a controller to realize real-time control, and can on average improve the fuel economy by 20.42%, compared to the charge depleting–charge sustaining (CDCS) strategy.

Deep reinforcement learning enabled self-learning control for energy efficient driving

2019 ◽  
Vol 99 ◽  
pp. 67-81 ◽  
Author(s):  
Xuewei Qi ◽  
Yadan Luo ◽  
Guoyuan Wu ◽  
Kanok Boriboonsomsin ◽  
Matthew Barth
Keyword(s):  
Reinforcement Learning ◽  
Energy Efficient ◽  
Learning Control ◽  
Self Learning

Stochastic multi-carrier energy management in the smart islands using reinforcement learning and unscented transform

2021 ◽  
Vol 130 ◽  
pp. 106988
Author(s):  
Hongbo Zou ◽  
Juan Tao ◽  
Salah K. Elsayed ◽  
Ehab E. Elattar ◽  
Abdulaziz Almalaq ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Energy Management ◽  
Unscented Transform ◽  
Carrier Energy

scholarly journals Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach

Energies ◽  
2021 ◽  
Vol 14 (9) ◽  
pp. 2700
Author(s):  
Grace Muriithi ◽  
Sunetra Chowdhury
Keyword(s):  
Renewable Energy ◽  
Reinforcement Learning ◽  
Energy Management ◽  
Renewable Energy Sources ◽  
Critical Role ◽  
Winter Season ◽  
Learning Approach ◽  
Solar Pv ◽  
Optimal Energy ◽  
Optimal Energy Management

In the near future, microgrids will become more prevalent as they play a critical role in integrating distributed renewable energy resources into the main grid. Nevertheless, renewable energy sources, such as solar and wind energy can be extremely volatile as they are weather dependent. These resources coupled with demand can lead to random variations on both the generation and load sides, thus complicating optimal energy management. In this article, a reinforcement learning approach has been proposed to deal with this non-stationary scenario, in which the energy management system (EMS) is modelled as a Markov decision process (MDP). A novel modification of the control problem has been presented that improves the use of energy stored in the battery such that the dynamic demand is not subjected to future high grid tariffs. A comprehensive reward function has also been developed which decreases infeasible action explorations thus improving the performance of the data-driven technique. A Q-learning algorithm is then proposed to minimize the operational cost of the microgrid under unknown future information. To assess the performance of the proposed EMS, a comparison study between a trading EMS model and a non-trading case is performed using a typical commercial load curve and PV profile over a 24-h horizon. Numerical simulation results indicate that the agent learns to select an optimized energy schedule that minimizes energy cost (cost of power purchased from the utility and battery wear cost) in all the studied cases. However, comparing the non-trading EMS to the trading EMS model operational costs, the latter one was found to decrease costs by 4.033% in summer season and 2.199% in winter season.

Sign in / Sign up

Export Citation Format

Share Document