scholarly journals Sensors Integrated Control of PEMFC Gas Supply System Based on Large-Scale Deep Reinforcement Learning

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 349
Author(s):  
Jiawen Li ◽  
Tao Yu
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Integrated Control ◽  
Supply System ◽  
Proton Exchange ◽  
Coordination Problem ◽  
Hydrogen Flow ◽  
Model Free ◽  
Integrated Controller ◽  
Gas Supply

In the proton exchange membrane fuel cell (PEMFC) system, the flow of air and hydrogen is the main factor influencing the output characteristics of PEMFC, and there is a coordination problem between their flow controls. Thus, the integrated controller of the PEMFC gas supply system based on distributed deep reinforcement learning (DDRL) is proposed to solve this problem, it combines the original airflow controller and hydrogen flow controller into one. Besides, edge-cloud collaborative multiple tricks distributed deep deterministic policy gradient (ECMTD-DDPG) algorithm is presented. In this algorithm, an edge exploration policy is adopted, suggesting that the edge explores including DDPG, soft actor-critic (SAC), and conventional control algorithm are employed to realize distributed exploration in the environment, and a classified experience replay mechanism is introduced to improve exploration efficiency. Moreover, various tricks are combined with the cloud centralized training policy to address the overestimation of Q-value in DDPG. Ultimately, a model-free integrated controller of the PEMFC gas supply system with better global searching ability and training efficiency is obtained. The simulation verifies that the controller enables the flows of air and hydrogen to respond more rapidly to the changing load.

scholarly journals Learning Representations in Model-Free Hierarchical Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 10009-10010
Author(s):  
Jacob Rafati ◽  
David C. Noelle
Keyword(s):  
Reinforcement Learning ◽  
Intrinsic Motivation ◽  
Large Scale ◽  
Temporal Abstraction ◽  
State Spaces ◽  
Hierarchical Reinforcement Learning ◽  
Model Free ◽  
Small Set ◽  
Multiple Levels ◽  
Novel Model

Common approaches to Reinforcement Learning (RL) are seriously challenged by large-scale applications involving huge state spaces and sparse delayed reward feedback. Hierarchical Reinforcement Learning (HRL) methods attempt to address this scalability issue by learning action selection policies at multiple levels of temporal abstraction. Abstraction can be had by identifying a relatively small set of states that are likely to be useful as subgoals, in concert with the learning of corresponding skill policies to achieve those subgoals. Many approaches to subgoal discovery in HRL depend on the analysis of a model of the environment, but the need to learn such a model introduces its own problems of scale. Once subgoals are identified, skills may be learned through intrinsic motivation, introducing an internal reward signal marking subgoal attainment. We present a novel model-free method for subgoal discovery using incremental unsupervised learning over a small memory of the most recent experiences of the agent. When combined with an intrinsic motivation learning mechanism, this method learns subgoals and skills together, based on experiences in the environment. Thus, we offer an original approach to HRL that does not require the acquisition of a model of the environment, suitable for large-scale applications. We demonstrate the efficiency of our method on a variant of the rooms environment.

scholarly journals Training Champion-level Race Car Drivers Using Deep Reinforcement Learning

2021 ◽  
Author(s):  
Peter Wurman ◽  
Samuel Barrett ◽  
Kenta Kawamoto ◽  
James MacGlashan ◽  
Kaushik Subramanian ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Learning Algorithm ◽  
Integrated Control ◽  
Extreme Case ◽  
Control Policy ◽  
Reward Function ◽  
Race Car ◽  
Model Free ◽  
Race Car Drivers

Abstract Many potential applications of artificial intelligence involve making real-time decisions in physical systems. Automobile racing represents an extreme case of real-time decision making in close proximity to other highly-skilled drivers while near the limits of vehicular control. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the nonlinear control challenges of real race cars while also encapsulating the complex multi-agent interactions. We attack, and solve for the first time, the simulated racing challenge using model-free deep reinforcement learning. We introduce a novel reinforcement learning algorithm and enhance the learning process with mixed scenario training to encourage the agent to incorporate racing tactics into an integrated control policy. In addition, we construct a reward function that enables the agent to adhere to the sport's under-specified racing etiquette rules. We demonstrate the capabilities of our agent, GT Sophy, by winning two of three races against four of the world's best Gran Turismo drivers and being competitive in the overall team score. By showing that these techniques can be successfully used to train championship-level race car drivers, we open up the possibility of their use in other complex dynamical systems and real-world applications.

Generalized Predictive Control for Gas Supply System in a Proton Exchange Membrane Fuel Cell

2012 ◽  
Vol 512-515 ◽  
pp. 1380-1388 ◽  
Author(s):  
Ai Min An ◽  
Hao Chen Zhang ◽  
Xin Liu ◽  
Li Wen Chen
Keyword(s):  
Fuel Cell ◽  
Predictive Control ◽  
Pressure Difference ◽  
Proton Exchange Membrane ◽  
Oxygen Supply ◽  
Supply System ◽  
Proton Exchange ◽  
Generalized Predictive Control ◽  
Exchange Membrane ◽  
Gas Supply

Gas supply system in a proton exchange membrane fuel cell power system consists of hydrogen supply and oxygen supply. In order to improve the system output performance and maintain the pressure difference between the anode and cathode at the setting points under the variational load currents, a generalized predictive control strategy is applied to the gas supply system of a proton exchange membrane fuel cell in this paper. The fuel cell stack and gas supply system were modeled for the purpose of performance analysis and controller design. And then the designed generalized predictive controller was implemented to control the hydrogen flow rate and oxygen compressor voltage. The simulation results illustrated that the proposed controller can provide better response characteristics of the pressure difference, hydrogen and oxygen supply system as compared with PID controller.

scholarly journals Hybrid Adaptive Control for PEMFC Gas Pressure

Energies ◽  
2020 ◽  
Vol 13 (20) ◽  
pp. 5334
Author(s):  
Jing Chen ◽  
Chenghui Zhang ◽  
Ke Li ◽  
Yuedong Zhan ◽  
Bo Sun
Keyword(s):  
Proton Exchange Membrane ◽  
Control Method ◽  
Supply System ◽  
Proton Exchange ◽  
Decoupling Control ◽  
Hybrid Adaptive Control ◽  
Classical Control ◽  
Exchange Membrane ◽  
Gas Supply ◽  
Fuzzy Adaptive

This paper addresses the issues of nonlinearity and coupling between anode pressure and cathode pressure in proton exchange membrane fuel cell (PEMFC) gas supply systems. A fuzzy adaptive PI decoupling control strategy with an improved advanced genetic algorithm (AGA) is proposed. This AGA s utilized to optimize the PI parameters offline, and the fuzzy adaptive algorithm s used to adjust the PI parameters dynamically online to achieve the approximate decoupling control of the PEMFC gas supply system. According to the proposed dynamic model, the PEMFC gas supply system with the fuzzy–AGA–PI decoupling control method was simulated for comparison. The simulation results demonstrate that the proposed control system can reduce the pressure difference more efficiently with the classical control method under different load changes.

Sign in / Sign up

Export Citation Format

Share Document