Reinforcement Learning Control Scheme for Electrical Submersible Pumps

2021 ◽  
Author(s):  
Jonathan Chong ◽  
Dwayne Kelly ◽  
Shivam Agrawal ◽  
Nam Nguyen ◽  
Mauricio Monzon
Keyword(s):  
Reinforcement Learning ◽  
Closed Loop ◽  
Large Range ◽  
Operating Conditions ◽  
Stressful Events ◽  
Course Of Action ◽  
Control Scheme ◽  
Electrical Submersible Pumps ◽  
Working Together ◽  
Events Detection

Abstract Electrical submersible pumps (ESPs) are closely monitored in surveillance operations because they operate in challenging environments and are subject to stressful events that, if left without intervention, may lead to unplanned shutdowns, decreased run life, or even failures. These events can occur unannounced with different magnitudes of severity due to the large range of operating conditions. Thus, a universally prescriptive response is challenging because each well may require a tailored and dynamic course of action over time. This paper proposes leveraging a powerful multidimensional state engine known as automated events detection (AED), working together with an artificial intelligence agent, to respond to these stressful events and subsequently improve actions using a reinforcement learning (RL) scheme. Motivations of this approach are to move toward more autonomous, self-protecting systems with closed-loop actions and to achieve this at scale across many wells.

scholarly journals Robust Quadrotor Control through Reinforcement Learning with Disturbance Compensation

2021 ◽  
Vol 11 (7) ◽  
pp. 3257
Author(s):  
Chen-Huan Pi ◽  
Wei-Yuan Ye ◽  
Stone Cheng
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Control Strategy ◽  
External Disturbance ◽  
Control Agent ◽  
Network Control ◽  
Outdoor Environment ◽  
Disturbance Compensation ◽  
Tracking Accuracy ◽  
Control Scheme

In this paper, a novel control strategy is presented for reinforcement learning with disturbance compensation to solve the problem of quadrotor positioning under external disturbance. The proposed control scheme applies a trained neural-network-based reinforcement learning agent to control the quadrotor, and its output is directly mapped to four actuators in an end-to-end manner. The proposed control scheme constructs a disturbance observer to estimate the external forces exerted on the three axes of the quadrotor, such as wind gusts in an outdoor environment. By introducing an interference compensator into the neural network control agent, the tracking accuracy and robustness were significantly increased in indoor and outdoor experiments. The experimental results indicate that the proposed control strategy is highly robust to external disturbances. In the experiments, compensation improved control accuracy and reduced positioning error by 75%. To the best of our knowledge, this study is the first to achieve quadrotor positioning control through low-level reinforcement learning by using a global positioning system in an outdoor environment.

scholarly journals Reinforcement-Learning-Based Asynchronous Formation Control Scheme for Multiple Unmanned Surface Vehicles

2021 ◽  
Vol 11 (2) ◽  
pp. 546
Author(s):  
Jiajia Xie ◽  
Rui Zhou ◽  
Yuan Liu ◽  
Jun Luo ◽  
Shaorong Xie ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Formation Control ◽  
Rapid Development ◽  
Gradient Algorithm ◽  
Robot System ◽  
Physical Relationship ◽  
Unmanned Surface Vehicles ◽  
Main Challenge ◽  
Control Scheme ◽  
Multi Robot

The high performance and efficiency of multiple unmanned surface vehicles (multi-USV) promote the further civilian and military applications of coordinated USV. As the basis of multiple USVs’ cooperative work, considerable attention has been spent on developing the decentralized formation control of the USV swarm. Formation control of multiple USV belongs to the geometric problems of a multi-robot system. The main challenge is the way to generate and maintain the formation of a multi-robot system. The rapid development of reinforcement learning provides us with a new solution to deal with these problems. In this paper, we introduce a decentralized structure of the multi-USV system and employ reinforcement learning to deal with the formation control of a multi-USV system in a leader–follower topology. Therefore, we propose an asynchronous decentralized formation control scheme based on reinforcement learning for multiple USVs. First, a simplified USV model is established. Simultaneously, the formation shape model is built to provide formation parameters and to describe the physical relationship between USVs. Second, the advantage deep deterministic policy gradient algorithm (ADDPG) is proposed. Third, formation generation policies and formation maintenance policies based on the ADDPG are proposed to form and maintain the given geometry structure of the team of USVs during movement. Moreover, three new reward functions are designed and utilized to promote policy learning. Finally, various experiments are conducted to validate the performance of the proposed formation control scheme. Simulation results and contrast experiments demonstrate the efficiency and stability of the formation control scheme.

scholarly journals A reinforcement learning method with closed-loop stability guarantee for systems with unknown parameters

2020 ◽  
Vol 53 (2) ◽  
pp. 8157-8162
Author(s):  
Thomas Göhrt ◽  
Fritjof Griesing-Scheiwe ◽  
Pavel Osinenko ◽  
Stefan Streif
Keyword(s):  
Reinforcement Learning ◽  
Closed Loop ◽  
Learning Method ◽  
Unknown Parameters ◽  
Loop Stability

scholarly journals 5G Poor and Rich Novel Control Scheme Based Load Frequency Regulation of a Two-Area System with 100% Renewables in Africa

2020 ◽  
Vol 5 (1) ◽  
pp. 2
Author(s):  
Hady H. Fayek
Keyword(s):  
Power System ◽  
Fractional Order ◽  
Packet Loss ◽  
Pid Controller ◽  
Frequency Control ◽  
Operating Conditions ◽  
Load Frequency Control ◽  
Load Frequency ◽  
Control Scheme ◽  
Non Linear

Remote farms in Africa are cultivated lands planned for 100% sustainable energy and organic agriculture in the future. This paper presents the load frequency control of a two-area power system feeding those farms. The power system is supplied by renewable technologies and storage facilities only which are photovoltaics, biogas, biodiesel, solar thermal, battery storage and flywheel storage systems. Each of those facilities has 150-kW capacity. This paper presents a model for each renewable energy technology and energy storage facility. The frequency is controlled by using a novel non-linear fractional order proportional integral derivative control scheme (NFOPID). The novel scheme is compared to a non-linear PID controller (NPID), fractional order PID controller (FOPID), and conventional PID. The effect of the different degradation factors related to the communication infrastructure, such as the time delay and packet loss, are modeled and simulated to assess the controlled system performance. A new cost function is presented in this research. The four controllers are tuned by novel poor and rich optimization (PRO) algorithm at different operating conditions. PRO controller design is compared to other state of the art techniques in this paper. The results show that the PRO design for a novel NFOPID controller has a promising future in load frequency control considering communication delays and packet loss. The simulation and optimization are applied on MATLAB/SIMULINK 2017a environment.

Adaptive attitude-tracking control of spacecraft considering on-orbit refuelling

2020 ◽  
pp. 014233122097313
Author(s):  
Yiqi Xu
Keyword(s):  
Tracking Control ◽  
Closed Loop ◽  
Closed Loop System ◽  
Tracking Errors ◽  
Attitude Tracking ◽  
Control Scheme ◽  
Inertia Model ◽  
And Performance ◽  
Attitude Tracking Control ◽  
The Stability

This paper studies the attitude-tracking control problem of spacecraft considering on-orbit refuelling. A time-varying inertia model is developed for spacecraft on-orbit refuelling, which actually includes two processes: fuel in the transfer pipe and fuel in the tank. Based upon the inertia model, an adaptive attitude-tracking controller is derived to guarantee the stability of the resulted closed-loop system, as well as asymptotic convergence of the attitude-tracking errors, despite performing refuelling operations. Finally, numerical simulations illustrate the effectiveness and performance of the proposed control scheme.

Logic-Based Switching Adaptive Control of DC-DC Buck Converter

2012 ◽  
Vol 229-231 ◽  
pp. 2209-2212
Author(s):  
Bao Bin Liu ◽  
Wei Zhou
Keyword(s):  
Adaptive Control ◽  
Closed Loop ◽  
Buck Converter ◽  
Uncertain Parameters ◽  
Small Signal ◽  
Closed Loop System ◽  
Unknown Parameters ◽  
Signal Model ◽  
Small Signal Model ◽  
Control Scheme

Logic-based switching adaptive control scheme is proposed for the model of DC-DC buck converter in presence of uncertain parameters and power supply disturbance. All uncertain parameters and the disturbance are estimated together through constructing Lyapunov function. And a switching mechanism is used to ensure global asymptotic stability of the closed-loop system. The results of simulation show that even if there are multiple unknown parameters in the small-signal model, the control system of DC-DC buck converter can estimate unknown parameters quickly and accurately.

An ESP Production Optimization Algorithm Applied to Unconventional Wells

2021 ◽  
Author(s):  
Fernando Bermudez ◽  
Noor Al Nahhas ◽  
Hafsa Yazdani ◽  
Michael LeTan ◽  
Mohammed Shono
Keyword(s):  
Gas Production ◽  
Oil Wells ◽  
Production Optimization ◽  
Operating Conditions ◽  
Tight Oil ◽  
Daily Production ◽  
Free Gas ◽  
Maximum Production ◽  
Electrical Submersible Pumps ◽  
High Production

Abstract The objectives and Scope is to evaluate the feasibility of a Production Maximization algorithm for ESPs on unconventional wells using projected operating conditions instead of current ones, which authors expect will be crucial in adjusting the well deliverability to optimum frequencies on the rapidly changing conditions of tight oil wells. Actual production data for an unconventional well was used, covering from the start of Natural Flow production up to 120 days afterwards. Simulating what the production would be if a VFD running on IMP Optimization algorithms had been installed, new values for well flowing pressures were calculated, daily production scenarios were evaluated, and recommended operating frequencies were plotted. Result, observations, and conclusions: A. Using the Intelligent Maximum Production (IMP) algorithm allows maximum production from tight oil wells during the initial high production stage, and the prevention of gas-locking at later stages when gas production increases. B. The adjustment of frequency at later stages for GOR wells is key to maintaining maximum production while controlling free gas at the intake when compared against controlling the surface choke. Novel/additive information: The use of Electrical Submersible Pumps for the production of unconventional wells paired with the use of a VFD and properly designed control algorithms allows faster recovery of investment by pumping maximum allowable daily rates while constraining detrimental conditions such as free gas at the intake.

Adaptive SRF-PLL Based Voltage and Frequency Control of Hybrid Standalone WECS with PMSG-BESS

2018 ◽  
Vol 19 (6) ◽  
Author(s):  
Anjana Jain ◽  
R. Saravanakumar ◽  
S. Shankar ◽  
V. Vanitha
Keyword(s):  
Reactive Power ◽  
Frequency Control ◽  
Storage System ◽  
Synchronous Generator ◽  
Energy Storage System ◽  
Voltage Regulation ◽  
Operating Conditions ◽  
Voltage Source ◽  
Voltage Source Converter ◽  
Control Scheme

Abstract The variable-speed Permanent Magnet Synchronous Generator (PMSG) based Wind Energy Conversion System (WECS) attracts the maximum power from wind, but voltage-regulation and frequency-control of the system in standalone operation is a challenging task A modern-control-based-tracking of power from wind for its best utilization is proposed in this paper for standalone PMSG based hybrid-WECS comprising Battery Energy Storage System (BESS). An Adaptive Synchronous Reference Frame Phase-Locked-Loop (SRF-PLL) based control scheme for load side bi-directional voltage source converter (VSC) is presented for the system. MATLAB/Simulink model is developed for simulation study for the proposed system and the effectiveness of the controller for bi-directional-converter is discussed under different operating conditions: like variable wind-velocity, sudden load variation, and load unbalancing. Converter control scheme enhances the power smoothening, supply-load power-matching. Also it is able to regulate the active & reactive power from PMSG-BESS hybrid system with control of fluctuations in voltage & frequency with respect to varying operating conditions. Proposed controller successfully offers reactive-power-compensation, harmonics-reduction, and power-balancing. The proposed scheme is based on proportional & integral (PI) controller. Also system is experimentally validated in the laboratory-environment and results are presented here.

Sign in / Sign up

Export Citation Format

Share Document