A Novel Deep Reinforcement Sensor Placement Method for Waterfront Tracking

2021 ◽  
Author(s):  
Klemens Katterbauer ◽  
Abdallah Al Shehri ◽  
Alberto Marsala
Keyword(s):  
Reinforcement Learning ◽  
Water Saturation ◽  
Learning Algorithm ◽  
Sensor Placement ◽  
Base Station ◽  
Carbonate Reservoir ◽  
Carbonate Reservoirs ◽  
Sensor Data ◽  
Sweep Efficiency ◽  
Reservoir Monitoring

Abstract Waterfront movement in fractured carbonate reservoirs occurs in micro-fractures, corridors and interconnected fracture channels (above 5 mm in size) that penetrate the carbonate reservoir structure. Determining the fracture channels and the waterfront movements within the flow corridors is critical to optimize sweep efficiency and increase hydrocarbon recovery. In this work, we present a new deep reinforcement learning algorithm for the optimization of sensor placement for waterfront movement detection in carbonate fracture channels. The framework deploys deep reinforcement learning approach for optimizing the location of sensors within the fracture channels to enhance waterfront tracking. The approach first deploys the deep learning algorithm for determining the water saturation levels within the fractures based on the sensor data.. Then, it updates the sensor locations in order to optimize the reservoir coverage. We test the deep reinforcement learning framework on a synthetic fracture carbonate reservoir box model exhibiting a complex fracture system. Fracture Robots (FracBots, around 5 mm in size) technology will be used to sense key reservoir parameters (e.g., temperature, pressure, pH and other chemical parameters). The technology is comprised of a wireless micro-sensor network for mapping and monitoring fractures in conventional and unconventional reservoirs [1]. It establish a wireless network connectivity via magnetic induction (MI)-based communication since it exhibits highly reliable and constant channel conditions with sufficient communication range in the oil reservoir environment. The system architecture of the FracBots network has two layers: FracBot nodes layer and a base station layer. A number of subsurface FracBot sensors are injected in the formation fractures that record data affected by changes in water saturation. The sensor placement can be adapted in the reservoir formation to improve sensor data quality, as well as better track the penetrating waterfronts. They will move with the injected fluids and distribute themselves in the fractures where they start sensing the surrounding environment's conditions and communicate data, including their location coordinates, among each other to finally send the information in multi-hop fashion to the base station installed inside the wellbore. The base station layer consists of a large antenna connected to an aboveground gateway. The data collected from the FracBots network will be transmitted to the control room via aboveground gateway for further processing. The results exhibited resilient performance in updating the sensor placement to capture the penetrating waterfronts in the formation. The framework performs well particularly when the distance between the sensors is sufficient to avoid measurement interference. The framework demonstrates the criticality of adequate sensor placement in the reservoir formation for accurate waterfront tracking. Also, it shows that itis a viable solution to optimize sensor placement for reservoir monitoring. This novel framework presents a vital component in the data analysis and interpretation of subsurface reservoir monitoring system for carbonate reservoirs. The results outline the opportunity to deploy advanced artificial intelligence algorithms, such as deep reinforcement methods, to optimally place downhole sensors to achieve best measurement success, and track the waterfronts as well as determine sweep efficiency.

scholarly journals Optimising Performance for NB-IoT UE Devices through Data Driven Models

2021 ◽  
Vol 10 (1) ◽  
pp. 21
Author(s):  
Omar Nassef ◽  
Toktam Mahmoodi ◽  
Foivos Michelinakis ◽  
Kashif Mahmood ◽  
Ahmed Elmokashfi
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Gradient Descent ◽  
Deep Neural Network ◽  
Narrow Band ◽  
Learning Algorithm ◽  
Base Station ◽  
User Equipment ◽  
Data Driven ◽  
Superior Performance

This paper presents a data driven framework for performance optimisation of Narrow-Band IoT user equipment. The proposed framework is an edge micro-service that suggests one-time configurations to user equipment communicating with a base station. Suggested configurations are delivered from a Configuration Advocate, to improve energy consumption, delay, throughput or a combination of those metrics, depending on the user-end device and the application. Reinforcement learning utilising gradient descent and genetic algorithm is adopted synchronously with machine and deep learning algorithms to predict the environmental states and suggest an optimal configuration. The results highlight the adaptability of the Deep Neural Network in the prediction of intermediary environmental states, additionally the results present superior performance of the genetic reinforcement learning algorithm regarding its performance optimisation.

Sparse Water Fracture Channel Detection from Subsurface Sensors Via a Smart Orthogonal Matching Pursuit

2021 ◽  
Author(s):  
Klemens Katterbauer ◽  
Abdallah Al Shehri ◽  
Alberto Marsala
Keyword(s):  
Deep Learning ◽  
Monitoring System ◽  
Water Saturation ◽  
Matching Pursuit ◽  
Base Station ◽  
Carbonate Reservoir ◽  
Subsurface Reservoir ◽  
Front Movement ◽  
Water Front

Abstract Water front movement in fractured carbonate reservoirs occurs in micro-fractures, corridors and interconnected fracture channels (above 5 mm in size) that penetrate the carbonate reservoir structure. Determining the fracture channels and the water front movements within the flow corridors is critical to optimize sweep efficiency and increase hydrocarbon recovery. In this work, we present a new smart orthogonal matching pursuit (OMP) algorithm for water front movement detection in carbonate fracture channels. The method utilizes a combined artificial intelligence) AI-OMP approach to first analyze and extract the potential fracture channels and then subsequently deploys a deep learning approach for estimating the water saturation patterns in the fracture channels. The OMP utilizes the sparse fracture to sensor correlation to determine the fracture channels impacting each individual sensor. The deep learning method then utilizes the fracture channel estimates to assess the water front movements. We tested the AI-OMP framework on a synthetic fracture carbonate reservoir box model exhibiting a complex fracture system. Fracture Robots (FracBots, about 5mm in size) technology will be used to sense key reservoir parameters (e.g., temperature, pressure, pH and other chemical parameters) and represent an important step towards enhancing reservoir surveillance (Al Shehri, et al. 2021). The technology is comprised of a wireless micro-sensor network for mapping and monitoring fracture channels in conventional and unconventional reservoirs. The system establishes wireless network connectivity via magnetic induction (MI)-based communication, since it exhibits highly reliable and constant channel conditions with sufficiently communication range inside an oil reservoir environment. The system architecture of the FracBots network has two layers: FracBot nodes layer and a base station layer. A number of subsurface FracBot sensors are injected in the formation fracture channels to record data affected by changes in water saturation. The sensor placement can be adapted in the reservoir formation in order to improve sensor measurement data quality, as well as better track the penetrating water fronts. They will move with the injected fluids and distribute themselves in the fracture channels where they start sensing the surrounding environment’s conditions; they communicate the data, including their location coordinates, among each other to finally transmit the information in multi-hop fashion to the base station installed inside the wellbore. The base station layer consists of a large antenna connected to an aboveground gateway. The data collected from the FracBots network are transmitted to the control room via aboveground gateway for further processing. The results exhibited strong estimation performance in both accurately determining the fracture channels and the saturation pattern in the subsurface reservoir. The results indicate that the framework performs well; especially for fracture channels that are rather shallow (about 20 m from the wellbore) with significant changes in the saturation levels. This makes the in-situ reservoir sensing a viable permanent reservoir monitoring system for the tracking of fluid fronts, and determination of fracture channels. The novel framework presents a vital component in the data analysis and interpretation of subsurface reservoir monitoring system of fracture channels flow in carbonate reservoirs. The results outline the capability of in-situ reservoir sensors to deliver accurate tracking water-fronts and fracture channels in order to optimize recovery.

scholarly journals Multi-Task Deep Reinforcement Learning for Continuous Action Control

2017 ◽  
Author(s):  
Zhaoyang Yang ◽  
Kathryn Merrick ◽  
Hussein Abbass ◽  
Lianwen Jin
Keyword(s):  
Reinforcement Learning ◽  
Network Architecture ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Action Control ◽  
Learning Performance ◽  
Sensor Data ◽  
Continuous Action ◽  
Single Task ◽  
Multiple Tasks

In this paper, we propose a deep reinforcement learning algorithm to learn multiple tasks concurrently. A new network architecture is proposed in the algorithm which reduces the number of parameters needed by more than 75% per task compared to typical single-task deep reinforcement learning algorithms. The proposed algorithm and network fuse images with sensor data and were tested with up to 12 movement-based control tasks on a simulated Pioneer 3AT robot equipped with a camera and range sensors. Results show that the proposed algorithm and network can learn skills that are as good as the skills learned by a comparable single-task learning algorithm. Results also show that learning performance is consistent even when the number of tasks and the number of constraints on the tasks increased.

Petrofacies classification using machine learning algorithms

Geophysics ◽  
2020 ◽  
Vol 85 (4) ◽  
pp. WA101-WA113 ◽  
Author(s):  
Adrielle A. Silva ◽  
Mônica W. Tavares ◽  
Abel Carrasquilla ◽  
Roseane Misságia ◽  
Marco Ceia
Keyword(s):  
Machine Learning ◽  
Oil And Gas ◽  
Water Saturation ◽  
Carbonate Reservoir ◽  
Machine Learning Algorithms ◽  
Carbonate Reservoirs ◽  
Training Data ◽  
Southeastern Brazil ◽  
Gradient Boosting ◽  
Data Set

Carbonate reservoirs represent a large portion of the world’s oil and gas reserves, exhibiting specific characteristics that pose complex challenges to the reservoirs’ characterization, production, and management. Therefore, the evaluation of the relationships between the key parameters, such as porosity, permeability, water saturation, and pore size distribution, is a complex task considering only well-log data, due to the geologic heterogeneity. Hence, the petrophysical parameters are the key to assess the original composition and postsedimentological aspects of the carbonate reservoirs. The concept of reservoir petrofacies was proposed as a tool for the characterization and prediction of the reservoir quality as it combines primary textural analysis with laboratory measurements of porosity, permeability, capillary pressure, photomicrograph descriptions, and other techniques, which contributes to understanding the postdiagenetic events. We have adopted a workflow to petrofacies classification of a carbonate reservoir from the Campos Basin in southeastern Brazil, using the following machine learning methods: decision tree, random forest, gradient boosting, K-nearest neighbors, and naïve Bayes. The data set comprised 1477 wireline data from two wells (A3 and A10) that had petrofacies classes already assigned based on core descriptions. It was divided into two subsets, one for training and one for testing the capability of the trained models to assign petrofacies. The supervised-learning models have used labeled training data to learn the relationships between the input measurements and the petrofacies to be assigned. Additionally, we have developed a comparison of the models’ performance using the testing set according to accuracy, precision, recall, and F1-score evaluation metrics. Our approach has proved to be a valuable ally in petrofacies classification, especially for analyzing a well-logging database with no prior petrophysical information.

Field Data and Laboratory Study to Investigate Water and Gas Injectivity and Cyclic Hysteresis in Miscible WAG Injection in Carbonate Reservoir

2021 ◽  
Author(s):  
Shehadeh Masalmeh ◽  
Aaesha Al-Keebali ◽  
Arit Igogo
Keyword(s):  
Field Data ◽  
Water Saturation ◽  
Carbonate Reservoir ◽  
Carbonate Reservoirs ◽  
Experimental Program ◽  
Water Mobility ◽  
Three Phase ◽  
Reservoir Conditions ◽  
Cycle Dependent ◽  
Better Than

Abstract The objective of this paper is to investigate the water and gas injectivity in water alternating gas (WAG) projects using laboratory and field scale data. It has been reported in the literature that both gas and water mobility has been significantly reduced in three-phase flow compared to two-phase flow. This behaviour has been attributed to a cycle dependent hysteresis effect which reduced both gas and water mobility in the different injection cycles. To address the gas and water injectivity and the cycle dependent hysteresis concept, the results of a detailed experimental program in addition to field injectivity data will be presented. The experimental program included three-phase experiments performed under reservoir conditions using live crude oil and carbonate reservoir core material. The core wettability was restored by ageing the core in crude oil for several weeks under reservoir conditions and CO2 was used as miscible injectant. The field injectivity data is obtained from two CO2 WAG pilots in a carbonate reservoir. The main conclusions of the study are: 1- Gas injectivity in the presence of mobile water is much lower than that in the absence of water, 2- Water injectivity in experiments starting with water cycle is better than that in experiments starting with gas cycle when compared at the same water saturation, 3- Cyclic hysteresis in gas relative permeability was observed when comparing the first and second gas cycle, however, no further hysteresis was observed in the subsequent gas injection cycles, 4- Cyclic hysteresis in water relative permeability was not observed, the injectivity was either the same or higher in the subsequent cycles. 5- The gas injectivity at similar gas saturation for experiments starting with gas is better than that for experiments starting with water, 6- Gas and water injectivity field data from ongoing CO2 WAG projects in carbonate reservoirs showed no cyclic hysteresis, the injectivity either the same or improved in the subsequent cycles, 7- The CO2 injectivity was lower than expected, in the same order as water injectivity, which could be due to injecting CO2 in high water saturation zone and 8) The low CO2 injectivity could have a positive impact on sweep efficiency and potential improvement of oil recovery. This paper presents the results of a well-designed experimental program and field data from two CO2 WAG pilots to systematically investigate water and gas injectivity in miscible WAG projects in carbonate reservoirs. The paper provides a rich and rarely available set of experimental and field data that can help improve and optimize gas and WAG injection projects in carbonate reservoirs. Detailed analysis of the field gas and water injectivity data will be presented and discussed in-light of the laboratory experimental data.

scholarly journals Optimizing the Sensor Placement for Foot Plantar Center of Pressure without Prior Knowledge Using Deep Reinforcement Learning

Sensors ◽  
2020 ◽  
Vol 20 (19) ◽  
pp. 5588
Author(s):  
Cheng-Wu Lin ◽  
Shanq-Jang Ruan ◽  
Wei-Chun Hsu ◽  
Ya-Wen Tu ◽  
Shao-Li Han
Keyword(s):  
Reinforcement Learning ◽  
Prior Knowledge ◽  
Center Of Pressure ◽  
Learning Algorithm ◽  
Sensor Placement ◽  
Ground Truth ◽  
Reward System ◽  
The Self ◽  
Anatomical Area ◽  
Reinforcement Learning Algorithm

We study the foot plantar sensor placement by a deep reinforcement learning algorithm without using any prior knowledge of the foot anatomical area. To apply a reinforcement learning algorithm, we propose a sensor placement environment and reward system that aims to optimize fitting the center of pressure (COP) trajectory during the self-selected speed running task. In this environment, the agent considers placing eight sensors within a 7 × 20 grid coordinate system, and then the final pattern becomes the result of sensor placement. Our results show that this method (1) can generate a sensor placement, which has a low mean square error in fitting ground truth COP trajectory, and (2) robustly discovers the optimal sensor placement in a large number of combinations, which is more than 116 quadrillion. This method is also feasible for solving different tasks, regardless of the self-selected speed running task.

scholarly journals Cementation exponent estimate in carbonate reservoirs: A new method

2020 ◽  
Vol 10 (2) ◽  
pp. 66-72
Author(s):  
Adel Shirazy ◽  
Keyvan Khayer ◽  
Aref Shirazi ◽  
Abdolhamid Ansari ◽  
Ardeshir Hezarkhani
Keyword(s):  
Water Saturation ◽  
Carbonate Reservoir ◽  
Carbonate Reservoirs ◽  
New Method ◽  
Matrix Density ◽  
Close Relationship ◽  
The Matrix ◽  
Log Interpretation ◽  
Fundamental Equations ◽  
Cementation Factor

There are two approaches for measuring hydrocarbon saturation: well log interpretation and usually developed formulas. Archie’s equation is one of the most fundamental equations used for water saturation calculation. Archie’s equation includes three factors: cementation factor, tortuosity and saturation exponent. Archie determines these factors based on lab results in sandstone and provides fixed value for them. Carbonate reservoirs have a variety of textures, shapes and distribution of pores; therefore, the mentioned factors, especially cementation are not considered constant. In this study, the relationship between cementation factor and density log was examined because cementation factor is defined as a parameter that has a close relationship with density. By calculating the matrix density and accordance factor between the matrix density and cementation factor from core’s analysis, a log will be generated that can estimate the variation of cementation factor around the borehole. This method is useable for calculating the cementation factor in carbonate rocks.   Keywords: Cementation factor, carbonate reservoir, density, new method, exponents.

scholarly journals Computational Design of Modular Robots Based on Genetic Algorithm and Reinforcement Learning

Symmetry ◽  
2021 ◽  
Vol 13 (3) ◽  
pp. 471
Author(s):  
Jai Hoon Park ◽  
Kang Hoon Lee
Keyword(s):  
Genetic Algorithm ◽  
Reinforcement Learning ◽  
Design Space ◽  
Learning Algorithm ◽  
Computational Design ◽  
Computational Method ◽  
Learning Ability ◽  
Modular Robots ◽  
Control Mechanisms ◽  
Candidate Structure

Designing novel robots that can cope with a specific task is a challenging problem because of the enormous design space that involves both morphological structures and control mechanisms. To this end, we present a computational method for automating the design of modular robots. Our method employs a genetic algorithm to evolve robotic structures as an outer optimization, and it applies a reinforcement learning algorithm to each candidate structure to train its behavior and evaluate its potential learning ability as an inner optimization. The size of the design space is reduced significantly by evolving only the robotic structure and by performing behavioral optimization using a separate training algorithm compared to that when both the structure and behavior are evolved simultaneously. Mutual dependence between evolution and learning is achieved by regarding the mean cumulative rewards of a candidate structure in the reinforcement learning as its fitness in the genetic algorithm. Therefore, our method searches for prospective robotic structures that can potentially lead to near-optimal behaviors if trained sufficiently. We demonstrate the usefulness of our method through several effective design results that were automatically generated in the process of experimenting with actual modular robotics kit.

Sign in / Sign up

Export Citation Format

Share Document