scholarly journals Autonomous Decision-Making While Drilling

Energies ◽  
2021 ◽  
Vol 14 (4) ◽  
pp. 969
Author(s):  
Eric Cayeux ◽  
Benoît Daireaux ◽  
Adrian Ambrus ◽  
Rodica Mihai ◽  
Liv Carlsen
Keyword(s):  
Decision Making ◽  
Internal State ◽  
Drilling Process ◽  
Autonomous Decision ◽  
Internal States ◽  
Markov Decision ◽  
Drilling Operations ◽  
Drilling System ◽  
Drilling Conditions ◽  
Erratic Behavior

The drilling process is complex because unexpected situations may occur at any time. Furthermore, the drilling system is extremely long and slender, therefore prone to vibrations and often being dominated by long transient periods. Adding the fact that measurements are not well distributed along the drilling system, with the majority of real-time measurements only available at the top side and having only access to very sparse data from downhole, the drilling process is poorly observed therefore making it difficult to use standard control methods. Therefore, to achieve completely autonomous drilling operations, it is necessary to utilize a method that is capable of estimating the internal state of the drilling system from parsimonious information while being able to make decisions that will keep the operation safe but effective. A solution enabling autonomous decision-making while drilling has been developed. It relies on an optimization of the time to reach the section total depth (TD). The estimated time to reach the section TD is decomposed into the effective time spent in conducting the drilling operation and the likely time lost to solve unexpected drilling events. This optimization problem is solved by using a Markov decision process method. Several example scenarios have been run in a virtual rig environment to test the validity of the concept. It is found that the system is capable to adapt itself to various drilling conditions, as for example being aggressive when the operation runs smoothly and the estimated uncertainty of the internal states is low, but also more cautious when the downhole drilling conditions deteriorate or when observations tend to indicate more erratic behavior, which is often observed prior to a drilling event.

Constrained representation learning for recurrent policy optimisation under uncertainty

2019 ◽  
pp. 105971231989164
Author(s):  
Viet-Hung Dang ◽  
Ngo Anh Vien ◽  
TaeChoong Chung
Keyword(s):  
Short Term Memory ◽  
Internal State ◽  
Representation Learning ◽  
Linear Mapping ◽  
State Representation ◽  
Future Observation ◽  
Non Linear ◽  
Internal States ◽  
Markov Decision ◽  
Partially Observable

Learning to make decisions in partially observable environments is a notorious problem that requires a complex representation of controllers. In most work, the controllers are designed as a non-linear mapping from a sequence of temporal observations to actions. These problems can, in principle, be formulated as a partially observable Markov decision process whose policy can be parameterised through the use of recurrent neural networks. In this paper, we will propose an alternative framework that (a) uses the Long-Short-Term-Memory (LSTM) Encoder-Decoder framework to learn an internal state representation for historical observations and then (b) integrates it into existing recurrent policy models to improve the task performance. The LSTM Encoder encodes a history of observations as input into a representation of internal states. The LSTM Decoder can perform two alternative decoding tasks: predicting the same input observation sequence or predicting future observation sequences. The first proposed decoder acts like an auto-encoder that will guide and constrain the learning of a useful internal state for the policy optimisation task. The second proposed decoder decodes the learnt internal state by the encoder to predict future observation sequences. This idea makes the network act like a non-linear predictive state representation model. Both these decoding parts, which introduce constraints to policy representation, will help guide both the policy optimisation problem and latent state representation learning. The integration of representation learning and policy optimisation aims to help learn more complex policies and improve the performance of policy learning tasks.

Ethics and Depression

2017 ◽  
Author(s):  
Richard Ashcroft
Keyword(s):  
Decision Making ◽  
Medical Ethics ◽  
Personal Perspective ◽  
Depression Treatment ◽  
Ethical Obligation ◽  
Autonomous Decision

This chapter discusses the ethics of depression from a personal perspective. The author, an academic who has worked in the field of medical ethics or bioethics, has suffered episodes of depression throughout his life, some lasting several months. Here he shares a few quite informal things about how these two facts about him are connected. He first considers the paradigm of autonomy and autonomous decision-making, as well as the problem with functional accounts of autonomy with regard to depression. He then reflects on an approach to ethics and depression that involves thinking about the ethics of being depressed. He also highlights two aspects of the ‘ethics of depression’: treatment and the ethical obligation to talk about it.

scholarly journals Optimal Policies for Quantum Markov Decision Processes

2021 ◽  
Author(s):  
Ming-Sheng Ying ◽  
Yuan Feng ◽  
Sheng-Gang Ying
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Quantum Systems ◽  
Sequential Decision Making ◽  
Mathematical Framework ◽  
Sequential Decision ◽  
Learning Techniques ◽  
Optimal Policies ◽  
Markov Decision ◽  
Programming Algorithms

AbstractMarkov decision process (MDP) offers a general framework for modelling sequential decision making where outcomes are random. In particular, it serves as a mathematical framework for reinforcement learning. This paper introduces an extension of MDP, namely quantum MDP (qMDP), that can serve as a mathematical model of decision making about quantum systems. We develop dynamic programming algorithms for policy evaluation and finding optimal policies for qMDPs in the case of finite-horizon. The results obtained in this paper provide some useful mathematical tools for reinforcement learning techniques applied to the quantum world.

Impaired discrimination of a subanesthetic dose of ketamine in a maternal immune activation model of schizophrenia risk

2021 ◽  
pp. 026988112110297
Author(s):  
Wayne Meighan ◽  
Thomas W Elston ◽  
David Bilkey ◽  
Ryan D Ward
Keyword(s):  
Animal Models ◽  
Drug Discrimination ◽  
Immune Activation ◽  
Internal State ◽  
Nmda Receptor Antagonist ◽  
Activation Model ◽  
Data Link ◽  
Maternal Immune Activation ◽  
Internal States ◽  
Reliable Methods

Background: Animal models of psychiatric diseases suffer from a lack of reliable methods for accurate assessment of subjective internal states in nonhumans. This gap makes translation of results from animal models to patients particularly challenging. Aims/methods: Here, we used the drug-discrimination paradigm to allow rats that model a risk factor for schizophrenia (maternal immune activation, MIA) to report on the subjective internal state produced by a subanesthetic dose of the N-methyl-D-aspartate (NMDA) receptor antagonist ketamine. Results/outcomes: The MIA rats’ discrimination of ketamine was impaired relative to controls, both in the total number of rats that acquired and the asymptotic level of discrimination accuracy. This deficit was not due to a general inability to learn to discriminate an internal drug cue or internal state generally, as MIA rats were unimpaired in the learning and acquisition of a morphine drug discrimination and were as sensitive to the internal state of satiety as controls. Furthermore, the deficit was not due to a decreased sensitivity to the physiological effects of ketamine, as MIA rats showed increased ketamine-induced locomotor activity. Finally, impaired discrimination of ketamine was only seen at subanesthetic doses which functionally correspond to psychotomimetic doses in humans. Conclusion: These data link changes in NMDA responses to the MIA model. Furthermore, they confirm the utility of the drug-discrimination paradigm for future inquiries into the subjective internal state produced in models of schizophrenia and other developmental diseases.

scholarly journals A Force Sensorless Method for CFRP/Ti Stack Interface Detection during Robotic Orbital Drilling Operations

2015 ◽  
Vol 2015 ◽  
pp. 1-11 ◽  
Author(s):  
Qiang Fang ◽  
Ze-Min Pan ◽  
Bing Han ◽  
Shao-Hua Fei ◽  
Guan-Hua Xu ◽  
...  
Keyword(s):  
Cutting Force ◽  
Thrust Force ◽  
Cutting Parameters ◽  
Force Model ◽  
Drilling Process ◽  
Reinforced Plastics ◽  
Orbital Drilling ◽  
Drilling Parameters ◽  
Drilling Operations ◽  
Machining Properties

Drilling carbon fiber reinforced plastics and titanium (CFRP/Ti) stacks is one of the most important activities in aircraft assembly. It is favorable to use different drilling parameters for each layer due to their dissimilar machining properties. However, large aircraft parts with changing profiles lead to variation of thickness along the profiles, which makes it challenging to adapt the cutting parameters for different materials being drilled. This paper proposes a force sensorless method based on cutting force observer for monitoring the thrust force and identifying the drilling material during the drilling process. The cutting force observer, which is the combination of an adaptive disturbance observer and friction force model, is used to estimate the thrust force. An in-process algorithm is developed to monitor the variation of the thrust force for detecting the stack interface between the CFRP and titanium materials. Robotic orbital drilling experiments have been conducted on CFRP/Ti stacks. The estimate error of the cutting force observer was less than 13%, and the stack interface was detected in 0.25 s (or 0.05 mm) before or after the tool transited it. The results show that the proposed method can successfully detect the CFRP/Ti stack interface for the cutting parameters adaptation.

Markov decision processes for integrating life cycle dynamics into fab-level decision making

1999 ◽  
Vol 32 (2) ◽  
pp. 4852-4857
Author(s):  
Shalabh Bhatnagar ◽  
Michael C. Fu ◽  
Steven I. Marcus ◽  
Ying He
Keyword(s):  
Decision Making ◽  
Life Cycle ◽  
Markov Decision Processes ◽  
Decision Processes ◽  
Markov Decision

Study of stick–slip suppression and robustness to parametric uncertainty in drill strings containing torsional vibration tool using sliding-mode control

2021 ◽  
pp. 146441932110452
Author(s):  
Jialin Tian ◽  
Jie Wang ◽  
Siqi Zhou ◽  
Yinglin Yang ◽  
Liming Dai
Keyword(s):  
Torsional Vibration ◽  
Complex Dynamics ◽  
Sliding Mode ◽  
Parametric Uncertainty ◽  
Drill String ◽  
Lumped Parameter Model ◽  
Drilling Process ◽  
Drill Bit ◽  
Stick Slip ◽  
Drilling Operations

Excessive stick–slip vibration of drill strings can cause inefficiency and unsafety of drilling operations. To suppress the stick–slip vibration that occurred during the downhole drilling process, a drill string torsional vibration system considering the torsional vibration tool has been proposed on the basis of the 4-degree of freedom lumped-parameter model. In the design of the model, the tool is approximated by a simple torsional pendulum that brings impact torque to the drill bit. Furthermore, two sliding mode controllers, U1 and U2, are used to suppress stick–slip vibrations while enabling the drill bit to track the desired angular velocity. Aiming at parameter uncertainty and system instability in the drilling operations, a parameter adaptation law is added to the sliding mode controller U2. Finally, the suppression effects of stick–slip and robustness of parametric uncertainty about the two proposed controllers are demonstrated and compared by simulation and field test results. This paper provides a reference for the suppression of stick–slip vibration and the further study of the complex dynamics of the drill string.

Deterministic policies based on maximum regrets in MDPs with imprecise rewards

2021 ◽  
pp. 1-16
Author(s):  
Pegah Alizadeh ◽  
Emiliano Traversi ◽  
Aomar Osmani
Keyword(s):  
Decision Making ◽  
Decision Process ◽  
Process Models ◽  
Sequential Decision Making ◽  
Sequential Decision ◽  
Exact Procedure ◽  
Markov Decision ◽  
Intuitive Idea ◽  
First Time ◽  
Maximum Regret

Markov Decision Process Models (MDPs) are a powerful tool for planning tasks and sequential decision-making issues. In this work we deal with MDPs with imprecise rewards, often used when dealing with situations where the data is uncertain. In this context, we provide algorithms for finding the policy that minimizes the maximum regret. To the best of our knowledge, all the regret-based methods proposed in the literature focus on providing an optimal stochastic policy. We introduce for the first time a method to calculate an optimal deterministic policy using optimization approaches. Deterministic policies are easily interpretable for users because for a given state they provide a unique choice. To better motivate the use of an exact procedure for finding a deterministic policy, we show some (theoretical and experimental) cases where the intuitive idea of using a deterministic policy obtained after “determinizing” the optimal stochastic policy leads to a policy far from the exact deterministic policy.

Sign in / Sign up

Export Citation Format

Share Document