Quadrotor Motion Control Using Deep Reinforcement Learning

We present a deep neural net-based controller trained by a model-free reinforcement learning (RL) algorithm to achieve hover stabilization for a quadrotor unmanned aerial vehicle (UAV). With RL, two neural nets are trained. One neural net is used as a stochastic controller which gives the distribution of control inputs. The other maps the UAV state to a scalar which estimates the reward of the controller. A proximal policy optimization (PPO) method, which is an actor-critic policy gradient approach, is used to train the neural nets. Simulation results show that the trained controller achieves a comparable level of performance to a manually-tuned PID controller, despite not depending on any model information. The paper considers different choices of reward function and their influence on controller performance.

Application of Geomechanics for Successful Drilling of Onshore Well with Multiple Challenges of Losses and Intense Gas Cut Mud in Naturally Fractured Zone.

Drilling of deviated development wells in O-field X has proven to be challenging. Drilling experience in several wells within the field has different issues of wellbore instability, most recent is when traversed through a pre-existing/naturally fractured intervals. Numerous lost-time incidents related to wellbore instability-related problems were experienced, ranging from tight hole (remedied by reaming) to Overpulls, pack-off followed by stuck pipe, fill on-bottom to difficulties in running casing, and tensile cavings to high gas associated with drilling breaks. These problems were observed particularly when drilling previous and current wells in the O-field X. Many of the wells in O-field X were drilled with water-based mud (WBM) for top-hole and POBM for intermediate hole section. However, drilling the most recent well became more challenging with issues of severe losses just below the 13-3/8inch shoe where an interbedded lignite formation characterized with pre-existing fractures was drilled through. Faced with continual non-productive time (NPT), the predrill GeoMechanical report was immediately reviewed coupled with the stress caging procedure adopted to further mitigate the loss circulation wellbore instability problem. The recommendations arising from the comprehensive review of the GeoMechanical window, stress caging and drilling experience analyses was immediately implemented to improve performance which has helped in drilling the well to final completion. This paper highlights the importance of integrating GeoMechanics, stress caging and with proper drilling practices which has helped in delivery of the candidate well. A full-scale GeoMechanical window review was proactively adopted considering the mid-line collapse gradient approach for unconsolidated, naturally fracture formations and critical depleted intervals. All the above strategies were adopted, which assisted in safe delivery of candidate well in O-field X.

Integration of Leaky-Integrate-and-Fire Neurons in Standard Machine Learning Architectures to Generate Hybrid Networks: A Surrogate Gradient Approach

Up to now, modern machine learning (ML) has been based on approximating big data sets with high-dimensional functions, taking advantage of huge computational resources. We show that biologically inspired neuron models such as the leaky-integrate-and-fire (LIF) neuron provide novel and efficient ways of information processing. They can be integrated in machine learning models and are a potential target to improve ML performance. Thus, we have derived simple update rules for LIF units to numerically integrate the differential equations. We apply a surrogate gradient approach to train the LIF units via backpropagation. We demonstrate that tuning the leak term of the LIF neurons can be used to run the neurons in different operating modes, such as simple signal integrators or coincidence detectors. Furthermore, we show that the constant surrogate gradient, in combination with tuning the leak term of the LIF units, can be used to achieve the learning dynamics of more complex surrogate gradients. To prove the validity of our method, we applied it to established image data sets (the Oxford 102 flower data set, MNIST), implemented various network architectures, used several input data encodings and demonstrated that the method is suitable to achieve state-of-the-art classification performance. We provide our method as well as further surrogate gradient methods to train spiking neural networks via backpropagation as an open-source KERAS package to make it available to the neuroscience and machine learning community. To increase the interpretability of the underlying effects and thus make a small step toward opening the black box of machine learning, we provide interactive illustrations, with the possibility of systematically monitoring the effects of parameter changes on the learning characteristics.

Predicting the Compressive Strength of Concrete Using an RBF-ANN Model

In this study, a radial basis function (RBF) artificial neural network (ANN) model for predicting the 28-day compressive strength of concrete is established. The database used in this study is the expansion by adding data from other works to the one used in the author’s previous work. The stochastic gradient approach presented in the textbook is employed for determining the centers of RBFs and their shape parameters. With an extremely large number of training iterations and just a few RBFs in the ANN, all the RBF-ANNs have converged to the solutions of global minimum error. So, the only consideration of whether the ANN can work in practical uses is just the issue of over-fitting. The ANN with only three RBFs is finally chosen. The results of verification imply that the present RBF-ANN model outperforms the BP-ANN model in the author’s previous work. The centers of the RBFs, their shape parameters, their weights, and the threshold are all listed in this article. With these numbers and using the formulae expressed in this article, anyone can predict the 28-day compressive strength of concrete according to the concrete mix proportioning on his/her own.

A NON-GRADIENT APPROACH FOR THREE DIMENSIONAL TOPOLOGY OPTIMIZATION

The present work is devoted to the extension of the non-gradient approach, namely Proportional Topology Optimization (PTO), for compliance minimization of three-dimensional (3D) structures. Two schemes of material interpolation within the framework of the solid isotropic material with penalization (SIMP), i.e. the power function and the logistic function are analyzed. Through a comparative study, the efficiency of the logistic-type interpolation scheme is highlighted. Since no sensitivity is involved in the approach, a density filter is applied instead of sensitivity filter to avoid checkerboard issue