scholarly journals Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: general algorithms and Lorenz96 case study (v1.0)

2020 ◽  
Author(s):  
Stephan Rasp
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Online Learning ◽  
High Resolution ◽  
Machine Learning Algorithms ◽  
Training Dataset ◽  
The Neural Network ◽  
Training Stage ◽  
Lorenz 96 ◽  
Resolution Simulation

Abstract. Over the last couple of years, machine learning parameterizations have emerged as a potential way to improve the representation of sub-grid processes in Earth System Models (ESMs). So far, all studies were based on the same three-step approach: first a training dataset was created from a high-resolution simulation, then a machine learning algorithms was fitted to this dataset, before the trained algorithms was implemented in the ESM. The resulting online simulations were frequently plagued by instabilities and biases. Here, coupled online learning is proposed as a way to combat these issues. Coupled learning can be seen as a second training stage in which the pretrained machine learning parameterization, specifically a neural network, is run in parallel with a high-resolution simulation. The high-resolution simulation is kept in sync with the neural network-driven ESM through constant nudging. This enables the neural network to learn from the tendencies that the high-resolution simulation would produce if it experienced the states the neural network creates. The concept is illustrated using the Lorenz 96 model, where coupled learning is able to recover the "true" parameterizations. Further, detailed algorithms for the implementation of coupled learning in 3D cloud-resolving models and the super parameterization framework are presented. Finally, outstanding challenges and issues not resolved by this approach are discussed.

scholarly journals Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: general algorithms and Lorenz 96 case study (v1.0)

2020 ◽  
Vol 13 (5) ◽  
pp. 2185-2196
Author(s):  
Stephan Rasp
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Online Learning ◽  
High Resolution ◽  
Learning Algorithm ◽  
Training Dataset ◽  
The Neural Network ◽  
Training Stage ◽  
Lorenz 96 ◽  
Resolution Simulation

Abstract. Over the last couple of years, machine learning parameterizations have emerged as a potential way to improve the representation of subgrid processes in Earth system models (ESMs). So far, all studies were based on the same three-step approach: first a training dataset was created from a high-resolution simulation, then a machine learning algorithm was fitted to this dataset, before the trained algorithm was implemented in the ESM. The resulting online simulations were frequently plagued by instabilities and biases. Here, coupled online learning is proposed as a way to combat these issues. Coupled learning can be seen as a second training stage in which the pretrained machine learning parameterization, specifically a neural network, is run in parallel with a high-resolution simulation. The high-resolution simulation is kept in sync with the neural network-driven ESM through constant nudging. This enables the neural network to learn from the tendencies that the high-resolution simulation would produce if it experienced the states the neural network creates. The concept is illustrated using the Lorenz 96 model, where coupled learning is able to recover the “true” parameterizations. Further, detailed algorithms for the implementation of coupled learning in 3D cloud-resolving models and the super parameterization framework are presented. Finally, outstanding challenges and issues not resolved by this approach are discussed.

A comparison of deep machine learning and Monte Carlo methods for facies classification from seismic data

Geophysics ◽  
2020 ◽  
Vol 85 (4) ◽  
pp. WA41-WA52 ◽  
Author(s):  
Dario Grana ◽  
Leonardo Azevedo ◽  
Mingliang Liu
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Monte Carlo ◽  
Monte Carlo Methods ◽  
Seismic Data ◽  
Transition Probabilities ◽  
Machine Learning Algorithms ◽  
Data Set ◽  
Facies Classification ◽  
The Neural Network

Among the large variety of mathematical and computational methods for estimating reservoir properties such as facies and petrophysical variables from geophysical data, deep machine-learning algorithms have gained significant popularity for their ability to obtain accurate solutions for geophysical inverse problems in which the physical models are partially unknown. Solutions of classification and inversion problems are generally not unique, and uncertainty quantification studies are required to quantify the uncertainty in the model predictions and determine the precision of the results. Probabilistic methods, such as Monte Carlo approaches, provide a reliable approach for capturing the variability of the set of possible models that match the measured data. Here, we focused on the classification of facies from seismic data and benchmarked the performance of three different algorithms: recurrent neural network, Monte Carlo acceptance/rejection sampling, and Markov chain Monte Carlo. We tested and validated these approaches at the well locations by comparing classification predictions to the reference facies profile. The accuracy of the classification results is defined as the mismatch between the predictions and the log facies profile. Our study found that when the training data set of the neural network is large enough and the prior information about the transition probabilities of the facies in the Monte Carlo approach is not informative, machine-learning methods lead to more accurate solutions; however, the uncertainty of the solution might be underestimated. When some prior knowledge of the facies model is available, for example, from nearby wells, Monte Carlo methods provide solutions with similar accuracy to the neural network and allow a more robust quantification of the uncertainty, of the solution.

scholarly journals Predicting Alert Source Device using Machine Learning Algorithms

2020 ◽  
Vol 9 (9) ◽  
pp. 1-10
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Model Building ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Research Work ◽  
Machine Learning Algorithms ◽  
Training Dataset ◽  
Imbalanced Dataset ◽  
Daunting Task

In a large distributed virtualized environment, predicting the alerting source from its text seems to be daunting task. This paper explores the option of using machine learning algorithm to solve this problem. Unfortunately, our training dataset is highly imbalanced. Where 96% of alerting data is reported by 24% of alerting sources. This is the expected dataset in any live distributed virtualized environment, where new version of device will have relatively less alert compared to older devices. Any classification effort with such imbalanced dataset present different set of challenges compared to binary classification. This type of skewed data distribution makes conventional machine learning less effective, especially while predicting the minority device type alerts. Our challenge is to build a robust model which can cope with this imbalanced dataset and achieves relative high level of prediction accuracy. This research work stared with traditional regression and classification algorithms using bag of words model. Then word2vec and doc2vec models are used to represent the words in vector formats, which preserve the sematic meaning of the sentence. With this alerting text with similar message will have same vector form representation. This vectorized alerting text is used with Logistic Regression for model building. This yields better accuracy, but the model is relatively complex and demand more computational resources. Finally, simple neural network is used for this multi-class text classification problem domain by using keras and tensorflow libraries. A simple two layered neural network yielded 99 % accuracy, even though our training dataset was not balanced. This paper goes through the qualitative evaluation of the different machine learning algorithms and their respective result. Finally, two layered deep learning algorithms is selected as final solution, since it takes relatively less resource and time with better accuracy values.

scholarly journals Effective Methodology for Co-Referential Aspect Based Sentiment Analysis of Tourist Reviews

2020 ◽  
pp. 523-529
Author(s):  
Kamlesh A. Waghmare ◽  
Sheetal K. Bhala
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Sentiment Analysis ◽  
High Accuracy ◽  
Machine Learning Algorithms ◽  
Data Sets ◽  
Knowledge Processing ◽  
The Neural Network ◽  
The World ◽  
Sentiment Score

Tourist reviews are the source of data that is going to be used for the travelers around the world to find the hotels for their stay according to their comfort. In this the hotels are ranked over the parameters or aspects considered keeping travelers in mind. This computation of data sets is done with the help of the machine learning algorithms and the neural network. The knowledge processing done over the reviews generates the sentiment score for each hotel with respect to the aspects defined. Here, the explicit , implicit and co-referential aspects are identified by suppressing the noise. This paper proposes the method that can be best used for the detection of the sentiments with the high accuracy.

scholarly journals Application of Machine Learning Algorithms for Solving Problems in the Oil and Gas Sector

2021 ◽  
Vol 19 (3) ◽  
pp. 55-64
Author(s):  
K. N. Maiorov ◽  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Reinforcement Learning ◽  
Machine Learning Algorithms ◽  
Design Stage ◽  
Learning Methods ◽  
Field Development ◽  
Advantages And Disadvantages ◽  
Machine Learning Methods ◽  
The Neural Network

The paper examines the life cycle of field development, analyzes the processes of the field development design stage for the application of machine learning methods. For each process, relevant problems are highlighted, existing solutions based on machine learning methods, ideas and problems are proposed that could be effectively solved by machine learning methods. For the main part of the processes, examples of solutions are briefly described; the advantages and disadvantages of the approaches are identified. The most common solution method is feed-forward neural networks. Subject to preliminary normalization of the input data, this is the most versatile algorithm for regression and classification problems. However, in the problem of selecting wells for hydraulic fracturing, a whole ensemble of machine learning models was used, where, in addition to a neural network, there was a random forest, gradient boosting and linear regression. For the problem of optimizing the placement of a grid of oil wells, the disadvantages of existing solutions based on a neural network and a simple reinforcement learning approach based on Markov decision-making process are identified. A deep reinforcement learning algorithm called Alpha Zero is proposed, which has previously shown significant results in the role of artificial intelligence for games. This algorithm is a decision tree search that directs the neural network: only those branches that have received the best estimates from the neural network are considered more thoroughly. The paper highlights the similarities between the tasks for which Alpha Zero was previously used, and the task of optimizing the placement of a grid of oil producing wells. Conclusions are made about the possibility of using and modifying the algorithm of the optimization problem being solved. Аn approach is proposed to take into account symmetric states in a Monte Carlo tree to reduce the number of required simulations.

scholarly journals Classification of potential blood donors using machine learning algorithms approach

2020 ◽  
Vol 8 (3) ◽  
pp. 217-221
Author(s):  
Merinda Lestandy ◽  
Lailis Syafa'ah ◽  
Amrul Faruq
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Blood Donor ◽  
Blood Donors ◽  
Naive Bayes ◽  
Learning Algorithms ◽  
Naïve Bayes ◽  
Machine Learning Algorithms ◽  
Blood Type ◽  
The Neural Network

Blood donation is the process of taking blood from someone used for blood transfusions. Blood type, sex, age, blood pressure, and hemoglobin are blood donor criteria that must be met and processed manually to classify blood donor eligibility. The manual process resulted in an irregular blood supply because blood donor candidates did not meet the criteria. This study implements machine learning algorithms includes kNN, naïve Bayes, and neural network methods to determine the eligibility of blood donors. This study used 600 training data divided into two classes, namely potential and non-potential donors. The test results show that the accuracy of the neural network is 84.3 %, higher than kNN and naïve Bayes, respectively of 75 % and 84.17 %. It indicates that the neural network method outperforms comparing with kNN and naïve Bayes.

scholarly journals Data Augmentation and Pretraining for Template-Based Retrosynthetic Prediction in Computer-Aided Synthesis Planning

2020 ◽  
Author(s):  
Michael Fortunato ◽  
Connor W. Coley ◽  
Brian Barnes ◽  
Klavs F. Jensen
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Data Augmentation ◽  
Machine Learning Algorithms ◽  
Learning Models ◽  
The Neural Network ◽  
Computer Aided ◽  
Synthesis Planning ◽  
The One ◽  
Machine Learning Models

This work presents efforts to augment the performance of data-driven machine learning algorithms for reaction template recommendation used in computer-aided synthesis planning software. Often, machine learning models designed to perform the task of prioritizing reaction templates or molecular transformations are focused on reporting high accuracy metrics for the one-to-one mapping of product molecules in reaction databases to the template extracted from the recorded reaction. The available templates that get selected for inclusion in these machine learning models have been previously limited to those that appear frequently in the reaction databases and exclude potentially useful transformations. By augmenting open-access datasets of organic reactions with artificially calculated template applicability and pretraining a template relevance neural network on this augmented applicability dataset, we report an increase in the template applicability recall and an increase in the diversity of predicted precursors. The augmentation and pretraining effectively teaches the neural network an increased set of templates that could theoretically lead to successful reactions for a given target. Even on a small dataset of well curated reactions, the data augmentation and pretraining methods resulted in an increase in top-1 accuracy, especially for rare templates, indicating these strategies can be very useful for small datasets.

scholarly journals Developing a Humanoid Robot Platform

2018 ◽  
Vol 8 (03) ◽  
Author(s):  
Nuralem Abizov ◽  
Jia Yuan Huang ◽  
Fei Gao
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Humanoid Robot ◽  
The Body ◽  
Machine Learning Algorithms ◽  
Current Position ◽  
The Neural Network ◽  
Fully Connected ◽  
And Robotics ◽  
Robot Platform

This paper is focused on developing a platform that helps researchers to create verify and implement their machine learning algorithms to a humanoid robot in real environment. The presented platform is durable, easy to fix, upgrade, fast to assemble and cheap. Also, using this platform we present an approach that solves a humanoid balancing problem, which uses only fully connected neural network as a basic idea for real time balancing. The method consists of 3 main conditions: 1) using different types of sensors detect the current position of the body and generate the input information for the neural network, 2) using fully connected neural network produce the correct output, 3) using servomotors make movements that will change the current position to the new one. During field test the humanoid robot can balance on the moving platform that tilts up to 10 degrees to any direction. Finally, we have shown that using our platform we can do research and compare different neural networks in similar conditions which can be important for the researchers to do analyses in machine learning and robotics

MultiResU-Net: Neural Network for Salt Bodies Delineation and QC Manual Interpretation

2021 ◽  
Author(s):  
Yesser HajNasser
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Ground Truth ◽  
Complex Salt ◽  
Training Dataset ◽  
Systematic Bias ◽  
Spatial Features ◽  
The Neural Network ◽  
Seismic Image ◽  
Machine Learning Approach

Abstract Accurate delineation of salt bodies is essential for the characterization of hydrocarbon accumulation and seal efficiency in offshore reservoirs. The interpretation of these subsurface features is heavily dependent on visual picking. This in turn could introduce systematic bias into the task of salt body interpretation. In this study, we introduce a novel machine learning approach of a deep neural network to mimic an experienced geophysical interpreter's intellect in interpreting salt bodies. Here, the benefits of using machine learning are demonstrated by implementing the MultiResU-Net network. The network is an improved form of the classic U-Net. It presents two key architectural improvements. First, it replaces the simple convolutional layers with inception-like blocks with varying kernel sizes to reconcile the spatial features learned from different seismic image contexts. Second, it incorporates residual convolutional layers along the skip connections between the downsampling and the upsampling paths. This aims at compensating for the disparity between the lower-level features coming from the early stages of the downsampling path and the much higher-level features coming from the upsampling path. From the primary results using the TGS Salt Identification Challenge dataset, the MultiResU-Net outperformed the classic U-Net in identifying salt bodies and showed good agreement with the ground truth. Additionally, in the case of complex salt body geometries, the MultiResU-Net predictions exhibited some intriguing differences with the ground truth interpretation. Although the network validation accuracy is about 95%, some of these occasional discrepancies between the neural network predictions and the ground truth highlighted the subjectivity of the manual interpretation. Consequently, this raises the need to incorporate these neural networks that are prone to random perturbations to QC manual geophysical interpretation. To bridge the gap between the human interpretation and the machine learning predictions, we propose a closed-loop-machine-learning workflow that aims at optimizing the training dataset by incorporating both the consistency of the neural network and the intellect of an experienced geophysical interpreter.

scholarly journals Machine learning algorithms for predicting coronary artery disease: efforts toward an open source solution

2021 ◽  
pp. FSO698
Author(s):  
Aravind Akella ◽  
Sudheer Akella
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Coronary Artery Disease ◽  
Coronary Artery ◽  
Open Source ◽  
Computer Code ◽  
Machine Learning Algorithms ◽  
Clinical Tool ◽  
The Neural Network ◽  
Artery Disease

Aim: The development of coronary artery disease (CAD), a highly prevalent disease worldwide, is influenced by several modifiable risk factors. Predictive models built using machine learning (ML) algorithms may assist clinicians in timely detection of CAD and may improve outcomes. Materials & methods: In this study, we applied six different ML algorithms to predict the presence of CAD amongst patients listed in ‘the Cleveland dataset.’ The generated computer code is provided as a working open source solution with the ultimate goal to achieve a viable clinical tool for CAD detection. Results: All six ML algorithms achieved accuracies greater than 80%, with the ‘neural network’ algorithm achieving accuracy greater than 93%. The recall achieved with the ‘neural network’ model is also the highest of the six models (0.93), indicating that predictive ML models may provide diagnostic value in CAD.

Sign in / Sign up

Export Citation Format

Share Document