scholarly journals Replacing Chemical Intuition by Machine Learning: a Mixed Design of Experiments - Reinforcement Learning Approach to the Construction of Training Sets for Model Hamiltonians

2021 ◽  
Author(s):  
Ruben Staub ◽  
Stephan Steinmann
Keyword(s):  
Machine Learning ◽  
Design Of Experiments ◽  
Computational Cost ◽  
Geometry Optimization ◽  
Training Set ◽  
Geometric Patterns ◽  
Co Oxidation Reaction ◽  
Model Hamiltonian ◽  
Efficient Exploration ◽  
Model Hamiltonians

Model Hamiltonians based on the so-called cluster expansion (CE), which consist of a linear fit of parameters corresponding to geometric patterns, provide an efficient and rigorous means to quickly evaluate the energy of diverse arrangements of adsorbate mixtures on reactive surfaces as typically relevant for heterogeneous catalysis. However, establishing the model Hamiltonian is a tedious task, requiring the construction and optimization of many geometries. Today, most of these geometries are constructed by hand, based on chemical intuition or random choices. Hence, the quality of the training set is unlikely to be optimal and its construction is not reproducible. Herein, we propose a reformulation of the construction of the training set as a strategy-based game, aiming at an efficient exploration of the relevant patterns constituting the model Hamiltonian. Based on this reformulation, we exploit a typical active learning solution for machine-learning such a strategy game: an upper confidence tree (UCT) based framework. However, in contrast to standard games, evaluating the true score is computationally expensive, as it requires a costly geometry optimization. Hence, we augment the UCT with a pre-exploration step inspired by the variance-based Design of Experiments (DoE) methods. This novel mixed UCT+DoE framework allows to automatically construct a well adapted training set, minimizing computational cost and user-intervention. As a proof of principle, we apply our UCT+DoE approach on the CO oxidation reaction on Pd(111), for which a relevant model Hamiltonian has been established previously. The results demonstrate the effectiveness of the custom built UCT and its significant benefits on a DoE-based approach.

BAND NN: A Deep Learning Framework For Energy Prediction and Geometry Optimization of Organic Small Molecules

2019 ◽  
Author(s):  
Siddhartha Laghuvarapu ◽  
Yashaswi Pathak ◽  
U. Deva Priyakumar
Keyword(s):  
Machine Learning ◽  
Density Functional ◽  
Computational Cost ◽  
Geometry Optimization ◽  
Dft Methods ◽  
Energy Prediction ◽  
Machine Learning Model ◽  
Equilibrium Structures ◽  
High Level ◽  
Non Equilibrium

Recent advances in artificial intelligence along with development of large datasets of energies calculated using quantum mechanical (QM)/density functional theory (DFT) methods have enabled prediction of accurate molecular energies at reasonably low computational cost. However, machine learning models that have been reported so far requires the atomic positions obtained from geometry optimizations using high level QM/DFT methods as input in order to predict the energies, and do not allow for geometry optimization. In this paper, a transferable and molecule-size independent machine learning model (BAND NN) based on a chemically intuitive representation inspired by molecular mechanics force fields is presented. The model predicts the atomization energies of equilibrium and non-equilibrium structures as sum of energy contributions from bonds (B), angles (A), nonbonds (N) and dihedrals (D) at remarkable accuracy. The robustness of the proposed model is further validated by calculations that span over the conformational, configurational and reaction space. The transferability of this model on systems larger than the ones in the dataset is demonstrated by performing calculations on select large molecules. Importantly, employing the BAND NN model, it is possible to perform geometry optimizations starting from non-equilibrium structures along with predicting their energies.

scholarly journals Size-Extensive Molecular Machine Learning with Global Descriptors

2019 ◽  
Author(s):  
Hyunwook Jung ◽  
Sina Stocker ◽  
Christian Kunkel ◽  
Harald Oberhofer ◽  
Byungchan Han ◽  
...  
Keyword(s):  
Machine Learning ◽  
Small Molecules ◽  
Computational Cost ◽  
Many Body ◽  
Training Set ◽  
Large Molecules ◽  
Large Size ◽  
Size Extensivity ◽  
The Many ◽  
Structure Calculations

<div> <div> <div> <p>Machine learning (ML) models are increasingly used to predict molecular prop- erties in a high-throughput setting at a much lower computational cost than con- ventional electronic structure calculations. Such ML models require descriptors that encode the molecular structure in a vector. These descriptors are generally designed to respect the symmetries and invariances of the target property. However, size- extensivity is usually not guaranteed for so-called global descriptors. In this contri- bution, we show how extensivity can be build into ML models with global descriptors such as the Many-Body Tensor Representation. Properties of extensive and non- extensive models for the atomization energy are systematically explored by training on small molecules and testing on small, medium and large molecules. Our result shows that the non-extensive model is only useful in the size-range of its training set, whereas the extensive models provide reasonable predictions across large size differences. Remaining sources of error for the extensive models are discussed. </p> </div> </div> </div>

scholarly journals Size-Extensive Molecular Machine Learning with Global Descriptors

2019 ◽  
Author(s):  
Hyunwook Jung ◽  
Sina Stocker ◽  
Christian Kunkel ◽  
Harald Oberhofer ◽  
Byungchan Han ◽  
...  
Keyword(s):  
Machine Learning ◽  
Small Molecules ◽  
Computational Cost ◽  
Many Body ◽  
Training Set ◽  
Large Molecules ◽  
Large Size ◽  
Size Extensivity ◽  
The Many ◽  
Structure Calculations

<div> <div> <div> <p>Machine learning (ML) models are increasingly used to predict molecular prop- erties in a high-throughput setting at a much lower computational cost than con- ventional electronic structure calculations. Such ML models require descriptors that encode the molecular structure in a vector. These descriptors are generally designed to respect the symmetries and invariances of the target property. However, size- extensivity is usually not guaranteed for so-called global descriptors. In this contri- bution, we show how extensivity can be build into ML models with global descriptors such as the Many-Body Tensor Representation. Properties of extensive and non- extensive models for the atomization energy are systematically explored by training on small molecules and testing on small, medium and large molecules. Our result shows that the non-extensive model is only useful in the size-range of its training set, whereas the extensive models provide reasonable predictions across large size differences. Remaining sources of error for the extensive models are discussed. </p> </div> </div> </div>

BAND NN: A Deep Learning Framework For Energy Prediction and Geometry Optimization of Organic Small Molecules

2019 ◽  
Author(s):  
Siddhartha Laghuvarapu ◽  
Yashaswi Pathak ◽  
U. Deva Priyakumar
Keyword(s):  
Machine Learning ◽  
Density Functional ◽  
Computational Cost ◽  
Geometry Optimization ◽  
Dft Methods ◽  
Energy Prediction ◽  
Machine Learning Model ◽  
Equilibrium Structures ◽  
High Level ◽  
Non Equilibrium

Recent advances in artificial intelligence along with development of large datasets of energies calculated using quantum mechanical (QM)/density functional theory (DFT) methods have enabled prediction of accurate molecular energies at reasonably low computational cost. However, machine learning models that have been reported so far requires the atomic positions obtained from geometry optimizations using high level QM/DFT methods as input in order to predict the energies, and do not allow for geometry optimization. In this paper, a transferable and molecule-size independent machine learning model (BAND NN) based on a chemically intuitive representation inspired by molecular mechanics force fields is presented. The model predicts the atomization energies of equilibrium and non-equilibrium structures as sum of energy contributions from bonds (B), angles (A), nonbonds (N) and dihedrals (D) at remarkable accuracy. The robustness of the proposed model is further validated by calculations that span over the conformational, configurational and reaction space. The transferability of this model on systems larger than the ones in the dataset is demonstrated by performing calculations on select large molecules. Importantly, employing the BAND NN model, it is possible to perform geometry optimizations starting from non-equilibrium structures along with predicting their energies.

Nanosecond Photodynamics Simulations of a Cis-Trans Isomerization Are Enabled by Machine Learning

2020 ◽  
Author(s):  
Jingbai Li ◽  
Patrick Reiser ◽  
André Eberhard ◽  
Pascal Friederich ◽  
Steven Lopez
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Excited State ◽  
Adaptive Sampling ◽  
Computational Cost ◽  
Ground Truth ◽  
Absolute Error ◽  
Photochemical Reactions ◽  
Computational Techniques ◽  
Full Potential

<p>Photochemical reactions are being increasingly used to construct complex molecular architectures with mild and straightforward reaction conditions. Computational techniques are increasingly important to understand the reactivities and chemoselectivities of photochemical isomerization reactions because they offer molecular bonding information along the excited-state(s) of photodynamics. These photodynamics simulations are resource-intensive and are typically limited to 1–10 picoseconds and 1,000 trajectories due to high computational cost. Most organic photochemical reactions have excited-state lifetimes exceeding 1 picosecond, which places them outside possible computational studies. Westermeyr <i>et al.</i> demonstrated that a machine learning approach could significantly lengthen photodynamics simulation times for a model system, methylenimmonium cation (CH<sub>2</sub>NH<sub>2</sub><sup>+</sup>).</p><p>We have developed a Python-based code, Python Rapid Artificial Intelligence <i>Ab Initio</i> Molecular Dynamics (PyRAI<sup>2</sup>MD), to accomplish the unprecedented 10 ns <i>cis-trans</i> photodynamics of <i>trans</i>-hexafluoro-2-butene (CF<sub>3</sub>–CH=CH–CF<sub>3</sub>) in 3.5 days. The same simulation would take approximately 58 years with ground-truth multiconfigurational dynamics. We proposed an innovative scheme combining Wigner sampling, geometrical interpolations, and short-time quantum chemical trajectories to effectively sample the initial data, facilitating the adaptive sampling to generate an informative and data-efficient training set with 6,232 data points. Our neural networks achieved chemical accuracy (mean absolute error of 0.032 eV). Our 4,814 trajectories reproduced the S<sub>1</sub> half-life (60.5 fs), the photochemical product ratio (<i>trans</i>: <i>cis</i> = 2.3: 1), and autonomously discovered a pathway towards a carbene. The neural networks have also shown the capability of generalizing the full potential energy surface with chemically incomplete data (<i>trans</i> → <i>cis</i> but not <i>cis</i> → <i>trans</i> pathways) that may offer future automated photochemical reaction discoveries.</p>

scholarly journals Multirate Processing with Selective Subbands and Machine Learning for Efficient Arrhythmia Classification

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1511
Author(s):  
Saeed Mian Qaisar ◽  
Alaeddine Mihoub ◽  
Moez Krichen ◽  
Humaira Nisar
Keyword(s):  
Machine Learning ◽  
Signal Reconstruction ◽  
Computational Cost ◽  
Machine Learning Techniques ◽  
Features Selection ◽  
Frequency Content ◽  
Fixed Rate ◽  
Ecg Signals ◽  
Learning Techniques ◽  
Multirate Processing

The usage of wearable gadgets is growing in the cloud-based health monitoring systems. The signal compression, computational and power efficiencies play an imperative part in this scenario. In this context, we propose an efficient method for the diagnosis of cardiovascular diseases based on electrocardiogram (ECG) signals. The method combines multirate processing, wavelet decomposition and frequency content-based subband coefficient selection and machine learning techniques. Multirate processing and features selection is used to reduce the amount of information processed thus reducing the computational complexity of the proposed system relative to the equivalent fixed-rate solutions. Frequency content-dependent subband coefficient selection enhances the compression gain and reduces the transmission activity and computational cost of the post cloud-based classification. We have used MIT-BIH dataset for our experiments. To avoid overfitting and biasness, the performance of considered classifiers is studied by using five-fold cross validation (5CV) and a novel proposed partial blind protocol. The designed method achieves more than 12-fold computational gain while assuring an appropriate signal reconstruction. The compression gain is 13 times compared to fixed-rate counterparts and the highest classification accuracies are 97.06% and 92.08% for the 5CV and partial blind cases, respectively. Results suggest the feasibility of detecting cardiac arrhythmias using the proposed approach.

scholarly journals Classification of multiwavelength transients with Machine Learning

2020 ◽  
Author(s):  
K Sooknunan ◽  
M Lochner ◽  
Bruce A Bassett ◽  
H V Peiris ◽  
R Fender ◽  
...  
Keyword(s):  
Machine Learning ◽  
Small Sample ◽  
Light Curves ◽  
Machine Learning Techniques ◽  
Optical Data ◽  
Test Time ◽  
Test Accuracy ◽  
Training Set ◽  
The Impact

Abstract With the advent of powerful telescopes such as the Square Kilometer Array and the Vera C. Rubin Observatory, we are entering an era of multiwavelength transient astronomy that will lead to a dramatic increase in data volume. Machine learning techniques are well suited to address this data challenge and rapidly classify newly detected transients. We present a multiwavelength classification algorithm consisting of three steps: (1) interpolation and augmentation of the data using Gaussian processes; (2) feature extraction using wavelets; (3) classification with random forests. Augmentation provides improved performance at test time by balancing the classes and adding diversity into the training set. In the first application of machine learning to the classification of real radio transient data, we apply our technique to the Green Bank Interferometer and other radio light curves. We find we are able to accurately classify most of the eleven classes of radio variables and transients after just eight hours of observations, achieving an overall test accuracy of 78%. We fully investigate the impact of the small sample size of 82 publicly available light curves and use data augmentation techniques to mitigate the effect. We also show that on a significantly larger simulated representative training set that the algorithm achieves an overall accuracy of 97%, illustrating that the method is likely to provide excellent performance on future surveys. Finally, we demonstrate the effectiveness of simultaneous multiwavelength observations by showing how incorporating just one optical data point into the analysis improves the accuracy of the worst performing class by 19%.

scholarly journals Effects of Training Set Size on Supervised Machine-Learning Land-Cover Classification of Large-Area High-Resolution Remotely Sensed Data

2021 ◽  
Vol 13 (3) ◽  
pp. 368
Author(s):  
Christopher A. Ramezan ◽  
Timothy A. Warner ◽  
Aaron E. Maxwell ◽  
Bradley S. Price
Keyword(s):  
Machine Learning ◽  
Sample Size ◽  
Remotely Sensed ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Sample Sizes ◽  
Remotely Sensed Data ◽  
Large Area ◽  
Training Set ◽  
Set Size

The size of the training data set is a major determinant of classification accuracy. Nevertheless, the collection of a large training data set for supervised classifiers can be a challenge, especially for studies covering a large area, which may be typical of many real-world applied projects. This work investigates how variations in training set size, ranging from a large sample size (n = 10,000) to a very small sample size (n = 40), affect the performance of six supervised machine-learning algorithms applied to classify large-area high-spatial-resolution (HR) (1–5 m) remotely sensed data within the context of a geographic object-based image analysis (GEOBIA) approach. GEOBIA, in which adjacent similar pixels are grouped into image-objects that form the unit of the classification, offers the potential benefit of allowing multiple additional variables, such as measures of object geometry and texture, thus increasing the dimensionality of the classification input data. The six supervised machine-learning algorithms are support vector machines (SVM), random forests (RF), k-nearest neighbors (k-NN), single-layer perceptron neural networks (NEU), learning vector quantization (LVQ), and gradient-boosted trees (GBM). RF, the algorithm with the highest overall accuracy, was notable for its negligible decrease in overall accuracy, 1.0%, when training sample size decreased from 10,000 to 315 samples. GBM provided similar overall accuracy to RF; however, the algorithm was very expensive in terms of training time and computational resources, especially with large training sets. In contrast to RF and GBM, NEU, and SVM were particularly sensitive to decreasing sample size, with NEU classifications generally producing overall accuracies that were on average slightly higher than SVM classifications for larger sample sizes, but lower than SVM for the smallest sample sizes. NEU however required a longer processing time. The k-NN classifier saw less of a drop in overall accuracy than NEU and SVM as training set size decreased; however, the overall accuracies of k-NN were typically less than RF, NEU, and SVM classifiers. LVQ generally had the lowest overall accuracy of all six methods, but was relatively insensitive to sample size, down to the smallest sample sizes. Overall, due to its relatively high accuracy with small training sample sets, and minimal variations in overall accuracy between very large and small sample sets, as well as relatively short processing time, RF was a good classifier for large-area land-cover classifications of HR remotely sensed data, especially when training data are scarce. However, as performance of different supervised classifiers varies in response to training set size, investigating multiple classification algorithms is recommended to achieve optimal accuracy for a project.

scholarly journals Predicting Increased Blood Pressure Using Machine Learning

2014 ◽  
Vol 2014 ◽  
pp. 1-12 ◽  
Author(s):  
Hudson Fernandes Golino ◽  
Liliany Souza de Brito Amaral ◽  
Stenio Fernando Pimentel Duarte ◽  
Cristiano Mauro Assis Gomes ◽  
Telma de Jesus Soares ◽  
...  
Keyword(s):  
Machine Learning ◽  
Blood Pressure ◽  
Predictive Power ◽  
Classification Tree ◽  
Training Group ◽  
Test Sample ◽  
Hip Circumference ◽  
Training Set ◽  
Learning Technique ◽  
Tree Data

The present study investigates the prediction of increased blood pressure by body mass index (BMI), waist (WC) and hip circumference (HC), and waist hip ratio (WHR) using a machine learning technique named classification tree. Data were collected from 400 college students (56.3% women) from 16 to 63 years old. Fifteen trees were calculated in the training group for each sex, using different numbers and combinations of predictors. The result shows that for women BMI, WC, and WHR are the combination that produces the best prediction, since it has the lowest deviance (87.42), misclassification (.19), and the higher pseudoR2(.43). This model presented a sensitivity of 80.86% and specificity of 81.22% in the training set and, respectively, 45.65% and 65.15% in the test sample. For men BMI, WC, HC, and WHC showed the best prediction with the lowest deviance (57.25), misclassification (.16), and the higher pseudoR2(.46). This model had a sensitivity of 72% and specificity of 86.25% in the training set and, respectively, 58.38% and 69.70% in the test set. Finally, the result from the classification tree analysis was compared with traditional logistic regression, indicating that the former outperformed the latter in terms of predictive power.

An Automated Machine Learning-Genetic Algorithm Framework With Active Learning for Design Optimization

2021 ◽  
Vol 143 (8) ◽  
Author(s):  
Opeoluwa Owoyele ◽  
Pinaki Pal ◽  
Alvaro Vidal Torreira
Keyword(s):  
Machine Learning ◽  
Active Learning ◽  
Design Optimization ◽  
Fuel Injection ◽  
A Priori ◽  
Computational Cost ◽  
Learning Approach ◽  
Data Generation ◽  
Compression Ignition Engine ◽  
Hyperparameter Selection

AbstractThe use of machine learning (ML)-based surrogate models is a promising technique to significantly accelerate simulation-driven design optimization of internal combustion (IC) engines, due to the high computational cost of running computational fluid dynamics (CFD) simulations. However, training the ML models requires hyperparameter selection, which is often done using trial-and-error and domain expertise. Another challenge is that the data required to train these models are often unknown a priori. In this work, we present an automated hyperparameter selection technique coupled with an active learning approach to address these challenges. The technique presented in this study involves the use of a Bayesian approach to optimize the hyperparameters of the base learners that make up a super learner model. In addition to performing hyperparameter optimization (HPO), an active learning approach is employed, where the process of data generation using simulations, ML training, and surrogate optimization is performed repeatedly to refine the solution in the vicinity of the predicted optimum. The proposed approach is applied to the optimization of a compression ignition engine with control parameters relating to fuel injection, in-cylinder flow, and thermodynamic conditions. It is demonstrated that by automatically selecting the best values of the hyperparameters, a 1.6% improvement in merit value is obtained, compared to an improvement of 1.0% with default hyperparameters. Overall, the framework introduced in this study reduces the need for technical expertise in training ML models for optimization while also reducing the number of simulations needed for performing surrogate-based design optimization.

Sign in / Sign up

Export Citation Format

Share Document