scholarly journals Scalable Gaussian Processes for Data-Driven Design using Big Data with Categorical Factors (IDETC2021-71570)

2021 ◽  
pp. 1-36
Author(s):  
Liwei Wang ◽  
Suraj Yerramilli ◽  
Akshay Iyer ◽  
Daniel Apley ◽  
Ping Zhu ◽  
...  
Keyword(s):  
Machine Learning ◽  
Gaussian Processes ◽  
Building Blocks ◽  
Variational Inference ◽  
Data Driven ◽  
Ternary Oxide ◽  
Latent Space ◽  
Multiple Materials ◽  
Qualitative Factors ◽  
Gp Model

Abstract Scientific and engineering problems often require the use of artificial intelligence to aid understanding and the search for promising designs. While Gaussian processes (GP) stand out as easy-to-use and interpretable learners, they have difficulties in accommodating big datasets, qualitative inputs, and multi-type responses obtained from different simulators, which has become a common challenge for data-driven design applications. In this paper, we propose a GP model that utilizes latent variables and functions obtained through variational inference to address the aforementioned challenges simultaneously. The method is built upon the latent variable Gaussian process (LVGP) model where qualitative factors are mapped into a continuous latent space to enable GP modeling of mixed-variable datasets. By extending variational inference to LVGP models, the large training dataset is replaced by a small set of inducing points to address the scalability issue. Output response vectors are represented by a linear combination of independent latent functions, forming a flexible kernel structure to handle multi-type responses. Comparative studies demonstrate that the proposed method scales well for large datasets, while outperforming state-of-the-art machine learning methods without requiring much hyperparameter tuning. In addition, an interpretable latent space is obtained to draw insights into the effect of qualitative factors, such as those associated with “building blocks” of architectures and element choices in metamaterial and materials design. Our approach is demonstrated for machine learning of ternary oxide materials and topology optimization of a multiscale compliant mechanism with aperiodic microstructures and multiple materials.

Data-Driven Design via Scalable Gaussian Processes for Multi-Response Big Data With Qualitative Factors

2021 ◽  
Author(s):  
Liwei Wang ◽  
Suraj Yerramilli ◽  
Akshay Iyer ◽  
Daniel Apley ◽  
Ping Zhu ◽  
...  
Keyword(s):  
Machine Learning ◽  
Gaussian Processes ◽  
Building Blocks ◽  
Variational Inference ◽  
Data Driven ◽  
Ternary Oxide ◽  
Latent Space ◽  
Multiple Materials ◽  
Qualitative Factors ◽  
Gp Model

Abstract Scientific and engineering problems often require an inexpensive surrogate model to aid understanding and the search for promising designs. While Gaussian processes (GP) stand out as easy-to-use and interpretable learners in surrogate modeling, they have difficulties in accommodating big datasets, qualitative inputs, and multi-type responses obtained from different simulators, which has become a common challenge for a growing number of data-driven design applications. In this paper, we propose a GP model that utilizes latent variables and functions obtained through variational inference to address the aforementioned challenges simultaneously. The method is built upon the latent variable Gaussian process (LVGP) model where qualitative factors are mapped into a continuous latent space to enable GP modeling of mixed-variable datasets. By extending variational inference to LVGP models, the large training dataset is replaced by a small set of inducing points to address the scalability issue. Output response vectors are represented by a linear combination of independent latent functions, forming a flexible kernel structure to handle multi-type responses. Comparative studies demonstrate that the proposed method scales well for large datasets with over 104 data points, while outperforming state-of-the-art machine learning methods without requiring much hyperparameter tuning. In addition, an interpretable latent space is obtained to draw insights into the effect of qualitative factors, such as those associated with “building blocks” of architectures and element choices in metamaterial and materials design. Our approach is demonstrated for machine learning of ternary oxide materials and topology optimization of a multiscale compliant mechanism with aperiodic microstructures and multiple materials.

Advances in Gaussian Processes for Earth Sciences: Physics-aware, interpretability and consistency

2020 ◽  
Author(s):  
Gustau Camps-Valls ◽  
Daniel Svendsen ◽  
Luca Martino ◽  
Adrian Pérez-Suay ◽  
Maria Piles ◽  
...  
Keyword(s):  
Machine Learning ◽  
Remote Sensing ◽  
Gaussian Processes ◽  
Earth Observation ◽  
Data Driven ◽  
Learning Models ◽  
Model Inversion ◽  
Gp Model ◽  
Machine Learning Models

<p>Earth observation from remote sensing satellites allows us to monitor the processes occurring on the land cover, water bodies and the atmosphere, as well as their interactions. In the last decade machine learning has impacted the field enormously due to the unprecedented data deluge and emergence of complex problems that need to be tackled (semi)automatically. One of the main problems is to perform estimation of bio-geo-physical parameters from remote sensing observations. In this model inversion setting, Gaussian processes (GPs) are one of the preferred choices for model inversion, emulation, gap filling and data assimilation. GPs do not only provide accurate predictions but also allow for feature ranking, deriving confidence intervals, and error propagation and uncertainty quantification in a principled Bayesian inference framework.</p><p>Here we introduce GPs for data analysis in general and to address the forward-inverse problem posed in remote sensing in particular. GPs are typically used for inverse modelling based on concurrent observations and in situ measurements only, or to invert model simulations. We often rely on forward radiative transfer model (RTM) encoding the well-understood physical relations to either perform model inversion with machine learning, or to replace the RTM model with machine learning models, a process known as emulation. We review four novel GP models that respect and learn the physics, and deploy useful machine learning models for remote sensing parameter retrieval and model emulation tasks. First, we will introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model for inversion. Second, we present a latent force model (LFM) for GP modelling that encodes ordinary differential equations to blend data and physical models of the system. The LFM performs multi-output regression, can cope with missing data in the time series, and provides explicit latent functions that allow system analysis, evaluation and understanding. Third, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Finally, we introduce a new GP model for data-driven regression that respects fundamental laws of physics via dependence-regularization, and provides consistency estimates. All models attain data-driven physics-aware modeling. Empirical evidence of performance of these models will be presented through illustrative examples of vegetation/land monitoring involving multispectral (Landsat, MODIS) and passive microwave (SMOS, SMAP) observations, as well as blending data with radiative transfer models, such as PROSAIL, SCOPE and MODTRAN.</p><p><br>References</p><p>"A Perspective on Gaussian Processes for Earth Observation". Camps-Valls et al. National Science Review 6 (4) :616-618, 2019</p><p>"Physics-aware Gaussian processes in remote sensing". Camps-Valls et al. Applied Soft Computing 68 :69-82, 2018</p><p>"A Survey on Gaussian Processes for Earth Observation Data Analysis: A Comprehensive Investigation". Camps-Valls et al. IEEE Geoscience and Remote Sensing Magazine 2016</p><p> </p>

scholarly journals Bayesian machine learning for financial modeling

2021 ◽  
Author(s):  
◽  
Rajbir Singh Nirwan
Keyword(s):  
Machine Learning ◽  
Order Statistics ◽  
Gaussian Processes ◽  
Asset Allocation ◽  
Latent Variable ◽  
Nonlinear Models ◽  
Rotational Symmetry ◽  
Variable Model ◽  
Advantages And Disadvantages ◽  
Latent Space

Machine Learning (ML) is so pervasive in our todays life that we don't even realise that, more often than expected, we are using systems based on it. It is also evolving faster than ever before. When deploying ML systems that make decisions on their own, we need to think about their ignorance of our uncertain world. The uncertainty might arise due to scarcity of the data, the bias of the data or even a mismatch between the real world and the ML-model. Given all these uncertainties, we need to think about how to build systems that are not totally ignorant thereof. Bayesian ML can to some extent deal with these problems. The specification of the model using probabilities provides a convenient way to quantify uncertainties, which can then be included in the decision making process. In this thesis, we introduce the Bayesian ansatz to modeling and apply Bayesian ML models in finance and economics. Especially, we will dig deeper into Gaussian processes (GP) and Gaussian process latent variable model (GPLVM). Applied to the returns of several assets, GPLVM provides the covariance structure and also a latent space embedding thereof. Several financial applications can be build upon the output of the GPLVM. To demonstrate this, we build an automated asset allocation system, a predictor for missing asset prices and identify other structure in financial data. It turns out that the GPLVM exhibits a rotational symmetry in the latent space, which makes it harder to fit. Our second publication reports, how to deal with that symmetry. We propose another parameterization of the model using Householder transformations, by which the symmetry is broken. Bayesian models are changed by reparameterization, if the prior is not changed accordingly. We provide the correct prior distribution of the new parameters, such that the model, i.e. the data density, is not changed under the reparameterization. After applying the reparametrization on Bayesian PCA, we show that the symmetry of nonlinear models can also be broken in the same way. In our last project, we propose a new method for matching quantile observations, which uses order statistics. The use of order statistics as the likelihood, instead of a Gaussian likelihood, has several advantages. We compare these two models and highlight their advantages and disadvantages. To demonstrate our method, we fit quantiled salary data of several European countries. Given several candidate models for the fit, our method also provides a metric to choose the best option. We hope that this thesis illustrates some benefits of Bayesian modeling (especially Gaussian processes) in finance and economics and its usage when uncertainties are to be quantified.

Physics Informed Machine Learning of Rainfall-Runoff Processes

2020 ◽  
Author(s):  
Vladan Babovic ◽  
Jayashree Chadalawada ◽  
Herath Mudiyanselage Viraj Vidura Herath
Keyword(s):  
Machine Learning ◽  
Model Building ◽  
Hydrological Model ◽  
Building Blocks ◽  
Data Driven ◽  
Machine Learning Techniques ◽  
Rainfall Runoff ◽  
Learning Techniques ◽  
Highly Nonlinear ◽  
Wide Range

<p>Modelling of rainfall-runoff phenomenon continues to be a challenging task at hand of hydrologists as the underlying processes are highly nonlinear, dynamic and interdependent. Numerous modelling strategies like empirical, conceptual, physically based, data driven, are used to develop rainfall-runoff models as no model type can be considered to be universally pertinent for a wide range of problems. Latest literature review emphasizes that the crucial step of hydrological model selection is often subjective and is based on legacy. As the research outcome depends on model choice, there is a necessity to automate the process of model evolution, evaluation and selection based on research objectives, temporal and spatial characteristics of available data and catchment properties. Therefore, this study proposes a novel automated model building algorithm relying on machine learning technique Genetic Programming (GP).</p><p>State of art GP applications in rainfall-runoff modelling as yet used the algorithm as a short-term forecasting tool which produces an expected future time series very much alike to neural networks application. Such simplistic applications of data driven black-box machine learning techniques may lead to development of accurate yet meaningless models which do not satisfy basic hydrological insights and may have severe difficulties with interpretation. Concurrently, it should be admitted that there is a vast amount of knowledge and understanding of physical processes that should not just be thrown away. Thus, we strongly believe that the most suitable way forward is to couple the already existing body of knowledge with machine learning techniques in a guided manner to enhance the meaningfulness and interpretability of the induced models.</p><p>In this suggested algorithm the domain knowledge is introduced through the incorporation of process knowledge by adding model building blocks from prevailing rainfall-runoff modelling frameworks into the GP function set. Presently, the function set library consists with Sugawara TANK model functions, generic components of two flexible rainfall-runoff modelling frameworks (FUSE and SUPERFLEX) and model equations of 46 existing hydrological models (MARRMoT). Nevertheless, perhaps more importantly, the algorithm is readily integratable with any other internal coherence building blocks. This approach contrasts from rest of machine learning applications in rainfall-runoff modelling as it not only produces the runoff predictions but develops a physically meaningful hydrological model which helps the hydrologist to better understand the catchment dynamics. The proposed algorithm considers the model space and automatically identifies the appropriate model configurations for a catchment of interest by optimizing user-defined learning objectives in a multi-objective optimization framework. The model induction capabilities of the proposed algorithm have been evaluated on the Blackwater River basin, Alabama, United States. The model configurations evolved through the model-building algorithm are compatible with the fieldwork investigations and previously reported research findings.</p>

Efficient Prediction of Structural and Electronic Properties of Hybrid 2D Materials Using DFT and Machine Learning

2018 ◽  
Author(s):  
Sherif Tawfik ◽  
Olexandr Isayev ◽  
Catherine Stampfl ◽  
Joseph Shapter ◽  
David Winkler ◽  
...  
Keyword(s):  
Machine Learning ◽  
Band Gap ◽  
Density Functional ◽  
2D Materials ◽  
Van Der Waals ◽  
Building Blocks ◽  
Machine Learning Techniques ◽  
Interlayer Distance ◽  
Computational Screening ◽  
Wide Range

Materials constructed from different van der Waals two-dimensional (2D) heterostructures offer a wide range of benefits, but these systems have been little studied because of their experimental and computational complextiy, and because of the very large number of possible combinations of 2D building blocks. The simulation of the interface between two different 2D materials is computationally challenging due to the lattice mismatch problem, which sometimes necessitates the creation of very large simulation cells for performing density-functional theory (DFT) calculations. Here we use a combination of DFT, linear regression and machine learning techniques in order to rapidly determine the interlayer distance between two different 2D heterostructures that are stacked in a bilayer heterostructure, as well as the band gap of the bilayer. Our work provides an excellent proof of concept by quickly and accurately predicting a structural property (the interlayer distance) and an electronic property (the band gap) for a large number of hybrid 2D materials. This work paves the way for rapid computational screening of the vast parameter space of van der Waals heterostructures to identify new hybrid materials with useful and interesting properties.

scholarly journals Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems

2021 ◽  
Author(s):  
Ekaterina Kochmar ◽  
Dung Do Vu ◽  
Robert Belfer ◽  
Varun Gupta ◽  
Iulian Vlad Serban ◽  
...  
Keyword(s):  
Machine Learning ◽  
Student Performance ◽  
Language Processing ◽  
Intelligent Tutoring Systems ◽  
Large Scale ◽  
Intelligent Tutoring ◽  
Performance Outcomes ◽  
Data Driven ◽  
Personalized Feedback ◽  
Tutoring Systems

AbstractIntelligent tutoring systems (ITS) have been shown to be highly effective at promoting learning as compared to other computer-based instructional approaches. However, many ITS rely heavily on expert design and hand-crafted rules. This makes them difficult to build and transfer across domains and limits their potential efficacy. In this paper, we investigate how feedback in a large-scale ITS can be automatically generated in a data-driven way, and more specifically how personalization of feedback can lead to improvements in student performance outcomes. First, in this paper we propose a machine learning approach to generate personalized feedback in an automated way, which takes individual needs of students into account, while alleviating the need of expert intervention and design of hand-crafted rules. We leverage state-of-the-art machine learning and natural language processing techniques to provide students with personalized feedback using hints and Wikipedia-based explanations. Second, we demonstrate that personalized feedback leads to improved success rates at solving exercises in practice: our personalized feedback model is used in , a large-scale dialogue-based ITS with around 20,000 students launched in 2019. We present the results of experiments with students and show that the automated, data-driven, personalized feedback leads to a significant overall improvement of 22.95% in student performance outcomes and substantial improvements in the subjective evaluation of the feedback.

scholarly journals A Data-Driven Method for the Temporal Estimation of Soil Water Potential and Its Application for Shallow Landslides Prediction

Water ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1208
Author(s):  
Massimiliano Bordoni ◽  
Fabrizio Inzaghi ◽  
Valerio Vivaldi ◽  
Roberto Valentino ◽  
Marco Bittelli ◽  
...  
Keyword(s):  
Machine Learning ◽  
Water Potential ◽  
Soil Water ◽  
Temporal Trends ◽  
Soil Water Potential ◽  
Shallow Landslides ◽  
Water Dynamics ◽  
Data Driven ◽  
Machine Learning Techniques ◽  
Physically Based

Soil water potential is a key factor to study water dynamics in soil and for estimating the occurrence of natural hazards, as landslides. This parameter can be measured in field or estimated through physically-based models, limited by the availability of effective input soil properties and preliminary calibrations. Data-driven models, based on machine learning techniques, could overcome these gaps. The aim of this paper is then to develop an innovative machine learning methodology to assess soil water potential trends and to implement them in models to predict shallow landslides. Monitoring data since 2012 from test-sites slopes in Oltrepò Pavese (northern Italy) were used to build the models. Within the tested techniques, Random Forest models allowed an outstanding reconstruction of measured soil water potential temporal trends. Each model is sensitive to meteorological and hydrological characteristics according to soil depths and features. Reliability of the proposed models was confirmed by correct estimation of days when shallow landslides were triggered in the study areas in December 2020, after implementing the modeled trends on a slope stability model, and by the correct choice of physically-based rainfall thresholds. These results confirm the potential application of the developed methodology to estimate hydrological scenarios that could be used for decision-making purposes.

Sign in / Sign up

Export Citation Format

Share Document