Scalable Fully Bayesian Gaussian Process Modeling and Calibration with Adaptive Sequential Monte Carlo for Industrial Applications

2021 ◽  
pp. 1-11
Author(s):  
Piyush Pandita ◽  
Panagiotis Tsilifis ◽  
Sayan Ghosh ◽  
Liping Wang
Keyword(s):  
Monte Carlo ◽  
Big Data ◽  
Gaussian Process ◽  
Industry 4.0 ◽  
Large Scale ◽  
Sequential Monte Carlo ◽  
Industrial Applications ◽  
Computational Time ◽  
Mcmc Sampling ◽  
Gp Model

Abstract Gaussian Process (GP) regression or kriging has been extensively applied in the engineering literature for the purposes of building a cheap-to-evaluate surrogate, within the contexts of multi-fidelity modeling, model calibration and design optimization. With the ongoing automation of manufacturing and industrial practices as a part of Industry 4.0, there has been greater need for advancing GP regression techniques to handle challenges such as high input dimensionality, data paucity or big data problems, these consist primarily of proposing efficient design of experiments, optimal data acquisition strategies, and other mathematical tricks. In this work, our attention is focused on the challenges of efficiently training a GP model, which, to the authors opinion, has attracted very little attention and is to-date, poorly addressed. The performance of widely used training approaches such as maximum likelihood estimation and Markov Chain Monte Carlo (MCMC) sampling can deteriorate significantly in high dimensional and big data problems and can lead to cost deficient implementations of critical importance to many industrial applications. Here, we compare an Adaptive Sequential Monte Carlo (ASMC) sampling algorithm to classic MCMC sampling strategies and we demonstrate the effectiveness of our implementation on several mathematical problems and challenging industry applications of varying complexity. The computational time savings of our ASMC approach manifest in large-scale problems helping us to push the boundary of GP regression applicability and scalability in various domain of Industry 4.0, including but not limited to design automation, design engineering, predictive maintenance, and supply chain manufacturing.

scholarly journals Towards Scalable Gaussian Process Modeling

2019 ◽  
Author(s):  
Piyush Pandita ◽  
Jesper Kristensen ◽  
Liping Wang
Keyword(s):  
Monte Carlo ◽  
Gaussian Process ◽  
Objective Function ◽  
Large Scale ◽  
Sequential Monte Carlo ◽  
Computational Time ◽  
Medium Size ◽  
Modeling Framework ◽  
Industry Applications ◽  
Mathematical Problems

Abstract Numerous engineering problems of interest to the industry are often characterized by expensive black-box objective function evaluations. These objective functions could be physical experiments or computer simulations. Obtaining a comprehensive idea of the problem and/or performing subsequent optimizations generally requires hundreds of thousands of evaluations of the objective function which is most often a practically unachievable task. Gaussian Process (GP) surrogate modeling replaces the expensive function with a cheap-to-evaluate data-driven probabilistic model. While the GP does not assume a functional form of the problem, it is defined by a set of parameters, called hyper-parameters, that need to be learned from the data. The hyperparameters define the characteristics of the objective function, such as smoothness, magnitude, periodicity, etc. Accurately estimating these hyperparameters is a key ingredient in developing a reliable and generalizable surrogate model. Markov chain Monte Carlo (MCMC) is a ubiquitously used Bayesian method to estimate these hyperparameters. At GEs Global Research Center, a customized industry-strength Bayesian hybrid modeling framework utilizing the GP, called GEBHM, has been employed and validated over many years. GEBHM is very effective on problems of small and medium size, typically less than 1000 training points. However, the GP does not scale well in time with a growing dataset and problem dimensionality which can be a major impediment in such problems. For some challenging industry applications, the predictive capability of the GP is required but each second during the training of the GP costs thousands of dollars. In this work, we apply a scalable MCMC-based methodology enabling the modeling of large-scale industry problems. Towards this, we extend and implement in GEBHM an Adaptive Sequential Monte Carlo (ASMC) methodology for training the GP. This implementation saves computational time (especially for large-scale problems) while not sacrificing predictability over the current MCMC implementation. We demonstrate the effectiveness and accuracy of GEBHM with ASMC on four mathematical problems and on two challenging industry applications of varying complexity.

scholarly journals A Systematic Analysis of Big Image Data Methodologies in Various Applications

2020 ◽  
Vol 9 (5) ◽  
pp. 483-487
Keyword(s):  
Big Data ◽  
Deep Learning ◽  
Large Scale ◽  
Image Data ◽  
Computational Time ◽  
Process Data ◽  
Systematic Analysis ◽  
Large Scale Data ◽  
Learning Techniques ◽  
Effective Performance

Big data is large-scale data collected for knowledge discovery, it has been widely used in various applications. Big data often has image data from the various applications and requires effective technique to process data. In this paper, survey has been done in the big image data researches to analysis the effective performance of the methods. Deep learning techniques provides the effective performance compared to other methods included wavelet based methods. The deep learning techniques has the problem of requiring more computational time, and this can be overcome by lightweight methods.

scholarly journals Sequential Monte Carlo Methods to Train Neural Network Models

2000 ◽  
Vol 12 (4) ◽  
pp. 955-993 ◽  
Author(s):  
J. F. G. de Freitas ◽  
M. Niranjan ◽  
A. H. Gee ◽  
A. Doucet
Keyword(s):  
Monte Carlo ◽  
Sequential Monte Carlo ◽  
Probability Distributions ◽  
Network Models ◽  
Computational Time ◽  
Optimization Strategy ◽  
Neural Network Models ◽  
Monte Carlo Techniques ◽  
Monte Carlo Algorithms ◽  
Importance Resampling

We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent/sampling importance resampling algorithm (HySIR). In terms of computational time and accuracy, the hybrid SIR is a clear improvement over conventional sequential Monte Carlo techniques. The new algorithm may be viewed as a global optimization strategy that allows us to learn the probability distributions of the network weights and outputs in a sequential framework. It is well suited to applications involving on-line, nonlinear, and nongaussian signal processing. We show how the new algorithm outperforms extended Kalman filter training on several problems. In particular, we address the problem of pricing option contracts, traded in financial markets. In this context, we are able to estimate the one-step-ahead probability density functions of the options prices.

scholarly journals Improving the efficiency of Monte Carlo Bayesian calibration of hydrologic models via model pre-emption

2015 ◽  
Vol 17 (5) ◽  
pp. 763-770 ◽  
Author(s):  
Mahyar Shafii ◽  
Bryan Tolson ◽  
L. Shawn Matott
Keyword(s):  
Monte Carlo ◽  
Sequential Monte Carlo ◽  
Hydrological Modelling ◽  
High Likelihood ◽  
Rainfall Runoff ◽  
Hydrologic Models ◽  
Bayesian Calibration ◽  
Computational Costs ◽  
Mcmc Sampling ◽  
Simulation Time

Bayesian inference via Markov Chain Monte Carlo (MCMC) sampling and sequential Monte Carlo (SMC) sampling are popular methods for uncertainty analysis in hydrological modelling. However, application of these methodologies can incur significant computational costs. This study investigated using model pre-emption for improving the computational efficiency of MCMC and SMC samplers in the context of hydrological modelling. The proposed pre-emption strategy facilitates early termination of low-likelihood simulations and results in reduction of unnecessary simulation time steps. The proposed approach is incorporated into two samplers and applied to the calibration of three rainfall–runoff models. Results show that overall pre-emption savings range from 5 to 21%. Furthermore, results indicate that pre-emption savings are greatest during the pre-convergence ‘burn-in’ period (i.e., between 8 and 39%) and decrease as the algorithms converge towards high likelihood regions of parameter space. The observed savings are achieved with absolutely no change in the posterior set of parameters.

scholarly journals A Cognitive Adopted Framework for IoT Big-Data Management and Knowledge Discovery Prospective

2015 ◽  
Vol 2015 ◽  
pp. 1-12 ◽  
Author(s):  
Nilamadhab Mishra ◽  
Chung-Chih Lin ◽  
Hsien-Tsung Chang
Keyword(s):  
Big Data ◽  
Data Management ◽  
Knowledge Discovery ◽  
Large Scale ◽  
Industrial Applications ◽  
Sensor Technology ◽  
Industrial Automation ◽  
Data Framework ◽  
Industrial Internet ◽  
Knowledge Exploration

In future IoT big-data management and knowledge discovery for large scale industrial automation application, the importance of industrial internet is increasing day by day. Several diversified technologies such as IoT (Internet of Things), computational intelligence, machine type communication, big-data, and sensor technology can be incorporated together to improve the data management and knowledge discovery efficiency of large scale automation applications. So in this work, we need to propose a Cognitive Oriented IoT Big-data Framework (COIB-framework) along with implementation architecture, IoT big-data layering architecture, and data organization and knowledge exploration subsystem for effective data management and knowledge discovery that is well-suited with the large scale industrial automation applications. The discussion and analysis show that the proposed framework and architectures create a reasonable solution in implementing IoT big-data based smart industrial applications.

scholarly journals Industry 4.0 Technologies for the Manufacturing and Distribution of COVID-19 Vaccines

2022 ◽  
Vol 13 ◽  
pp. 215013192110686
Author(s):  
Azza Sarfraz ◽  
Zouina Sarfraz ◽  
Muzna Sarfraz ◽  
Aminah Abdul Razzack ◽  
Shehar Bano ◽  
...  
Keyword(s):  
Big Data ◽  
Value Chain ◽  
Disease Transmission ◽  
Industry 4.0 ◽  
Data Analytics ◽  
Large Scale ◽  
Process Development ◽  
Big Data Analytics ◽  
Middle Income ◽  
Resource Limited

Background The evolutionary stages of manufacturing have led us to conceptualize the use of Industry 4.0 for COVID-19 (coronavirus disease 2019), powered by Industry 4.0 technologies. Using applications of integrated process optimizations reliant on digitized data, we propose novel intelligent networks along the vaccine value chain. Vaccine 4.0 may enable maintenance processes, streamline logistics, and enable optimal production of COVID-19 vaccines. Vaccine 4.0 Framework The challenge in applying Vaccine 4.0 includes the requirement of large-scale technologies for digitally transforming manufacturing, producing, rolling-out, and distributing vaccines. With our framework, Vaccine 4.0 analytics will target process performance, process development, process stability, compliance, quality assessment, and optimized maintenance. The benefits of digitization during and post the COVID-19 pandemic include first, the continual assurance of process control, and second, the efficacy of big-data analytics in streamlining set parameter limits. Digitization including big data-analytics may potentially improve the quality of large-scale vaccine production, profitability, and manufacturing processes. The path to Vaccine 4.0 will enhance vaccine quality, improve efficacy, and compliance with data-regulated requirements. Discussion Fiscal and logistical barriers are prevalent across resource-limited countries worldwide. The Vaccine 4.0 framework accounts for expected barriers of manufacturing and equitably distributing COVID-19 vaccines. With amalgamating big data analytics and biometrics, we enable the identification of vulnerable populations who are at higher risk of disease transmission. Artificial intelligence powered sensors and robotics support thermostable vaccine distribution in limited capacity regions, globally. Biosensors isolate COVID-19 vaccinations with low or limited efficacy. Finally, Vaccine 4.0 blockchain systems address low- and middle-income countries with limited distribution capacities. Conclusion Vaccine 4.0 is a viable framework to optimize manufacturing of vaccines during and post the COVID-19 pandemic.

Sign in / Sign up

Export Citation Format

Share Document