Bayesian Variational Seismic Tomography using Normalizing Flows

2021 ◽  
Author(s):  
Xuebin Zhao ◽  
Andrew Curtis ◽  
Xin Zhang
Keyword(s):  
Monte Carlo ◽  
Seismic Tomography ◽  
Computational Cost ◽  
Large Data ◽  
Optimization Methods ◽  
Monte Carlo Sampling ◽  
New Method ◽  
Posterior Probability Distribution ◽  
Computationally Expensive ◽  
A Chain

<p>Seismic travel time tomography is used widely to image the Earth's interior structure and to infer subsurface properties. Tomography is an inverse problem, and computationally expensive nonlinear inverse methods are often deployed in order to understand uncertainties in the tomographic results. Monte Carlo sampling methods estimate the posterior probability distribution which describes the solution to Bayesian tomographic problems, but they are computationally expensive and often intractable for high dimensional model spaces and large data sets due to the curse of dimensionality. We therefore introduce a new method of variational inference to solve Bayesian seismic tomography problems using optimization methods, while still providing fully nonlinear, probabilistic results. The new method, known as normalizing flows, warps a simple and known distribution (for example a Uniform or Gaussian distribution) into an optimal approximation to the posterior distribution through a chain of invertible transforms. These transforms are selected from a library of suitable functions, some of which invoke neural networks internally. We test the method using both synthetic and field data. The results show that normalizing flows can produce similar mean and uncertainty maps to those obtained from both Monte Carlo and another variational method (Stein varational gradient descent), at significantly decreased computational cost. In our tomographic tests, normalizing flows improves both accuracy and efficiency, producing maps of UK surface wave speeds and their uncertainties at the finest resolution and the lowest computational cost to-date, allowing results to be interrogated efficiently and quantitatively for subsurface structure.</p>

scholarly journals Seismic Tomography Using Variational Inference Methods

2020 ◽  
Author(s):  
Xin Zhang ◽  
Andrew Curtis
Keyword(s):  
Monte Carlo ◽  
Probability Distribution ◽  
Seismic Tomography ◽  
Posterior Probability ◽  
Sampling Methods ◽  
Probability Distributions ◽  
Monte Carlo Sampling ◽  
Variational Inference ◽  
Posterior Probability Distribution ◽  
Inference Methods

<p><span>In a variety of geoscientific applications we require maps of subsurface properties together with the corresponding maps of uncertainties to assess their reliability. Seismic tomography is a method that is widely used to generate those maps. Since tomography is significantly nonlinear, Monte Carlo sampling methods are often used for this purpose, but they are generally computationally intractable for large data sets and high-dimensionality parameter spaces. To extend uncertainty analysis to larger systems, we introduce variational inference methods to conduct seismic tomography. In contrast to Monte Carlo sampling, variational methods solve the Bayesian inference problem as an optimization problem yet still provide fully nonlinear, probabilistic results. This is achieved by minimizing the Kullback-Leibler (KL) divergence between approximate and target probability distributions within a predefined family of probability distributions.</span></p><p><span>We introduce two variational inference methods: automatic differential variational inference (ADVI) and Stein variational gradient descent (SVGD). In ADVI a Gaussian probability distribution is assumed and optimized to approximate the posterior probability distribution. In SVGD a smooth transform is iteratively applied to an initial probability distribution to obtain an approximation to the posterior probability distribution. At each iteration the transform is determined by seeking the steepest descent direction that minimizes the KL-divergence. </span></p><p><span>We apply the two variational inference methods to 2D travel time tomography using both synthetic and real data, and compare the results to those obtained from two different Monte Carlo sampling methods: Metropolis-Hastings Markov chain Monte Carlo (MH-McMC) and reversible jump Markov chain Monte Carlo (rj-McMC). The results show that ADVI provides a biased approximation because of its Gaussian approximation, whereas SVGD produces more accurate approximations to the results of MH-McMC. In comparison rj-McMC produces smoother mean velocity models and lower standard deviations because the parameterization used in rj-McMC (Voronoi cells) imposes prior restrictions on the pixelated form of models: all pixels within each Voronoi cell have identical velocities. This suggests that the results of rj-McMC need to be interpreted in the light of the specific prior information imposed by the parameterization. Both variational methods estimate the posterior distribution at significantly lower computational cost, provided that gradients of parameters with respect to data can be calculated efficiently. We therefore expect that the methods can be applied fruitfully to many other types of geophysical inverse problems.</span></p>

Toward a full 4D seismic tomography: a case study of an active mine

2021 ◽  
Author(s):  
Nicola Piana Agostinetti ◽  
Christina Dahnér-Lindkvist ◽  
Savka Dineva
Keyword(s):  
Monte Carlo ◽  
Seismic Tomography ◽  
Water Injection ◽  
Elastic Field ◽  
Monte Carlo Sampling ◽  
P Wave ◽  
Travel Times ◽  
S Wave ◽  
Posterior Probability Distribution ◽  
Space And Time

<p>Rock elasticity in the subsurface can change in response to natural phenomena (e.g. massive precipitation, magmatic processes) and human activities (e.g. water injection in geothermal wells, ore-body exploitation). However, understanding and monitoring the evolution of physical properties of the crust is a challenging due to the limited possibility of reaching such depths and making direct measurements of the state of the rocks. Indirect measurements, like seismic tomography, can give some insights, but are generally biased by the un-even distribution (in space and time) of the information collected from seismic observations (travel-times and/or waveforms). Here we apply a Bayesian approach to overcome such limitations, so that data uncertainties and data distribution are fully accounted in the reconstruction of the posterior probability distribution of the rock elasticity  We compute a full 4D local earthquake tomography based on trans-dimensional Markov chain Monte Carlo sampling of 4D elastic models, where the resolution in space and time is fully data-driven. To test our workflow, we make use of a “controlled laboratory”: we record seismic data during one month of mining activities across a 800x700x600 m volume of Kiruna mine (LKAB, Sweden). During such period, we obtain about 260 000 P-wave and 240 000 S-wave travel-times coming from about 36000  seismic events. We operate a preliminary selection of the well-located events, using a Monte Carlo search. Arrival-times of about 19 000 best-located events (location errors less than 20m) are used as input to the tomography workflow. Preliminary results indicate that: (1) short-term (few hours) evolutions of the elastic field are mainly driven by seismic activation, i.e. the occurrence of a seismic swarm, close to the mine ore-passes. Such phenomena partially mask the effects of explosions; (2) long-term (2-3 days) evolutions of the elastic field closely match the local measurements of the stress field at a colocated stress cell. </p>

Asteroid masses obtained with INPOP planetary ephemerides

2019 ◽  
Vol 492 (1) ◽  
pp. 589-602 ◽  
Author(s):  
A Fienga ◽  
C Avdellidou ◽  
J Hanuš
Keyword(s):  
Monte Carlo ◽  
Monte Carlo Sampling ◽  
New Method ◽  
Navigation Data ◽  
The Earth ◽  
Better Than ◽  
Juno Mission

ABSTRACT In this paper, we present masses of 103 asteroids deduced from their perturbations on the orbits of the inner planets, in particular Mars and the Earth. These determinations and the INPOP19a planetary ephemerides are improved by the recent Mars orbiter navigation data and the updated orbit of Jupiter based on the Juno mission data. More realistic mass estimates are computed by a new method based on random Monte Carlo sampling that uses up-to-date knowledge of asteroid bulk densities. We provide masses with uncertainties better than 33${{\ \rm per\ cent}}$ for 103 asteroids. Deduced bulk densities are consistent with those observed within the main spectroscopic complexes.

scholarly journals Sample-efficient Optimization Using Neural Networks

2021 ◽  
Author(s):  
◽  
Mashall Aryan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Monte Carlo ◽  
Gaussian Process ◽  
Optimization Methods ◽  
Optimization Method ◽  
Monte Carlo Sampling ◽  
Bayesian Optimization ◽  
Expected Improvement ◽  
Bayesian Neural Network

<p>The solution to many science and engineering problems includes identifying the minimum or maximum of an unknown continuous function whose evaluation inflicts non-negligible costs in terms of resources such as money, time, human attention or computational processing. In such a case, the choice of new points to evaluate is critical. A successful approach has been to choose these points by considering a distribution over plausible surfaces, conditioned on all previous points and their evaluations. In this sequential bi-step strategy, also known as Bayesian Optimization, first a prior is defined over possible functions and updated to a posterior in the light of available observations. Then using this posterior, namely the surrogate model, an infill criterion is formed and utilized to find the next location to sample from. By far the most common prior distribution and infill criterion are Gaussian Process and Expected Improvement, respectively.    The popularity of Gaussian Processes in Bayesian optimization is partially due to their ability to represent the posterior in closed form. Nevertheless, the Gaussian Process is afflicted with several shortcomings that directly affect its performance. For example, inference scales poorly with the amount of data, numerical stability degrades with the number of data points, and strong assumptions about the observation model are required, which might not be consistent with reality. These drawbacks encourage us to seek better alternatives. This thesis studies the application of Neural Networks to enhance Bayesian Optimization. It proposes several Bayesian optimization methods that use neural networks either as their surrogates or in the infill criterion.    This thesis introduces a novel Bayesian Optimization method in which Bayesian Neural Networks are used as a surrogate. This has reduced the computational complexity of inference in surrogate from cubic (on the number of observation) in GP to linear. Different variations of Bayesian Neural Networks (BNN) are put into practice and inferred using a Monte Carlo sampling. The results show that Monte Carlo Bayesian Neural Network surrogate could performed better than, or at least comparably to the Gaussian Process-based Bayesian optimization methods on a set of benchmark problems.  This work develops a fast Bayesian Optimization method with an efficient surrogate building process. This new Bayesian Optimization algorithm utilizes Bayesian Random-Vector Functional Link Networks as surrogate. In this family of models the inference is only performed on a small subset of the entire model parameters and the rest are randomly drawn from a prior. The proposed methods are tested on a set of benchmark continuous functions and hyperparameter optimization problems and the results show the proposed methods are competitive with state-of-the-art Bayesian Optimization methods.  This study proposes a novel Neural network-based infill criterion. In this method locations to sample from are found by minimizing the joint conditional likelihood of the new point and parameters of a neural network. The results show that in Bayesian Optimization methods with Bayesian Neural Network surrogates, this new infill criterion outperforms the expected improvement.   Finally, this thesis presents order-preserving generative models and uses it in a variational Bayesian context to infer Implicit Variational Bayesian Neural Network (IVBNN) surrogates for a new Bayesian Optimization. This new inference mechanism is more efficient and scalable than Monte Carlo sampling. The results show that IVBNN could outperform Monte Carlo BNN in Bayesian optimization of hyperparameters of machine learning models.</p>

scholarly journals MCPeSe: Monte Carlo penalty selection for graphical lasso

2020 ◽  
Author(s):  
Markku Kuismin ◽  
Mikko J Sillanpää
Keyword(s):  
Monte Carlo ◽  
Regulatory Networks ◽  
Regularization Parameter ◽  
Computational Cost ◽  
Selection Criterion ◽  
Supplementary Information ◽  
Tuning Parameter ◽  
Posterior Probability Distribution ◽  
Graphical Lasso ◽  
Selection For

Abstract Motivation Graphical lasso (Glasso) is a widely used tool for identifying gene regulatory networks in systems biology. However, its computational efficiency depends on the choice of regularization parameter (tuning parameter), and selecting this parameter can be highly time consuming. Although fully Bayesian implementations of Glasso alleviate this problem somewhat by specifying a priori distribution for the parameter, these approaches lack the scalability of their frequentist counterparts. Results Here, we present a new Monte Carlo Penalty Selection method (MCPeSe), a computationally efficient approach to regularization parameter selection for Glasso. MCPeSe combines the scalability and low computational cost of the frequentist Glasso with the ability to automatically choose the regularization by Bayesian Glasso modeling. MCPeSe provides a state-of-the-art ‘tuning-free’ model selection criterion for Glasso and allows exploration of the posterior probability distribution of the tuning parameter. Availability and implementation R source code of MCPeSe, a step by step example showing how to apply MCPeSe and a collection of scripts used to prepare the material in this article are publicly available at GitHub under GPL (https://github.com/markkukuismin/MCPeSe/). Supplementary information Supplementary data are available at Bioinformatics online.

Asteroid masses obtained with INPOP planetary ephemerides

2020 ◽  
Author(s):  
Agnes Fienga ◽  
Chrysa Avdellidou ◽  
Josef Hanus
Keyword(s):  
Monte Carlo ◽  
Monte Carlo Sampling ◽  
New Method ◽  
Navigation Data ◽  
The Earth ◽  
Better Than ◽  
Juno Mission

&lt;p&gt;We present here masses of 103 asteroids deduced from their perturbations on the&lt;br /&gt;orbits of the inner planets, in particular Mars and the Earth. These determinations and the&lt;br /&gt;INPOP19a planetary ephemerides are improved by the recent Mars orbiter navigation data&lt;br /&gt;and the updated orbit of Jupiter based on the Juno mission data. More realistic mass estimates&lt;br /&gt;are computed by a new method based on random Monte-Carlo sampling that uses up-to-date&lt;br /&gt;knowledge of asteroid bulk densities. We provide masses with uncertainties better than 33%&lt;br /&gt;for 103 asteroids. Deduced bulk densities are consistent with those observed within the main&lt;br /&gt;spectroscopic complexes.&lt;/p&gt;

A New Reliability Prediction Method Based on Physics of Failure Method for Product Design and Manufacture

2012 ◽  
Vol 548 ◽  
pp. 521-526 ◽  
Author(s):  
Xing Hao Wang ◽  
Jiang Shao ◽  
Xiao Yu Liu
Keyword(s):  
Monte Carlo ◽  
Monte Carlo Calculation ◽  
Computational Cost ◽  
Prediction Method ◽  
New Method ◽  
Reliability Prediction ◽  
Time To Failure ◽  
Physics Of Failure ◽  
First Order ◽  
Environment Stress

Different from the reliability prediction method on handbook, the reliability prediction method based on Physics of Failure (PoF) model takes failure mechanism as theoretical basis, and combines the design in-formation with the environment stress of the product to predict the time to failure. When the uncertain of the parameters is considered to predict the reliability, Monte-Carlo calculation method is always used here. How-ever, the Monte-Carlo method needs large computational cost, especially for large and complicated electronic systems. A new reliability prediction method which combines the first order reliability with the reliability pre-diction method based on PoF model was proposed. The new method utilized the first order method to calculate the position of design point and reliability index, thus Monte-Carlo calculation process was avoided. Example calculation results showed that the new method improves the prediction efficiency without decreasing the accuracy of reliability, thus it is feasible for reliability prediction of electronic product in engineering.

scholarly journals From offshore to onshore probabilistic tsunami hazard assessment via efficient Monte-Carlo sampling

2021 ◽  
Author(s):  
Gareth Davies ◽  
Rikki Weber ◽  
Kaya Wilson ◽  
Phil Cummins
Keyword(s):  
Monte Carlo ◽  
High Resolution ◽  
Large Scale ◽  
Computational Cost ◽  
Deep Ocean ◽  
Analytical Techniques ◽  
Monte Carlo Sampling ◽  
Tsunami Hazard ◽  
Practical Applications ◽  
Inundation Simulation

Offshore Probabilistic Tsunami Hazard Assessments (offshore PTHAs) provide large-scale analyses of earthquake-tsunami frequencies and uncertainties in the deep ocean, but do not provide high-resolution onshore tsunami hazard information as required for many risk-management applications. To understand the implications of an offshore PTHA for the onshore hazard at any site, in principle the tsunami inundation should be simulated locally for every scenario in the offshore PTHA. In practice this is rarely feasible due to the computational expense of inundation models, and the large number of scenarios in offshore PTHAs. Monte-Carlo methods offer a practical and rigorous alternative for approximating the onshore hazard, using a random subset of scenarios. The resulting Monte-Carlo errors can be quantified and controlled, enabling high-resolution onshore PTHAs to be implemented at a fraction of the computational cost. This study develops novel Monte-Carlo sampling approaches for offshore-to-onshore PTHA. Modelled offshore PTHA wave heights are used to preferentially sample scenarios that have large offshore waves near an onshore site of interest. By appropriately weighting the scenarios, the Monte-Carlo errors are reduced without introducing any bias. The techniques are applied to a high-resolution onshore PTHA for the island of Tongatapu in Tonga. In this region, the new approaches lead to efficiency improvements equivalent to using 4-18 times more random scenarios, as compared with stratified-sampling by magnitude, which is commonly used for onshore PTHA. The greatest efficiency improvements are for rare, large tsunamis, and for calculations that represent epistemic uncertainties in the tsunami hazard. To facilitate the control of Monte-Carlo errors in practical applications, this study also provides analytical techniques for estimating the errors both before and after inundation simulations are conducted. Before inundation simulation, this enables a proposed Monte-Carlo sampling scheme to be checked, and potentially improved, at minimal computational cost. After inundation simulation, it enables the remaining Monte-Carlo errors to be quantified at onshore sites, without additional inundation simulations. In combination these techniques enable offshore PTHAs to be rigorously transformed into onshore PTHAs, with full characterisation of epistemic uncertainties, while controlling Monte-Carlo errors.

scholarly journals Joining forces of Bayesian and frequentist methodology: a study for inference in the presence of non-identifiability

2013 ◽  
Vol 371 (1984) ◽  
pp. 20110544 ◽  
Author(s):  
Andreas Raue ◽  
Clemens Kreutz ◽  
Fabian Joachim Theis ◽  
Jens Timmer
Keyword(s):  
Experimental Data ◽  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Probability Distribution ◽  
Cell Biology ◽  
Posterior Probability ◽  
Monte Carlo Sampling ◽  
Posterior Probability Distribution ◽  
Model Predictions

Increasingly complex applications involve large datasets in combination with nonlinear and high-dimensional mathematical models. In this context, statistical inference is a challenging issue that calls for pragmatic approaches that take advantage of both Bayesian and frequentist methods. The elegance of Bayesian methodology is founded in the propagation of information content provided by experimental data and prior assumptions to the posterior probability distribution of model predictions. However, for complex applications, experimental data and prior assumptions potentially constrain the posterior probability distribution insufficiently. In these situations, Bayesian Markov chain Monte Carlo sampling can be infeasible. From a frequentist point of view, insufficient experimental data and prior assumptions can be interpreted as non-identifiability. The profile-likelihood approach offers to detect and to resolve non-identifiability by experimental design iteratively. Therefore, it allows one to better constrain the posterior probability distribution until Markov chain Monte Carlo sampling can be used securely. Using an application from cell biology, we compare both methods and show that a successive application of the two methods facilitates a realistic assessment of uncertainty in model predictions.

Sign in / Sign up

Export Citation Format

Share Document