A Randomized Maximum A Posteriori Method for Posterior Sampling of High Dimensional Nonlinear Bayesian Inverse Problems

2018 ◽  
Vol 40 (1) ◽  
pp. A142-A171 ◽  
Author(s):  
Kainan Wang ◽  
Tan Bui-Thanh ◽  
Omar Ghattas
Keyword(s):  
Inverse Problems ◽  
High Dimensional ◽  
Maximum A Posteriori ◽  
A Posteriori ◽  
Bayesian Inverse Problems ◽  
Posterior Sampling

scholarly journals Randomized maximum likelihood based posterior sampling

2021 ◽  
Author(s):  
Yuming Ba ◽  
Jana de Wiljes ◽  
Dean S. Oliver ◽  
Sebastian Reich
Keyword(s):  
Inverse Problems ◽  
Flow Problem ◽  
Low Rank ◽  
High Dimensional ◽  
Posterior Distributions ◽  
High Dimensions ◽  
Bayesian Inverse Problems ◽  
Posterior Sampling ◽  
Multimodal Distributions ◽  
Randomized Maximum Likelihood

AbstractMinimization of a stochastic cost function is commonly used for approximate sampling in high-dimensional Bayesian inverse problems with Gaussian prior distributions and multimodal posterior distributions. The density of the samples generated by minimization is not the desired target density, unless the observation operator is linear, but the distribution of samples is useful as a proposal density for importance sampling or for Markov chain Monte Carlo methods. In this paper, we focus on applications to sampling from multimodal posterior distributions in high dimensions. We first show that sampling from multimodal distributions is improved by computing all critical points instead of only minimizers of the objective function. For applications to high-dimensional geoscience inverse problems, we demonstrate an efficient approximate weighting that uses a low-rank Gauss-Newton approximation of the determinant of the Jacobian. The method is applied to two toy problems with known posterior distributions and a Darcy flow problem with multiple modes in the posterior.

scholarly journals High Dimensional Inference With Random Maximum A-Posteriori Perturbations

2019 ◽  
Vol 65 (10) ◽  
pp. 6539-6560 ◽  
Author(s):  
Tamir Hazan ◽  
Francesco Orabona ◽  
Anand D. Sarwate ◽  
Subhransu Maji ◽  
Tommi S. Jaakkola
Keyword(s):  
High Dimensional ◽  
Maximum A Posteriori ◽  
A Posteriori ◽  
High Dimensional Inference

scholarly journals Technical Note: Improving the computational efficiency of sparse matrix multiplication in linear atmospheric inverse problems

2016 ◽  
Author(s):  
Vineet Yadav ◽  
Anna M. Michalak
Keyword(s):  
Inverse Problems ◽  
Sparse Matrix ◽  
Sparse Matrices ◽  
Matrix Multiplication ◽  
Technical Note ◽  
Covariance Matrices ◽  
A Posteriori ◽  
Bayesian Inverse Problems ◽  
Dense Matrices ◽  
The Cost

Abstract. Matrix multiplication of two sparse matrices is a fundamental operation in linear Bayesian inverse problems for computing covariance matrices of observations and a posteriori uncertainties. Applications of sparse-sparse matrix multiplication algorithms for specific use-cases in such inverse problems remain unexplored. Here we present a hybrid-parallel sparse-sparse matrix multiplication approach that is more efficient by a third in terms of execution time and operation count relative to standard sparse matrix multiplication algorithms available in most libraries. Two modifications of this hybrid-parallel algorithm are also proposed for the types of operations typical of atmospheric inverse problems, which further reduce the cost of sparse matrix multiplication by yielding only upper triangular and/or dense matrices.

scholarly journals New approach to Bayesian high-dimensional linear regression

2018 ◽  
Vol 7 (4) ◽  
pp. 605-655 ◽  
Author(s):  
Shirin Jalali ◽  
Arian Maleki
Keyword(s):  
Linear Regression ◽  
Sampling Rate ◽  
High Dimensional ◽  
Maximum A Posteriori ◽  
Map Estimation ◽  
A Posteriori ◽  
Fundamental Limits ◽  
New Approach ◽  
Estimation Scheme ◽  
Response Variables

Abstract Consider the problem of estimating parameters $X^n \in \mathbb{R}^n $, from $m$ response variables $Y^m = AX^n+Z^m$, under the assumption that the distribution of $X^n$ is known. Lack of computationally feasible algorithms that employ generic prior distributions and provide a good estimate of $X^n$ has limited the set of distributions researchers use to model the data. To address this challenge, in this article, a new estimation scheme named quantized maximum a posteriori (Q-MAP) is proposed. The new method has the following properties: (i) In the noiseless setting, it has similarities to maximum a posteriori (MAP) estimation. (ii) In the noiseless setting, when $X_1,\ldots,X_n$ are independent and identically distributed, asymptotically, as $n$ grows to infinity, its required sampling rate ($m/n$) for an almost zero-distortion recovery approaches the fundamental limits. (iii) It scales favorably with the dimensions of the problem and therefore is applicable to high-dimensional setups. (iv) The solution of the Q-MAP optimization can be found via a proposed iterative algorithm that is provably robust to error (noise) in response variables.

Sign in / Sign up

Export Citation Format

Share Document