Optimal scaling of random-walk metropolis algorithms on general target distributions

Scaling of proposals for Metropolis algorithms is an important practical problem in Markov chain Monte Carlo implementation. Analyses of the random walk Metropolis for high-dimensional targets with specific functional forms have shown that in many cases the optimal scaling is achieved when the acceptance rate is approximately 0.234, but that there are exceptions. We present a general set of sufficient conditions which are invariant to orthonormal transformation of the coordinate axes and which ensure that the limiting optimal acceptance rate is 0.234. The criteria are shown to hold for the joint distribution of successive elements of a stationary pth-order multivariate Markov process.

Download Full-text

Optimal Scaling of the Random Walk Metropolis: General Criteria for the 0.234 Acceptance Rule

Journal of Applied Probability ◽

10.1017/s0021900200013073 ◽

2013 ◽

Vol 50 (01) ◽

pp. 1-15 ◽

Cited By ~ 1

Author(s):

Chris Sherlock

Keyword(s):

Random Walk ◽

Joint Distribution ◽

Practical Problem ◽

Sufficient Conditions ◽

Optimal Scaling ◽

Acceptance Rate ◽

High Dimensional ◽

Random Walk Metropolis ◽

Functional Forms ◽

Monte Carlo Implementation

Scaling of proposals for Metropolis algorithms is an important practical problem in Markov chain Monte Carlo implementation. Analyses of the random walk Metropolis for high-dimensional targets with specific functional forms have shown that in many cases the optimal scaling is achieved when the acceptance rate is approximately 0.234, but that there are exceptions. We present a general set of sufficient conditions which are invariant to orthonormal transformation of the coordinate axes and which ensure that the limiting optimal acceptance rate is 0.234. The criteria are shown to hold for the joint distribution of successive elements of a stationary pth-order multivariate Markov process.

Download Full-text

Optimal scaling of the random walk Metropolis algorithm under Lp mean differentiability

Journal of Applied Probability ◽

10.1017/jpr.2017.61 ◽

2017 ◽

Vol 54 (4) ◽

pp. 1233-1260 ◽

Cited By ~ 1

Author(s):

Alain Durmus ◽

Sylvain Le Corff ◽

Eric Moulines ◽

Gareth O. Roberts

Keyword(s):

Random Walk ◽

Scaling Limit ◽

Optimal Scaling ◽

High Dimensional ◽

Random Walk Metropolis ◽

Langevin Diffusion ◽

Random Walk Metropolis Algorithm ◽

The One ◽

Original Derivation ◽

Practical Implications

Abstract In this paper we consider the optimal scaling of high-dimensional random walk Metropolis algorithms for densities differentiable in the Lp mean but which may be irregular at some points (such as the Laplace density, for example) and/or supported on an interval. Our main result is the weak convergence of the Markov chain (appropriately rescaled in time and space) to a Langevin diffusion process as the dimension d goes to ∞. As the log-density might be nondifferentiable, the limiting diffusion could be singular. The scaling limit is established under assumptions which are much weaker than the one used in the original derivation of Roberts et al. (1997). This result has important practical implications for the use of random walk Metropolis algorithms in Bayesian frameworks based on sparsity inducing priors.

Download Full-text