scholarly journals Approximations of geometrically ergodic reversible markov chains

2021 ◽  
Vol 53 (4) ◽  
pp. 981-1022
Author(s):  
Jeffrey Negrea ◽  
Jeffrey S. Rosenthal
Keyword(s):  
Hilbert Space ◽  
Monte Carlo ◽  
Markov Chain ◽  
Stationary Distribution ◽  
Regularity Conditions ◽  
Geometric Ergodicity ◽  
Mcmc Algorithms ◽  
Reversible Markov Chains ◽  
Speed Up ◽  
Additional Regularity

AbstractA common tool in the practice of Markov chain Monte Carlo (MCMC) is to use approximating transition kernels to speed up computation when the desired kernel is slow to evaluate or is intractable. A limited set of quantitative tools exists to assess the relative accuracy and efficiency of such approximations. We derive a set of tools for such analysis based on the Hilbert space generated by the stationary distribution we intend to sample, $L_2(\pi)$. Our results apply to approximations of reversible chains which are geometrically ergodic, as is typically the case for applications to MCMC. The focus of our work is on determining whether the approximating kernel will preserve the geometric ergodicity of the exact chain, and whether the approximating stationary distribution will be close to the original stationary distribution. For reversible chains, our results extend the results of Johndrow et al. (2015) from the uniformly ergodic case to the geometrically ergodic case, under some additional regularity conditions. We then apply our results to a number of approximate MCMC algorithms.

scholarly journals On the convergence rates of some adaptive Markov chain Monte Carlo algorithms

2015 ◽  
Vol 52 (3) ◽  
pp. 811-825
Author(s):  
Yves Atchadé ◽  
Yizao Wang
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Convergence Rates ◽  
Mixing Time ◽  
Regularity Conditions ◽  
Monte Carlo Algorithms ◽  
Mcmc Algorithms ◽  
Variation Distance ◽  
Importance Resampling

In this paper we study the mixing time of certain adaptive Markov chain Monte Carlo (MCMC) algorithms. Under some regularity conditions, we show that the convergence rate of importance resampling MCMC algorithms, measured in terms of the total variation distance, is O(n-1). By means of an example, we establish that, in general, this algorithm does not converge at a faster rate. We also study the interacting tempering algorithm, a simplified version of the equi-energy sampler, and establish that its mixing time is of order O(n-1/2).

scholarly journals On the convergence rates of some adaptive Markov chain Monte Carlo algorithms

2015 ◽  
Vol 52 (03) ◽  
pp. 811-825
Author(s):  
Yves Atchadé ◽  
Yizao Wang
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Convergence Rates ◽  
Mixing Time ◽  
Regularity Conditions ◽  
Monte Carlo Algorithms ◽  
Mcmc Algorithms ◽  
Variation Distance ◽  
Importance Resampling

In this paper we study the mixing time of certain adaptive Markov chain Monte Carlo (MCMC) algorithms. Under some regularity conditions, we show that the convergence rate of importance resampling MCMC algorithms, measured in terms of the total variation distance, isO(n-1). By means of an example, we establish that, in general, this algorithm does not converge at a faster rate. We also study the interacting tempering algorithm, a simplified version of the equi-energy sampler, and establish that its mixing time is of orderO(n-1/2).

A Distributed Computer System for Parallel Markov Chain Monte Carlo (MCMC)

2018 ◽  
Author(s):  
Michael Hynes
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Parameter Space ◽  
Heterogeneous Computing ◽  
Peer To Peer ◽  
Parallel Tempering ◽  
Single Chain ◽  
Distributed Computing Systems ◽  
Mcmc Algorithms

A ubiquitous problem in physics is to determine expectation values of observables associated with a system. This problem is typically formulated as an integration of some likelihood over a multidimensional parameter space. In Bayesian analysis, numerical Markov Chain Monte Carlo (MCMC) algorithms are employed to solve such integrals using a fixed number of samples in the Markov Chain. In general, MCMC algorithms are computationally expensive for large datasets and have difficulties sampling from multimodal parameter spaces. An MCMC implementation that is robust and inexpensive for researchers is desired. Distributed computing systems have shown the potential to act as virtual supercomputers, such as in the SETI@home project in which millions of private computers participate. We propose that a clustered peer-to-peer (P2P) computer network serves as an ideal structure to run Markovian state exchange algorithms such as Parallel Tempering (PT). PT overcomes the difficulty in sampling from multimodal distributions by running multiple chains in parallel with different target distributions andexchanging their states in a Markovian manner. To demonstrate the feasibility of peer-to-peer Parallel Tempering (P2P PT), a simple two-dimensional dataset consisting of two Gaussian signals separated by a region of low probability was used in a Bayesian parameter fitting algorithm. A small connected peer-to-peer network was constructed using separate processes on a linux kernel, and P2P PT was applied to the dataset. These sampling results were compared with those obtained from sampling the parameter space with a single chain. It was found that the single chain was unable to sample both modes effectively, while the P2P PT method explored the target distribution well, visiting both modes approximately equally. Future work will involve scaling to many dimensions and large networks, and convergence conditions with highly heterogeneous computing capabilities of members within the network.

Convergence Properties of Perturbed Markov Chains

1998 ◽  
Vol 35 (01) ◽  
pp. 1-11 ◽  
Author(s):  
Gareth O. Roberts ◽  
Jeffrey S. Rosenthal ◽  
Peter O. Schwartz
Keyword(s):  
Computer Simulation ◽  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chains ◽  
Rates Of Convergence ◽  
Geometric Ergodicity ◽  
Roundoff Error ◽  
Convergence Properties ◽  
Small Perturbations ◽  
Monte Carlo Algorithms

In this paper, we consider the question of which convergence properties of Markov chains are preserved under small perturbations. Properties considered include geometric ergodicity and rates of convergence. Perturbations considered include roundoff error from computer simulation. We are motivated primarily by interest in Markov chain Monte Carlo algorithms.

scholarly journals Exact likelihood-free Markov chain Monte Carlo for elliptically contoured distributions

2015 ◽  
Vol 14 (4) ◽  
Author(s):  
Patrick Muchmore ◽  
Paul Marjoram
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Stationary Distribution ◽  
Large Family ◽  
Multivariate Normal ◽  
Elliptically Contoured Distributions ◽  
Simulation Based ◽  
A Chain ◽  
Elliptically Contoured

AbstractRecent results in Markov chain Monte Carlo (MCMC) show that a chain based on an unbiased estimator of the likelihood can have a stationary distribution identical to that of a chain based on exact likelihood calculations. In this paper we develop such an estimator for elliptically contoured distributions, a large family of distributions that includes and generalizes the multivariate normal. We then show how this estimator, combined with pseudorandom realizations of an elliptically contoured distribution, can be used to run MCMC in a way that replicates the stationary distribution of a likelihood based chain, but does not require explicit likelihood calculations. Because many elliptically contoured distributions do not have closed form densities, our simulation based approach enables exact MCMC based inference in a range of cases where previously it was impossible.

Is Partial-Dimension Convergence a Problem for Inferences from MCMC Algorithms?

2007 ◽  
Vol 16 (2) ◽  
pp. 153-178 ◽  
Author(s):  
Jeff Gill
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Convergence Problem ◽  
Slow Convergence ◽  
Mcmc Algorithms ◽  
Multiple Dimensions ◽  
Analytical Work ◽  
Complex Models ◽  
Slow Mixing

Increasingly, political science researchers are turning to Markov chain Monte Carlo methods to solve inferential problems with complex models and problematic data. This is an enormously powerful set of tools based on replacing difficult or impossible analytical work with simulated empirical draws from the distributions of interest. Although practitioners are generally aware of the importance of convergence of the Markov chain, many are not fully aware of the difficulties in fully assessing convergence across multiple dimensions. In most applied circumstances, every parameter dimension must be converged for the others to converge. The usual culprit is slow mixing of the Markov chain and therefore slow convergence towards the target distribution. This work demonstrates the partial convergence problem for the two dominant algorithms and illustrates these issues with empirical examples.

Experiences With Markov Chain Monte Carlo Convergence Assessment in Two Psychometric Examples

2004 ◽  
Vol 29 (4) ◽  
pp. 461-488 ◽  
Author(s):  
Sandip Sinharay
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Diagnostic Tool ◽  
Statistical Models ◽  
Real Data ◽  
Mcmc Algorithm ◽  
Mcmc Algorithms ◽  
Convergence Diagnostics ◽  
Number Of Iterations

There is an increasing use of Markov chain Monte Carlo (MCMC) algorithms for fitting statistical models in psychometrics, especially in situations where the traditional estimation techniques are very difficult to apply. One of the disadvantages of using an MCMC algorithm is that it is not straightforward to determine the convergence of the algorithm. Using the output of an MCMC algorithm that has not converged may lead to incorrect inferences on the problem at hand. The convergence is not one to a point, but that of the distribution of a sequence of generated values to another distribution, and hence is not easy to assess; there is no guaranteed diagnostic tool to determine convergence of an MCMC algorithm in general. This article examines the convergence of MCMC algorithms using a number of convergence diagnostics for two real data examples from psychometrics. Findings from this research have the potential to be useful to researchers using the algorithms. For both the examples, the number of iterations required (suggested by the diagnostics) to be reasonably confident that the MCMC algorithm has converged may be larger than what many practitioners consider to be safe.

Sign in / Sign up

Export Citation Format

Share Document