Adaptive Markov Chain Monte Carlo for Bayesian Variable Selection

Summary The availability of datasets with large numbers of variables is rapidly increasing. The effective application of Bayesian variable selection methods for regression with these datasets has proved difficult since available Markov chain Monte Carlo methods do not perform well in typical problem sizes of interest. We propose new adaptive Markov chain Monte Carlo algorithms to address this shortcoming. The adaptive design of these algorithms exploits the observation that in large-$p$, small-$n$ settings, the majority of the $p$ variables will be approximately uncorrelated a posteriori. The algorithms adaptively build suitable nonlocal proposals that result in moves with squared jumping distance significantly larger than standard methods. Their performance is studied empirically in high-dimensional problems and speed-ups of up to four orders of magnitude are observed.

Download Full-text

Reversible jump Markov chain Monte Carlo algorithms for Bayesian variable selection in logistic mixed models

Communications in Statistics - Simulation and Computation ◽

10.1080/03610918.2017.1341525 ◽

2017 ◽

Vol 47 (8) ◽

pp. 2234-2247 ◽

Cited By ~ 1

Author(s):

Jia-Chiun Pan ◽

Mei-Hsien Lee ◽

Miao-Yu Tsai

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Variable Selection ◽

Mixed Models ◽

Bayesian Variable Selection ◽

Reversible Jump ◽

Monte Carlo Algorithms

Download Full-text

Population-based reversible jump Markov chain Monte Carlo methods for Bayesian variable selection and evaluation under cost limit restrictions

Journal of the Royal Statistical Society Series C (Applied Statistics) ◽

10.1111/j.1467-9876.2008.00658.x ◽

2009 ◽

Vol 58 (3) ◽

pp. 383-403

Author(s):

D. Fouskakis ◽

I. Ntzoufras ◽

D. Draper

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Variable Selection ◽

Monte Carlo Methods ◽

Population Based ◽

Bayesian Variable Selection ◽

Reversible Jump ◽

Cost Limit

Download Full-text

Extended stochastic gradient Markov chain Monte Carlo for large-scale Bayesian variable selection

Biometrika ◽

10.1093/biomet/asaa029 ◽

2020 ◽

Vol 107 (4) ◽

pp. 997-1004

Author(s):

Qifan Song ◽

Yan Sun ◽

Mao Ye ◽

Faming Liang

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Latent Variables ◽

Large Scale ◽

Bayesian Variable Selection ◽

Stochastic Gradient ◽

Monte Carlo Algorithm ◽

Monte Carlo Algorithms ◽

Fixed Dimension

Summary Stochastic gradient Markov chain Monte Carlo algorithms have received much attention in Bayesian computing for big data problems, but they are only applicable to a small class of problems for which the parameter space has a fixed dimension and the log-posterior density is differentiable with respect to the parameters. This paper proposes an extended stochastic gradient Markov chain Monte Carlo algorithm which, by introducing appropriate latent variables, can be applied to more general large-scale Bayesian computing problems, such as those involving dimension jumping and missing data. Numerical studies show that the proposed algorithm is highly scalable and much more efficient than traditional Markov chain Monte Carlo algorithms.

Download Full-text