Extended stochastic gradient Markov chain Monte Carlo for large-scale Bayesian variable selection

Summary Stochastic gradient Markov chain Monte Carlo algorithms have received much attention in Bayesian computing for big data problems, but they are only applicable to a small class of problems for which the parameter space has a fixed dimension and the log-posterior density is differentiable with respect to the parameters. This paper proposes an extended stochastic gradient Markov chain Monte Carlo algorithm which, by introducing appropriate latent variables, can be applied to more general large-scale Bayesian computing problems, such as those involving dimension jumping and missing data. Numerical studies show that the proposed algorithm is highly scalable and much more efficient than traditional Markov chain Monte Carlo algorithms.

Download Full-text

In search of lost mixing time: adaptive Markov chain Monte Carlo schemes for Bayesian variable selection with very large p

Biometrika ◽

10.1093/biomet/asaa055 ◽

2020 ◽

Author(s):

J E Griffin ◽

K G Łatuszyński ◽

M F J Steel

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Variable Selection ◽

Adaptive Design ◽

Mixing Time ◽

Bayesian Variable Selection ◽

Monte Carlo Algorithms ◽

Large Numbers ◽

Large P Small N

Summary The availability of datasets with large numbers of variables is rapidly increasing. The effective application of Bayesian variable selection methods for regression with these datasets has proved difficult since available Markov chain Monte Carlo methods do not perform well in typical problem sizes of interest. We propose new adaptive Markov chain Monte Carlo algorithms to address this shortcoming. The adaptive design of these algorithms exploits the observation that in large-$p$, small-$n$ settings, the majority of the $p$ variables will be approximately uncorrelated a posteriori. The algorithms adaptively build suitable nonlocal proposals that result in moves with squared jumping distance significantly larger than standard methods. Their performance is studied empirically in high-dimensional problems and speed-ups of up to four orders of magnitude are observed.

Download Full-text

A conditional density estimation partition model using logistic Gaussian processes

Biometrika ◽

10.1093/biomet/asz064 ◽

2019 ◽

Vol 107 (1) ◽

pp. 173-190

Author(s):

R D Payne ◽

N Guha ◽

Y Ding ◽

B K Mallick

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Density Estimation ◽

Gaussian Processes ◽

Latent Variables ◽

Monte Carlo Algorithm ◽

Conditional Density ◽

Partition Model ◽

Conditional Density Estimation

Summary Conditional density estimation seeks to model the distribution of a response variable conditional on covariates. We propose a Bayesian partition model using logistic Gaussian processes to perform conditional density estimation. The partition takes the form of a Voronoi tessellation and is learned from the data using a reversible jump Markov chain Monte Carlo algorithm. The methodology models data in which the density changes sharply throughout the covariate space, and can be used to determine where important changes in the density occur. The Markov chain Monte Carlo algorithm involves a Laplace approximation on the latent variables of the logistic Gaussian process model which marginalizes the parameters in each partition element, allowing an efficient search of the approximate posterior distribution of the tessellation. The method is consistent when the density is piecewise constant in the covariate space or when the density is Lipschitz continuous with respect to the covariates. In simulation and application to wind turbine data, the model successfully estimates the partition structure and conditional distribution.

Download Full-text

Reversible jump Markov chain Monte Carlo algorithms for Bayesian variable selection in logistic mixed models

Communications in Statistics - Simulation and Computation ◽

10.1080/03610918.2017.1341525 ◽

2017 ◽

Vol 47 (8) ◽

pp. 2234-2247 ◽

Cited By ~ 1

Author(s):

Jia-Chiun Pan ◽

Mei-Hsien Lee ◽

Miao-Yu Tsai

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Variable Selection ◽

Mixed Models ◽

Bayesian Variable Selection ◽

Reversible Jump ◽

Monte Carlo Algorithms

Download Full-text

Accelerating Overlapping Community Detection: Performance Tuning a Stochastic Gradient Markov Chain Monte Carlo Algorithm

Euro-Par 2020: Parallel Processing - Lecture Notes in Computer Science ◽

10.1007/978-3-030-57675-2_32 ◽

2020 ◽

pp. 510-526

Author(s):

Ismail El-Helw ◽

Rutger Hofman ◽

Henri E. Bal

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Community Detection ◽

Detection Performance ◽

Stochastic Gradient ◽

Monte Carlo Algorithm ◽

Performance Tuning ◽

Overlapping Community Detection ◽

Overlapping Community

Download Full-text

Bridging trees for posterior inference on ancestral recombination graphs

Proceedings of The Royal Society A Mathematical Physical and Engineering Sciences ◽

10.1098/rspa.2018.0568 ◽

2018 ◽

Vol 474 (2220) ◽

pp. 20180568

Author(s):

K. Heine ◽

A. Beskos ◽

A. Jasra ◽

D. Balding ◽

M. De Iorio

Keyword(s):

Monte Carlo ◽

Markov Chain ◽

Markov Chain Monte Carlo ◽

Dna Sequence ◽

Dna Sequences ◽

Large Scale ◽

Monte Carlo Algorithm ◽

Posterior Inference ◽

History Of ◽

Ancestral Recombination Graphs

We present a new Markov chain Monte Carlo algorithm, implemented in the software Arbores, for inferring the history of a sample of DNA sequences. Our principal innovation is a bridging procedure, previously applied only for simple stochastic processes, in which the local computations within a bridge can proceed independently of the rest of the DNA sequence, facilitating large-scale parallelization.

Download Full-text