Monte Carlo simulation and large deviations theory for uniformly recurrent Markov chains

1990 ◽  
Vol 27 (01) ◽  
pp. 44-59 ◽  
Author(s):  
James A Bucklew ◽  
Peter Ney ◽  
John S. Sadowsky
Keyword(s):  
Monte Carlo Simulation ◽  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chains ◽  
Large Deviations ◽  
Large Deviation ◽  
Homogeneous Markov Chain ◽  
Underlying Distribution ◽  
Monte Carlo Simulation Technique ◽  
Large Deviations Theory

Importance sampling is a Monte Carlo simulation technique in which the simulation distribution is different from the true underlying distribution. In order to obtain an unbiased Monte Carlo estimate of the desired parameter, simulated events are weighted to reflect their true relative frequency. In this paper, we consider the estimation via simulation of certain large deviations probabilities for time-homogeneous Markov chains. We first demonstrate that when the simulation distribution is also a homogeneous Markov chain, the estimator variance will vanish exponentially as the sample size n tends to∞. We then prove that the estimator variance is asymptotically minimized by the same exponentially twisted Markov chain which arises in large deviation theory, and furthermore, this optimization is unique among uniformly recurrent homogeneous Markov chain simulation distributions.

Monte Carlo simulation and large deviations theory for uniformly recurrent Markov chains

1990 ◽  
Vol 27 (1) ◽  
pp. 44-59 ◽  
Author(s):  
James A Bucklew ◽  
Peter Ney ◽  
John S. Sadowsky
Keyword(s):  
Monte Carlo Simulation ◽  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chains ◽  
Large Deviations ◽  
Large Deviation ◽  
Homogeneous Markov Chain ◽  
Underlying Distribution ◽  
Monte Carlo Simulation Technique ◽  
Large Deviations Theory

Importance sampling is a Monte Carlo simulation technique in which the simulation distribution is different from the true underlying distribution. In order to obtain an unbiased Monte Carlo estimate of the desired parameter, simulated events are weighted to reflect their true relative frequency. In this paper, we consider the estimation via simulation of certain large deviations probabilities for time-homogeneous Markov chains. We first demonstrate that when the simulation distribution is also a homogeneous Markov chain, the estimator variance will vanish exponentially as the sample size n tends to∞. We then prove that the estimator variance is asymptotically minimized by the same exponentially twisted Markov chain which arises in large deviation theory, and furthermore, this optimization is unique among uniformly recurrent homogeneous Markov chain simulation distributions.

scholarly journals Extended Laplace principle for empirical measures of a Markov chain

2019 ◽  
Vol 51 (01) ◽  
pp. 136-167 ◽  
Author(s):  
Stephan Eckstein
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Large Deviations ◽  
Large Deviation ◽  
Dual Pair ◽  
Wasserstein Distance ◽  
Large Deviations Principle ◽  
Empirical Measures ◽  
Leibler Divergence ◽  
Laplace Principle

AbstractWe consider discrete-time Markov chains with Polish state space. The large deviations principle for empirical measures of a Markov chain can equivalently be stated in Laplace principle form, which builds on the convex dual pair of relative entropy (or Kullback– Leibler divergence) and cumulant generating functional f ↦ ln ʃ exp (f). Following the approach by Lacker (2016) in the independent and identically distributed case, we generalize the Laplace principle to a greater class of convex dual pairs. We present in depth one application arising from this extension, which includes large deviation results and a weak law of large numbers for certain robust Markov chains—similar to Markov set chains—where we model robustness via the first Wasserstein distance. The setting and proof of the extended Laplace principle are based on the weak convergence approach to large deviations by Dupuis and Ellis (2011).

scholarly journals Large Deviations Properties of Maximum Entropy Markov Chains from Spike Trains

Entropy ◽  
2018 ◽  
Vol 20 (8) ◽  
pp. 573 ◽  
Author(s):  
Rodrigo Cofré ◽  
Cesar Maldonado ◽  
Fernando Rosas
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Large Deviations ◽  
Maximum Entropy ◽  
Statistical Physics ◽  
Large Deviation ◽  
Thermodynamic Formalism ◽  
Spike Trains ◽  
Statistical Fluctuation ◽  
Inference Problem

We consider the maximum entropy Markov chain inference approach to characterize the collective statistics of neuronal spike trains, focusing on the statistical properties of the inferred model. To find the maximum entropy Markov chain, we use the thermodynamic formalism, which provides insightful connections with statistical physics and thermodynamics from which large deviations properties arise naturally. We provide an accessible introduction to the maximum entropy Markov chain inference problem and large deviations theory to the community of computational neuroscience, avoiding some technicalities while preserving the core ideas and intuitions. We review large deviations techniques useful in spike train statistics to describe properties of accuracy and convergence in terms of sampling size. We use these results to study the statistical fluctuation of correlations, distinguishability, and irreversibility of maximum entropy Markov chains. We illustrate these applications using simple examples where the large deviation rate function is explicitly obtained for maximum entropy models of relevance in this field.

scholarly journals A Bayesian model for binary Markov chains

2004 ◽  
Vol 2004 (8) ◽  
pp. 421-429 ◽  
Author(s):  
Souad Assoudou ◽  
Belkheir Essebbar
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chains ◽  
Bayesian Estimation ◽  
Bayesian Model ◽  
Transition Probabilities ◽  
Simulated Data ◽  
Bayesian Estimator ◽  
Jeffreys Prior ◽  
Data Set

This note is concerned with Bayesian estimation of the transition probabilities of a binary Markov chain observed from heterogeneous individuals. The model is founded on the Jeffreys' prior which allows for transition probabilities to be correlated. The Bayesian estimator is approximated by means of Monte Carlo Markov chain (MCMC) techniques. The performance of the Bayesian estimates is illustrated by analyzing a small simulated data set.

Sign in / Sign up

Export Citation Format

Share Document