scholarly journals Computable bounds of an 𝓁²-spectral gap for discrete Markov chains with band transition matrices

2016 ◽  
Vol 53 (3) ◽  
pp. 946-952
Author(s):  
Loï Hervé ◽  
James Ledoux
Keyword(s):  
Convergence Rate ◽  
Markov Chains ◽  
Spectral Radius ◽  
Spectral Gap ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Geometric Ergodicity ◽  
Transition Matrices ◽  
Essential Spectral Radius ◽  
Band Transition

AbstractWe analyse the 𝓁²(𝜋)-convergence rate of irreducible and aperiodic Markov chains with N-band transition probability matrix P and with invariant distribution 𝜋. This analysis is heavily based on two steps. First, the study of the essential spectral radius ress(P|𝓁²(𝜋)) of P|𝓁²(𝜋) derived from Hennion’s quasi-compactness criteria. Second, the connection between the spectral gap property (SG2) of P on 𝓁²(𝜋) and the V-geometric ergodicity of P. Specifically, the (SG2) is shown to hold under the condition α0≔∑m=−NNlim supi→+∞(P(i,i+m)P*(i+m,i)1∕2<1. Moreover, ress(P|𝓁²(𝜋)≤α0. Effective bounds on the convergence rate can be provided from a truncation procedure.

Cheeger inequalities for absorbing Markov chains

2016 ◽  
Vol 48 (3) ◽  
pp. 631-647
Author(s):  
Gary Froyland ◽  
Robyn M. Stuart
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Transition Matrices ◽  
Stochastic Transition ◽  
Absorbing Markov Chains ◽  
Second Eigenvalue

Abstract We construct Cheeger-type bounds for the second eigenvalue of a substochastic transition probability matrix in terms of the Markov chain's conductance and metastability (and vice versa) with respect to its quasistationary distribution, extending classical results for stochastic transition matrices.

Explicit transient probabilities of various Markov models

2021 ◽  
pp. 97-151
Author(s):  
Alan Krinik ◽  
Hubertus von Bremen ◽  
Ivan Ventura ◽  
Uyen Nguyen ◽  
Jeremy Lin ◽  
...  
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Scale Up ◽  
Transient Behavior ◽  
Transition Probability Matrix ◽  
Circulant Matrices ◽  
Transition Matrices ◽  
Content Type ◽  
Sample Paths ◽  
Birth Death

In analyzing finite-state Markov chains knowing the exact eigenvalues of the transition probability matrix P P is important information for predicting the explicit transient behavior of the system. Once the eigenvalues of P P are known, linear algebra and duality theory are used to find P k P^{k} where k = 2 , 3 , 4 , … k= 2,3,4,\ldots . This article is about finding explicit eigenvalue formulas, that scale up with the dimension of P P for various Markov chains. Eigenvalue formulas and expressions of P k P^{k} are first presented when P P is tridiagonal and Toeplitz. These results are generalized to tridiagonal matrices with alternating birth-death probabilities. More general eigenvalue formulas and expression of P k P^{k} are obtained for non-tridiagonal transition matrices P P that have both catastrophe-like and birth-death transitions. Similar results for circulant matrices are also explored. Applications include finding probabilities of sample paths restricted to a strip and generalized ballot box problems. These results generalize to Markov processes with P k P^{k} being replaced by e Q t e^{Qt} where Q Q is a transition rate matrix.

Characterization of convergence rates for the approximation of the stationary distribution of infinite monotone stochastic matrices

1996 ◽  
Vol 33 (04) ◽  
pp. 974-985 ◽  
Author(s):  
F. Simonot ◽  
Y. Q. Song
Keyword(s):  
Convergence Rate ◽  
Generating Function ◽  
Convergence Rates ◽  
Transition Probability ◽  
Stochastic Matrix ◽  
Input Sequence ◽  
Transition Probability Matrix ◽  
Stochastic Matrices ◽  
Preceding Result

Let P be an infinite irreducible stochastic matrix, recurrent positive and stochastically monotone and Pn be any n × n stochastic matrix with Pn ≧ Tn , where Tn denotes the n × n northwest corner truncation of P. These assumptions imply the existence of limit distributions π and π n for P and Pn respectively. We show that if the Markov chain with transition probability matrix P meets the further condition of geometric recurrence then the exact convergence rate of π n to π can be expressed in terms of the radius of convergence of the generating function of π. As an application of the preceding result, we deal with the random walk on a half line and prove that the assumption of geometric recurrence can be relaxed. We also show that if the i.i.d. input sequence (A(m)) is such that we can find a real number r 0 &gt; 1 with , then the exact convergence rate of π n to π is characterized by r 0. Moreover, when the generating function of A is not defined for |z| &gt; 1, we derive an upper bound for the distance between π n and π based on the moments of A.

Conductance bounds on the L 2 convergence rate of Metropolis algorithms on unbounded state spaces

2004 ◽  
Vol 36 (01) ◽  
pp. 243-266
Author(s):  
Søren F. Jarner ◽  
Wai Kong Yuen
Keyword(s):  
Convergence Rate ◽  
Markov Chains ◽  
Spectral Gap ◽  
Unbounded Domains ◽  
Decomposition Theorem ◽  
Metropolis Algorithm ◽  
The State ◽  
Bounded Domains ◽  
Target Density ◽  
State Spaces

In this paper we derive bounds on the conductance and hence on the spectral gap of a Metropolis algorithm with a monotone, log-concave target density on an interval of ℝ. We show that the minimal conductance set has measure ½ and we use this characterization to bound the conductance in terms of the conductance of the algorithm restricted to a smaller domain. Whereas previous work on conductance has resulted in good bounds for Markov chains on bounded domains, this is the first conductance bound applicable to unbounded domains. We then show how this result can be combined with the state-decomposition theorem of Madras and Randall (2002) to bound the spectral gap of Metropolis algorithms with target distributions with monotone, log-concave tails on ℝ.

scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (01) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

scholarly journals Computing Hitting Probabilities of Markov Chains: Structural Results with regard to the Solution Space of the Corresponding System of Equations

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Hendrik Baumann ◽  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Solution Space ◽  
Transition Probability Matrix ◽  
Discrete State ◽  
Side Condition ◽  
Hitting Probabilities ◽  
All Solutions ◽  
The Difference ◽  
State Difference

In a previous paper, we have shown that forward use of the steady-state difference equations arising from homogeneous discrete-state space Markov chains may be subject to inherent numerical instability. More precisely, we have proven that, under some appropriate assumptions on the transition probability matrix P, the solution space S of the difference equation may be partitioned into two subspaces S=S1⊕S2, where the stationary measure of P is an element of S1, and all solutions in S1 are asymptotically dominated by the solutions corresponding to S2. In this paper, we discuss the analogous problem of computing hitting probabilities of Markov chains, which is affected by the same numerical phenomenon. In addition, we have to fulfill a somewhat complicated side condition which essentially differs from those conditions one is usually confronted with when solving initial and boundary value problems. To extract the desired solution, an efficient and numerically stable generalized-continued-fraction-based algorithm is developed.

The square root law in the embedding detection problem for Markov chains with unknown matrix of transition probabilities

2019 ◽  
Vol 29 (1) ◽  
pp. 59-68
Author(s):  
Artem V. Volgin
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Classical Model ◽  
Transition Probability Matrix ◽  
Square Root ◽  
Detection Problem ◽  
Asymptotic Growth

Abstract We consider the classical model of embeddings in a simple binary Markov chain with unknown transition probability matrix. We obtain conditions on the asymptotic growth of lengths of the original and embedded sequences sufficient for the consistency of the proposed statistical embedding detection test.

scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (1) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

scholarly journals On the Convergence Rate of Bonus-Malus Systems

1992 ◽  
Vol 22 (2) ◽  
pp. 217-223 ◽  
Author(s):  
Heikki Bonsdorff
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Convergence Rate ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Limit Probability ◽  
Step Transition

AbstractUnder certain conditions, a Bonus-Malus system can be interpreted as a Markov chain whose n-step transition probabilities converge to a limit probability distribution. In this paper, the rate of the convergence is studied by means of the eigenvalues of the transition probability matrix of the Markov chain.

Sign in / Sign up

Export Citation Format

Share Document