Cheeger inequalities for absorbing Markov chains

2016 ◽  
Vol 48 (3) ◽  
pp. 631-647
Author(s):  
Gary Froyland ◽  
Robyn M. Stuart
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Transition Matrices ◽  
Stochastic Transition ◽  
Absorbing Markov Chains ◽  
Second Eigenvalue

Abstract We construct Cheeger-type bounds for the second eigenvalue of a substochastic transition probability matrix in terms of the Markov chain's conductance and metastability (and vice versa) with respect to its quasistationary distribution, extending classical results for stochastic transition matrices.

scholarly journals Computable bounds of an 𝓁²-spectral gap for discrete Markov chains with band transition matrices

2016 ◽  
Vol 53 (3) ◽  
pp. 946-952
Author(s):  
Loï Hervé ◽  
James Ledoux
Keyword(s):  
Convergence Rate ◽  
Markov Chains ◽  
Spectral Radius ◽  
Spectral Gap ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Geometric Ergodicity ◽  
Transition Matrices ◽  
Essential Spectral Radius ◽  
Band Transition

AbstractWe analyse the 𝓁²(𝜋)-convergence rate of irreducible and aperiodic Markov chains with N-band transition probability matrix P and with invariant distribution 𝜋. This analysis is heavily based on two steps. First, the study of the essential spectral radius ress(P|𝓁²(𝜋)) of P|𝓁²(𝜋) derived from Hennion’s quasi-compactness criteria. Second, the connection between the spectral gap property (SG2) of P on 𝓁²(𝜋) and the V-geometric ergodicity of P. Specifically, the (SG2) is shown to hold under the condition α0≔∑m=−NNlim supi→+∞(P(i,i+m)P*(i+m,i)1∕2<1. Moreover, ress(P|𝓁²(𝜋)≤α0. Effective bounds on the convergence rate can be provided from a truncation procedure.

Explicit transient probabilities of various Markov models

2021 ◽  
pp. 97-151
Author(s):  
Alan Krinik ◽  
Hubertus von Bremen ◽  
Ivan Ventura ◽  
Uyen Nguyen ◽  
Jeremy Lin ◽  
...  
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Scale Up ◽  
Transient Behavior ◽  
Transition Probability Matrix ◽  
Circulant Matrices ◽  
Transition Matrices ◽  
Content Type ◽  
Sample Paths ◽  
Birth Death

In analyzing finite-state Markov chains knowing the exact eigenvalues of the transition probability matrix P P is important information for predicting the explicit transient behavior of the system. Once the eigenvalues of P P are known, linear algebra and duality theory are used to find P k P^{k} where k = 2 , 3 , 4 , … k= 2,3,4,\ldots . This article is about finding explicit eigenvalue formulas, that scale up with the dimension of P P for various Markov chains. Eigenvalue formulas and expressions of P k P^{k} are first presented when P P is tridiagonal and Toeplitz. These results are generalized to tridiagonal matrices with alternating birth-death probabilities. More general eigenvalue formulas and expression of P k P^{k} are obtained for non-tridiagonal transition matrices P P that have both catastrophe-like and birth-death transitions. Similar results for circulant matrices are also explored. Applications include finding probabilities of sample paths restricted to a strip and generalized ballot box problems. These results generalize to Markov processes with P k P^{k} being replaced by e Q t e^{Qt} where Q Q is a transition rate matrix.

scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (01) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

scholarly journals Computing Hitting Probabilities of Markov Chains: Structural Results with regard to the Solution Space of the Corresponding System of Equations

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Hendrik Baumann ◽  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Solution Space ◽  
Transition Probability Matrix ◽  
Discrete State ◽  
Side Condition ◽  
Hitting Probabilities ◽  
All Solutions ◽  
The Difference ◽  
State Difference

In a previous paper, we have shown that forward use of the steady-state difference equations arising from homogeneous discrete-state space Markov chains may be subject to inherent numerical instability. More precisely, we have proven that, under some appropriate assumptions on the transition probability matrix P, the solution space S of the difference equation may be partitioned into two subspaces S=S1⊕S2, where the stationary measure of P is an element of S1, and all solutions in S1 are asymptotically dominated by the solutions corresponding to S2. In this paper, we discuss the analogous problem of computing hitting probabilities of Markov chains, which is affected by the same numerical phenomenon. In addition, we have to fulfill a somewhat complicated side condition which essentially differs from those conditions one is usually confronted with when solving initial and boundary value problems. To extract the desired solution, an efficient and numerically stable generalized-continued-fraction-based algorithm is developed.

The square root law in the embedding detection problem for Markov chains with unknown matrix of transition probabilities

2019 ◽  
Vol 29 (1) ◽  
pp. 59-68
Author(s):  
Artem V. Volgin
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Classical Model ◽  
Transition Probability Matrix ◽  
Square Root ◽  
Detection Problem ◽  
Asymptotic Growth

Abstract We consider the classical model of embeddings in a simple binary Markov chain with unknown transition probability matrix. We obtain conditions on the asymptotic growth of lengths of the original and embedded sequences sufficient for the consistency of the proposed statistical embedding detection test.

scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (1) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

Computer Simulation Forecast of the Markov Chain for Payback Period of Construction Projects Investment

2013 ◽  
Vol 671-674 ◽  
pp. 3096-3099
Author(s):  
Meng Fang Zhang ◽  
Liang Huang ◽  
Lu Yang Shan
Keyword(s):  
Computer Simulation ◽  
Markov Chains ◽  
Cash Flow ◽  
Construction Projects ◽  
Transition Probability ◽  
Economic Effect ◽  
Cash Flows ◽  
Scientific Basis ◽  
Transition Probability Matrix ◽  
Payback Period

The investment payback period of construction projects is an important index that evaluate and measure economic effect of project investment. It is difficult that the investment payback period of construction projects is calculated generally using analytic method.We established the mathematical model of the payback period, the annual net cash flows are Markov chains. According to the similar projects, collected net yearly cash flow and each quarter net cash flow, A one-step transition probability matrix was described by using the computer simulation of Markov chains, forecasted the dynamic and static payback period of construction projects investment. so as to provide the scientific basis data for decision makers.

scholarly journals A geometric invariant in weak lumpability of finite Markov chains

1997 ◽  
Vol 34 (4) ◽  
pp. 847-858 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Spectral Properties ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Homogeneous Markov Chain ◽  
Geometric Invariant ◽  
Initial State ◽  
Direct Sum ◽  
Finite Markov Chains

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

scholarly journals Computation of Invariant Measures and Stationary Expectations for Markov Chains with Block-Band Transition Matrix

2020 ◽  
Vol 2020 ◽  
pp. 1-16
Author(s):  
Hendrik Baumann ◽  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Large Scale ◽  
Invariant Measures ◽  
Markov Models ◽  
Transition Probability ◽  
Geometric Approach ◽  
Strong Relationship ◽  
Transition Probability Matrix ◽  
Step Transition ◽  
Matrix Continued Fractions

This paper deals with the computation of invariant measures and stationary expectations for discrete-time Markov chains governed by a block-structured one-step transition probability matrix. The method generalizes in some respect Neuts’ matrix-geometric approach to vector-state Markov chains. The method reveals a strong relationship between Markov chains and matrix continued fractions which can provide valuable information for mastering the growing complexity of real-world applications of large-scale grid systems and multidimensional level-dependent Markov models. The results obtained are extended to continuous-time Markov chains.

Sign in / Sign up

Export Citation Format

Share Document