scholarly journals Monotone measures of ergodicity for Markov chains

1998 ◽  
Vol 11 (3) ◽  
pp. 283-288
Author(s):  
J. Keilson ◽  
O. A. Vasicek
Keyword(s):  
Markov Chain ◽  
Relaxation Time ◽  
Markov Chains ◽  
Small Subset ◽  
Finite Markov Chain ◽  
Time Reversibility

The following paper, first written in 1974, was never published other than as part of an internal research series. Its lack of publication is unrelated to the merits of the paper and the paper is of current importance by virtue of its relation to the relaxation time. A systematic discussion is provided of the approach of a finite Markov chain to ergodicity by proving the monotonicity of an important set of norms, each measures of egodicity, whether or not time reversibility is present. The paper is of particular interest because the discussion of the relaxation time of a finite Markov chain [2] has only been clean for time reversible chains, a small subset of the chains of interest. This restriction is not present here. Indeed, a new relaxation time quoted quantifies the relaxation time for all finite ergodic chains (cf. the discussion of Q1(t) below Equation (1.7)]. This relaxation time was developed by Keilson with A. Roy in his thesis [6], yet to be published.

scholarly journals Normal distributions of finite Markov chains

2019 ◽  
Vol 29 (08) ◽  
pp. 1431-1449
Author(s):  
John Rhodes ◽  
Anne Schilling
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Planar Graphs ◽  
Finite Semigroup ◽  
Finite Markov Chain ◽  
Normal Distributions ◽  
Straight Line ◽  
Finite Markov Chains ◽  
The Right

We show that the stationary distribution of a finite Markov chain can be expressed as the sum of certain normal distributions. These normal distributions are associated to planar graphs consisting of a straight line with attached loops. The loops touch only at one vertex either of the straight line or of another attached loop. Our analysis is based on our previous work, which derives the stationary distribution of a finite Markov chain using semaphore codes on the Karnofsky–Rhodes and McCammond expansion of the right Cayley graph of the finite semigroup underlying the Markov chain.

Geometric bounds on iterative approximations for nearly completely decomposable Markov chains

1990 ◽  
Vol 27 (03) ◽  
pp. 521-529 ◽  
Author(s):  
Guy Louchard ◽  
Guy Latouche
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Perturbation Analysis ◽  
Stochastic Matrix ◽  
Stationary Probability ◽  
Probability Vector ◽  
Finite Markov Chain ◽  
Stationary Probability Vector

We consider a finite Markov chain with nearly-completely decomposable stochastic matrix. We determine bounds for the error, when the stationary probability vector is approximated via a perturbation analysis.

scholarly journals SERIES EXPANSIONS FOR FINITE-STATE MARKOV CHAINS

2007 ◽  
Vol 21 (3) ◽  
pp. 381-400 ◽  
Author(s):  
Bernd Heidergott ◽  
Arie Hordijk ◽  
Miranda van Uitert
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Numerical Algorithm ◽  
Series Expansions ◽  
Finite Markov Chain ◽  
Numerical Examples ◽  
Finite State

This article provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

Geometric bounds on iterative approximations for nearly completely decomposable Markov chains

1990 ◽  
Vol 27 (3) ◽  
pp. 521-529 ◽  
Author(s):  
Guy Louchard ◽  
Guy Latouche
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Perturbation Analysis ◽  
Stochastic Matrix ◽  
Stationary Probability ◽  
Probability Vector ◽  
Finite Markov Chain ◽  
Stationary Probability Vector

We consider a finite Markov chain with nearly-completely decomposable stochastic matrix. We determine bounds for the error, when the stationary probability vector is approximated via a perturbation analysis.

Invariant measures of time-reversible Markov chains

1979 ◽  
Vol 16 (01) ◽  
pp. 226-229 ◽  
Author(s):  
P. Suomela
Keyword(s):  
Markov Chain ◽  
Invariant Measure ◽  
Markov Chains ◽  
Explicit Formula ◽  
Invariant Measures ◽  
Transition Probabilities ◽  
Time Reversibility ◽  
Reversible Markov Chain ◽  
Reversible Markov Chains

An explicit formula for an invariant measure of a time-reversible Markov chain is presented. It is based on a characterization of time reversibility in terms of the transition probabilities alone.

XXIV.—Conditions for the Ergodicity of Non-homogeneous Finite Markov Chains

1957 ◽  
Vol 64 (4) ◽  
pp. 369-380 ◽  
Author(s):  
J. L. Mott
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
Transition Probabilities ◽  
Finite Markov Chain ◽  
Finite Markov Chains ◽  
Product Of Matrices

SynopsisIn this note we study the asymptotic behaviour of a product of matrices where Pj is a matrix of transition probabilities in a non-homogeneous finite Markov chain. We give conditions that (i) the rows of P(n) tend to identity and that (ii) P(n) tends to a limit matrix with identical rows.

Some passage-time generating functions for discrete-time and continuous-time finite Markov chains

1967 ◽  
Vol 4 (03) ◽  
pp. 496-507 ◽  
Author(s):  
J. N. Darroch ◽  
K. W. Morris
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Generating Functions ◽  
Continuous Time ◽  
Passage Time ◽  
Finite Markov Chain ◽  
Finite Markov Chains

Let T denote a subset of the possible transitions between the states of a finite Markov chain and let Yk denote the time of the kth occurrence of a T-transition. Formulae are derived for the generating functions of Yk , of Yj + k — Yj and of Yj + k — Yj in the limit as j → ∞, for both discrete-time and continuoustime chains. Several particular cases are briefly discussed.

Weak lumpability in finite Markov chains

1982 ◽  
Vol 19 (03) ◽  
pp. 685-691 ◽  
Author(s):  
Atef M. Abdel-moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Transition Probability ◽  
Linear Equations ◽  
Transition Probability Matrix ◽  
Finite Markov Chain ◽  
Systems Of Linear Equations ◽  
Finite Markov Chains ◽  
The Given

Criteria are given to determine whether a given finite Markov chain can be lumped weakly with respect to a given partition of its state space. These conditions are given in terms of solution classes of systems of linear equations associated with the transition probability matrix of the Markov chain and the given partition.

Weak lumpability in finite Markov chains

1982 ◽  
Vol 19 (3) ◽  
pp. 685-691 ◽  
Author(s):  
Atef M. Abdel-moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Transition Probability ◽  
Linear Equations ◽  
Transition Probability Matrix ◽  
Finite Markov Chain ◽  
Systems Of Linear Equations ◽  
Finite Markov Chains ◽  
The Given

Criteria are given to determine whether a given finite Markov chain can be lumped weakly with respect to a given partition of its state space. These conditions are given in terms of solution classes of systems of linear equations associated with the transition probability matrix of the Markov chain and the given partition.

Some passage-time generating functions for discrete-time and continuous-time finite Markov chains

1967 ◽  
Vol 4 (3) ◽  
pp. 496-507 ◽  
Author(s):  
J. N. Darroch ◽  
K. W. Morris
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Generating Functions ◽  
Continuous Time ◽  
Passage Time ◽  
Finite Markov Chain ◽  
Finite Markov Chains

Let T denote a subset of the possible transitions between the states of a finite Markov chain and let Yk denote the time of the kth occurrence of a T-transition. Formulae are derived for the generating functions of Yk, of Yj + k — Yj and of Yj + k — Yj in the limit as j → ∞, for both discrete-time and continuoustime chains. Several particular cases are briefly discussed.

Sign in / Sign up

Export Citation Format

Share Document