Accurate calculations of Stationary Distributions and Mean First Passage Times in Markov Renewal Processes and Markov Chains

AbstractThis article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30, 197–207, (1986) procedure. The technique is numerically stable in that it doesn’t involve subtractions. Algebraic expressions for the special cases of one, two, three and four states are derived.Aconsequence of the procedure is that the stationary distribution of the embedded Markov chain does not need to be derived in advance but can be found accurately from the derived mean first passage times. MatLab is utilized to carry out the computations, using some test problems from the literature.

Download Full-text

SIMPLE PROCEDURES FOR FINDING MEAN FIRST PASSAGE TIMES IN MARKOV CHAINS

Asia Pacific Journal of Operational Research ◽

10.1142/s0217595907001553 ◽

2007 ◽

Vol 24 (06) ◽

pp. 813-829 ◽

Cited By ~ 8

Author(s):

JEFFREY J. HUNTER

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Stationary Distribution ◽

Linear Equations ◽

First Passage Times ◽

First Passage ◽

Mean First Passage Times ◽

The Mean ◽

Computational Procedures ◽

Passage Times

The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell's fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse of I - P + eπT where eT = (1, 1, …, 1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures.

Download Full-text

Mean first-passage times of Brownian motion and related problems

Proceedings of the Royal Society of London Series A - Mathematical and Physical Sciences ◽

10.1098/rspa.1952.0051 ◽

1952 ◽

Vol 211 (1106) ◽

pp. 431-443 ◽

Cited By ~ 42

Keyword(s):

Brownian Motion ◽

First Passage Time ◽

Passage Time ◽

First Passage Times ◽

Analytical Relationship ◽

First Passage ◽

Special Boundaries ◽

Mean First Passage Times ◽

Special Cases ◽

Passage Times

The theory of first-passage times of Brownian motion is developed in general, and it is shown that for certain special boundaries—the only ones of any importance—mean first-passage times can be derived very simply, avoiding the usual method involving series. Moreover, these formulae have a close analytical relationship to the better-known type of formulae for average 'displacements’ in given intervals; there exist certain pairs of reciprocal relations. Some new formulae, of mathematical interest, for translational Brownian motion are given. The main application of the general theory, however, lies in the derivation of experimentally particularly useful formulae for rotational Brownian motion. Special cases when external forces are present, and mean reciprocal first-passage times are discussed briefly, and finally it is shown how finite times of observation modify the mean first-passage time formulae of free Brownian motion.

Download Full-text

Markov chain sensitivity measured by mean first passage times

Linear Algebra and its Applications ◽

10.1016/s0024-3795(99)00263-3 ◽

2000 ◽

Vol 316 (1-3) ◽

pp. 21-28 ◽

Cited By ~ 46

Author(s):

Grace E. Cho ◽

Carl D. Meyer

Keyword(s):

Markov Chain ◽

First Passage Times ◽

First Passage ◽

Mean First Passage Times ◽

Passage Times

Download Full-text

THE DISTRIBUTION OF MIXING TIMES IN MARKOV CHAINS

Asia Pacific Journal of Operational Research ◽

10.1142/s0217595912500455 ◽

2013 ◽

Vol 30 (01) ◽

pp. 1250045 ◽

Cited By ~ 1

Author(s):

JEFFREY J. HUNTER

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Linear Algebra ◽

Mixing Time ◽

Probability Generating Function ◽

First Passage Times ◽

Mixing Times ◽

First Passage ◽

Special Cases ◽

Passage Times

The distribution of the "mixing time" or the "time to stationarity" in a discrete time irreducible Markov chain, starting in state i, can be defined as the number of trials to reach a state sampled from the stationary distribution of the Markov chain. Expressions for the probability generating function, and hence the probability distribution of the mixing time, starting in state i, are derived and special cases explored. This extends the results of the author regarding the expected time to mixing [Hunter, JJ (2006). Mixing times with applications to perturbed Markov chains. Linear Algebra and Its Applications, 417, 108–123] and the variance of the times to mixing, [Hunter, JJ (2008). Variances of first passage times in a Markov chain with applications to mixing times. Linear Algebra and Its Applications, 429, 1135–1162]. Some new results for the distribution of the recurrence and the first passage times in a general irreducible three-state Markov chain are also presented.

Download Full-text