scholarly journals THE DISTRIBUTION OF MIXING TIMES IN MARKOV CHAINS

2013 ◽  
Vol 30 (01) ◽  
pp. 1250045 ◽  
Author(s):  
JEFFREY J. HUNTER
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Linear Algebra ◽  
Mixing Time ◽  
Probability Generating Function ◽  
First Passage Times ◽  
Mixing Times ◽  
First Passage ◽  
Special Cases ◽  
Passage Times

The distribution of the "mixing time" or the "time to stationarity" in a discrete time irreducible Markov chain, starting in state i, can be defined as the number of trials to reach a state sampled from the stationary distribution of the Markov chain. Expressions for the probability generating function, and hence the probability distribution of the mixing time, starting in state i, are derived and special cases explored. This extends the results of the author regarding the expected time to mixing [Hunter, JJ (2006). Mixing times with applications to perturbed Markov chains. Linear Algebra and Its Applications, 417, 108–123] and the variance of the times to mixing, [Hunter, JJ (2008). Variances of first passage times in a Markov chain with applications to mixing times. Linear Algebra and Its Applications, 429, 1135–1162]. Some new results for the distribution of the recurrence and the first passage times in a general irreducible three-state Markov chain are also presented.

scholarly journals SIMPLE PROCEDURES FOR FINDING MEAN FIRST PASSAGE TIMES IN MARKOV CHAINS

2007 ◽  
Vol 24 (06) ◽  
pp. 813-829 ◽  
Author(s):  
JEFFREY J. HUNTER
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Linear Equations ◽  
First Passage Times ◽  
First Passage ◽  
Mean First Passage Times ◽  
The Mean ◽  
Computational Procedures ◽  
Passage Times

The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell's fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse of I - P + eπT where eT = (1, 1, …, 1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures.

scholarly journals Accurate calculations of Stationary Distributions and Mean First Passage Times in Markov Renewal Processes and Markov Chains

2016 ◽  
Vol 4 (1) ◽  
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Markov Chain ◽  
First Passage Times ◽  
Embedded Markov Chain ◽  
Test Problems ◽  
Markov Renewal Process ◽  
First Passage ◽  
Mean First Passage Times ◽  
Special Cases ◽  
Accurate Procedure ◽  
Passage Times

AbstractThis article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30, 197–207, (1986) procedure. The technique is numerically stable in that it doesn’t involve subtractions. Algebraic expressions for the special cases of one, two, three and four states are derived.Aconsequence of the procedure is that the stationary distribution of the embedded Markov chain does not need to be derived in advance but can be found accurately from the derived mean first passage times. MatLab is utilized to carry out the computations, using some test problems from the literature.

Spectral Theory for Skip-Free Markov Chains

1989 ◽  
Vol 3 (1) ◽  
pp. 77-88 ◽  
Author(s):  
Joseph Abate ◽  
Ward Whitt
Keyword(s):  
Markov Chains ◽  
Spectral Theory ◽  
First Passage Time ◽  
Passage Time ◽  
First Passage Times ◽  
Simple Relationship ◽  
First Passage ◽  
Birth And Death Processes ◽  
The Laplace Transform ◽  
Passage Times

The distribution of upward first passage times in skip-free Markov chains can be expressed solely in terms of the eigenvalues in the spectral representation, without performing a separate calculation to determine the eigenvectors. We provide insight into this result and skip-free Markov chains more generally by showing that part of the spectral theory developed for birth-and-death processes extends to skip-free chains. We show that the eigenvalues and eigenvectors of skip-free chains can be characterized in terms of recursively defined polynomials. Moreover, the Laplace transform of the upward first passage time from 0 to n is the reciprocal of the nth polynomial. This simple relationship holds because the Laplace transforms of the first passage times satisfy the same recursion as the polynomials except for a normalization.

Geometric Markov chains

1995 ◽  
Vol 32 (02) ◽  
pp. 349-374
Author(s):  
William Rising
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Nearest Neighbor ◽  
First Passage Times ◽  
First Passage ◽  
Death Rates ◽  
Passage Times ◽  
Birth Death

A generalization of the familiar birth–death chain, called the geometric chain, is introduced and explored. By the introduction of two families of parameters in addition to the infinitesimal birth and death rates, the geometric chain allows transitions beyond the nearest neighbor, but is shown to retain the simple computational formulas of the birth–death chain for the stationary distribution and the expected first-passage times between states. It is also demonstrated that even when not reversible, a reversed geometric chain is again a geometric chain.

scholarly journals First Passage and Collective Marks

2019 ◽  
Vol 8 (6) ◽  
pp. 47
Author(s):  
Yiping Zhang ◽  
Myron Hlynka ◽  
Percy H. Brill
Keyword(s):  
Markov Chains ◽  
Generating Functions ◽  
First Passage Times ◽  
System Of Equations ◽  
First Passage ◽  
Probability Generating Functions ◽  
Collective Marks ◽  
Passage Times

Probability generating functions for first passage times of Markov chains are found using the method of collective marks. A system of equations is found which can be used to obtain moments of the first passage times. Second passage probabilities are discussed.

scholarly journals First Passage Times for Markov Additive Processes with Positive Jumps of Phase Type

2008 ◽  
Vol 45 (03) ◽  
pp. 779-799 ◽  
Author(s):  
Lothar Breuer
Keyword(s):  
Inverse Function ◽  
First Passage Times ◽  
Phase Type Distribution ◽  
First Passage ◽  
Additive Processes ◽  
Special Cases ◽  
Phase Type ◽  
Spectrally Negative Lévy Processes ◽  
Passage Times ◽  
Markov Additive Processes

The present paper generalises some results for spectrally negative Lévy processes to the setting of Markov additive processes (MAPs). A prominent role is assumed by the first passage times, which will be determined in terms of their Laplace transforms. These have the form of a phase-type distribution, with a rate matrix that can be regarded as an inverse function of the cumulant matrix. A numerically stable iteration to compute this matrix is given. The theory is first developed for MAPs without positive jumps and then extended to include positive jumps having phase-type distributions. Numerical and analytical examples show agreement with existing results in special cases.

Sign in / Sign up

Export Citation Format

Share Document