Computing mean first passage times for a Markov chain

1995 ◽  
Vol 26 (5) ◽  
pp. 729-735 ◽  
Author(s):  
Theodore J. Sheskin
Keyword(s):  
Markov Chain ◽  
First Passage Times ◽  
First Passage ◽  
Mean First Passage Times ◽  
Passage Times

scholarly journals SIMPLE PROCEDURES FOR FINDING MEAN FIRST PASSAGE TIMES IN MARKOV CHAINS

2007 ◽  
Vol 24 (06) ◽  
pp. 813-829 ◽  
Author(s):  
JEFFREY J. HUNTER
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Linear Equations ◽  
First Passage Times ◽  
First Passage ◽  
Mean First Passage Times ◽  
The Mean ◽  
Computational Procedures ◽  
Passage Times

The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell's fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse of I - P + eπT where eT = (1, 1, …, 1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures.

scholarly journals Accurate calculations of Stationary Distributions and Mean First Passage Times in Markov Renewal Processes and Markov Chains

2016 ◽  
Vol 4 (1) ◽  
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Markov Chain ◽  
First Passage Times ◽  
Embedded Markov Chain ◽  
Test Problems ◽  
Markov Renewal Process ◽  
First Passage ◽  
Mean First Passage Times ◽  
Special Cases ◽  
Accurate Procedure ◽  
Passage Times

AbstractThis article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30, 197–207, (1986) procedure. The technique is numerically stable in that it doesn’t involve subtractions. Algebraic expressions for the special cases of one, two, three and four states are derived.Aconsequence of the procedure is that the stationary distribution of the embedded Markov chain does not need to be derived in advance but can be found accurately from the derived mean first passage times. MatLab is utilized to carry out the computations, using some test problems from the literature.

scholarly journals Correlation functions, mean first passage times, and the Kemeny constant

2020 ◽  
Vol 152 (10) ◽  
pp. 104108 ◽  
Author(s):  
Adam Kells ◽  
Vladimir Koskin ◽  
Edina Rosta ◽  
Alessia Annibale
Keyword(s):  
Correlation Functions ◽  
First Passage Times ◽  
First Passage ◽  
Mean First Passage Times ◽  
Passage Times

scholarly journals RaTrav: a tool for calculating mean first-passage times on biochemical networks

2013 ◽  
Vol 7 (1) ◽  
pp. 130 ◽  
Author(s):  
Mieczyslaw Torchala ◽  
Przemyslaw Chelminiak ◽  
Michal Kurzynski ◽  
Paul A Bates
Keyword(s):  
First Passage Times ◽  
Biochemical Networks ◽  
First Passage ◽  
Mean First Passage Times ◽  
Passage Times
Sign in / Sign up

Export Citation Format

Share Document