On passage and conditional passage times for Markov chains in continuous time

Let X(t) be a temporally homogeneous irreducible Markov chain in continuous time defined on . For k < i < j, let H = {k + 1, ···, j − 1} and let kTij ( jTik ) be the upward (downward) conditional first-passage time of X(t) from i to j(k) given no visit to . These conditional passage times are studied through first-passage times of a modified chain HX(t) constructed by making the set of states absorbing. It will be shown that the densities of kTij and jTik for any birth-death process are unimodal and the modes kmij ( jmik ) of the unimodal densities are non-increasing (non-decreasing) with respect to i. Some distribution properties of kTij and jTik for a time-reversible Markov chain are presented. Symmetry among kTij, jTik , and is also discussed, where , and are conditional passage times of the reversed process of X(t).

Download Full-text

First-passage-time moments of Markov processes

Journal of Applied Probability ◽

10.2307/3213962 ◽

1985 ◽

Vol 22 (4) ◽

pp. 939-945 ◽

Cited By ~ 13

Author(s):

David D. Yao

Keyword(s):

Markov Chains ◽

Markov Processes ◽

Continuous Time ◽

Fundamental Matrix ◽

Generalized Inverse ◽

First Passage Time ◽

Passage Time ◽

First Passage ◽

Continuous Time Markov Chains ◽

Passage Times

We consider the first-passage times of continuous-time Markov chains. Based on the approach of generalized inverse, moments of all orders are derived and expressed in simple, explicit forms in terms of the ‘fundamental matrix'. The formulas are new and are also efficient for computation.

Download Full-text

First-passage-time moments of Markov processes

Journal of Applied Probability ◽

10.1017/s0021900200108186 ◽

1985 ◽

Vol 22 (04) ◽

pp. 939-945

Author(s):

David D. Yao

Keyword(s):

Markov Chains ◽

Markov Processes ◽

Continuous Time ◽

Fundamental Matrix ◽

Generalized Inverse ◽

First Passage Time ◽

Passage Time ◽

First Passage ◽

Continuous Time Markov Chains ◽

Passage Times

We consider the first-passage times of continuous-time Markov chains. Based on the approach of generalized inverse, moments of all orders are derived and expressed in simple, explicit forms in terms of the ‘fundamental matrix'. The formulas are new and are also efficient for computation.

Download Full-text

Spectral Theory for Skip-Free Markov Chains

Probability in the Engineering and Informational Sciences ◽

10.1017/s026996480000098x ◽

1989 ◽

Vol 3 (1) ◽

pp. 77-88 ◽

Cited By ~ 9

Author(s):

Joseph Abate ◽

Ward Whitt

Keyword(s):

Markov Chains ◽

Spectral Theory ◽

First Passage Time ◽

Passage Time ◽

First Passage Times ◽

Simple Relationship ◽

First Passage ◽

Birth And Death Processes ◽

The Laplace Transform ◽

Passage Times

The distribution of upward first passage times in skip-free Markov chains can be expressed solely in terms of the eigenvalues in the spectral representation, without performing a separate calculation to determine the eigenvectors. We provide insight into this result and skip-free Markov chains more generally by showing that part of the spectral theory developed for birth-and-death processes extends to skip-free chains. We show that the eigenvalues and eigenvectors of skip-free chains can be characterized in terms of recursively defined polynomials. Moreover, the Laplace transform of the upward first passage time from 0 to n is the reciprocal of the nth polynomial. This simple relationship holds because the Laplace transforms of the first passage times satisfy the same recursion as the polynomials except for a normalization.

Download Full-text

Hazard rate and reversed hazard rate monotonicities in continuous-time Markov chains

Journal of Applied Probability ◽

10.1239/jap/1032265203 ◽

1998 ◽

Vol 35 (3) ◽

pp. 545-556 ◽

Cited By ~ 11

Author(s):

Masaaki Kijima

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

Hazard Rate ◽

First Passage Time ◽

Death Process ◽

Continuous Time Markov Chain ◽

Reversed Hazard Rate ◽

The Right ◽

Birth Death

A continuous-time Markov chain on the non-negative integers is called skip-free to the right (left) if only unit increments to the right (left) are permitted. If a Markov chain is skip-free both to the right and to the left, it is called a birth–death process. Karlin and McGregor (1959) showed that if a continuous-time Markov chain is monotone in the sense of likelihood ratio ordering then it must be an (extended) birth–death process. This paper proves that if an irreducible Markov chain in continuous time is monotone in the sense of hazard rate (reversed hazard rate) ordering then it must be skip-free to the right (left). A birth–death process is then characterized as a continuous-time Markov chain that is monotone in the sense of both hazard rate and reversed hazard rate orderings. As an application, the first-passage-time distributions of such Markov chains are also studied.

Download Full-text

Spectral structure of the first-passage-time densities for classes of Markov chains

Journal of Applied Probability ◽

10.1017/s0021900200031363 ◽

1987 ◽

Vol 24 (03) ◽

pp. 631-643 ◽

Cited By ~ 4

Author(s):

Masaaki Kijima

Keyword(s):

Markov Chains ◽

First Passage Time ◽

Passage Time ◽

Death Process ◽

Spectral Structure ◽

First Passage ◽

Completely Monotone ◽

Spectral Representations ◽

Absorbing States ◽

Birth Death

Keilson [7] showed that for a birth-death process defined on non-negative integers with reflecting barrier at 0 the first-passage-time density from 0 to N (N to N + 1) has Pólya frequency of order infinity (is completely monotone). Brown and Chaganty [3] and Assaf et al. [1] studied the first-passage-time distribution for classes of discrete-time Markov chains and then produced the essentially same results as these through a uniformization. This paper addresses itself to an extension of Keilson's results to classes of Markov chains such as time-reversible Markov chains, skip-free Markov chains and birth-death processes with absorbing states. The extensions are due to the spectral representations of the infinitesimal generators governing these Markov chains. Explicit densities for those first-passage times are also given.

Download Full-text

Spectral structure of the first-passage-time densities for classes of Markov chains

Journal of Applied Probability ◽

10.2307/3214095 ◽

1987 ◽

Vol 24 (3) ◽

pp. 631-643 ◽

Cited By ~ 10

Author(s):

Masaaki Kijima

Keyword(s):

Markov Chains ◽

First Passage Time ◽

Passage Time ◽

Death Process ◽

Spectral Structure ◽

First Passage ◽

Completely Monotone ◽

Spectral Representations ◽

Absorbing States ◽

Birth Death

Keilson [7] showed that for a birth-death process defined on non-negative integers with reflecting barrier at 0 the first-passage-time density from 0 to N (N to N + 1) has Pólya frequency of order infinity (is completely monotone). Brown and Chaganty [3] and Assaf et al. [1] studied the first-passage-time distribution for classes of discrete-time Markov chains and then produced the essentially same results as these through a uniformization. This paper addresses itself to an extension of Keilson's results to classes of Markov chains such as time-reversible Markov chains, skip-free Markov chains and birth-death processes with absorbing states. The extensions are due to the spectral representations of the infinitesimal generators governing these Markov chains. Explicit densities for those first-passage times are also given.

Download Full-text

Hazard rate and reversed hazard rate monotonicities in continuous-time Markov chains

Journal of Applied Probability ◽

10.1017/s002190020001620x ◽

1998 ◽

Vol 35 (03) ◽

pp. 545-556 ◽

Cited By ~ 4

Author(s):

Masaaki Kijima

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

Hazard Rate ◽

First Passage Time ◽

Death Process ◽

Continuous Time Markov Chain ◽

Reversed Hazard Rate ◽

The Right ◽

Birth Death

A continuous-time Markov chain on the non-negative integers is called skip-free to the right (left) if only unit increments to the right (left) are permitted. If a Markov chain is skip-free both to the right and to the left, it is called a birth–death process. Karlin and McGregor (1959) showed that if a continuous-time Markov chain is monotone in the sense of likelihood ratio ordering then it must be an (extended) birth–death process. This paper proves that if an irreducible Markov chain in continuous time is monotone in the sense of hazard rate (reversed hazard rate) ordering then it must be skip-free to the right (left). A birth–death process is then characterized as a continuous-time Markov chain that is monotone in the sense of both hazard rate and reversed hazard rate orderings. As an application, the first-passage-time distributions of such Markov chains are also studied.

Download Full-text

On limiting behavior of ordinary and conditional first-passage times for a class of birth-death processes

Journal of Applied Probability ◽

10.2307/3214074 ◽

1987 ◽

Vol 24 (1) ◽

pp. 235-240 ◽

Cited By ~ 4

Author(s):

U. Sumita

Keyword(s):

First Passage Time ◽

Arrival Rate ◽

Passage Time ◽

First Passage Times ◽

Death Process ◽

Service Rate ◽

First Passage ◽

Limiting Behavior ◽

Passage Times ◽

Birth Death

Let N(t) be a birth-death process on 𝒩= {0, 1, 2, ·· ·} governed by the transition rates λn > 0 (n ≧ 0) and μn > 0 (n ≧ 1) where λn → λ > 0 and μn → μ > 0 as n → ∞ and ρ = λ/μ. Let Tmn be the first-passage time of N(t) from m to n and define It is shown that, when converges in distribution to TBP(μ,λ) as n → ∞ where TΒΡ (μ,λ) is the server busy period of an M/M/1 queueing system with arrival rate μ and service rate λ. Correspondingly T0n/E[T0n] converges to 1 with probability 1 as n →∞. Of related interest is the conditional first-passage time mTrn of N(t) from r to n given no visit to m where m < r < n. As we shall see, the conditional first-passage time of N(t) can be viewed as an ordinary first-passage time of a modified birth-death process M(t) governed by where are generated from λn and μn. Furthermore it is shown that for and while for and This enables one to establish the relation between the limiting behavior of the ordinary first-passage times and that of the conditional first-passage times.

Download Full-text

On limiting behavior of ordinary and conditional first-passage times for a class of birth-death processes

Journal of Applied Probability ◽

10.1017/s002190020003076x ◽

1987 ◽

Vol 24 (01) ◽

pp. 235-240 ◽

Cited By ~ 1

Author(s):

U. Sumita

Keyword(s):

First Passage Time ◽

Arrival Rate ◽

Passage Time ◽

First Passage Times ◽

Death Process ◽

Service Rate ◽

First Passage ◽

Limiting Behavior ◽

Passage Times ◽

Birth Death

LetN(t) be a birth-death process on 𝒩= {0, 1, 2, ·· ·} governed by the transition ratesλn> 0 (n≧ 0) andμn> 0 (n≧ 1) whereλn→λ> 0 andμn→μ> 0 asn→ ∞ andρ=λ/μ. LetTmnbe the first-passage time ofN(t) frommtonand defineIt is shown that, whenconverges in distribution toTBP(μ,λ)asn → ∞whereTΒΡ (μ,λ)is the server busy period of anM/M/1 queueing system with arrival rateμand service rateλ. CorrespondinglyT0n/E[T0n] converges to 1 with probability 1 asn→∞. Of related interest is the conditional first-passage timemTrnofN(t) from r tongiven no visit tomwherem < r < n.As we shall see, the conditional first-passage time ofN(t) can be viewed as an ordinary first-passage time of a modified birth-death processM(t) governed bywhereare generated fromλnandμn. Furthermore it is shown that forandwhile forandThis enables one to establish the relation between the limiting behavior of the ordinary first-passage times and that of the conditional first-passage times.

Download Full-text