Spectral structure of the first-passage-time densities for classes of Markov chains

1987 ◽  
Vol 24 (3) ◽  
pp. 631-643 ◽  
Author(s):  
Masaaki Kijima
Keyword(s):  
Markov Chains ◽  
First Passage Time ◽  
Passage Time ◽  
Death Process ◽  
Spectral Structure ◽  
First Passage ◽  
Completely Monotone ◽  
Spectral Representations ◽  
Absorbing States ◽  
Birth Death

Keilson [7] showed that for a birth-death process defined on non-negative integers with reflecting barrier at 0 the first-passage-time density from 0 to N (N to N + 1) has Pólya frequency of order infinity (is completely monotone). Brown and Chaganty [3] and Assaf et al. [1] studied the first-passage-time distribution for classes of discrete-time Markov chains and then produced the essentially same results as these through a uniformization. This paper addresses itself to an extension of Keilson's results to classes of Markov chains such as time-reversible Markov chains, skip-free Markov chains and birth-death processes with absorbing states. The extensions are due to the spectral representations of the infinitesimal generators governing these Markov chains. Explicit densities for those first-passage times are also given.

Spectral structure of the first-passage-time densities for classes of Markov chains

1987 ◽  
Vol 24 (03) ◽  
pp. 631-643 ◽  
Author(s):  
Masaaki Kijima
Keyword(s):  
Markov Chains ◽  
First Passage Time ◽  
Passage Time ◽  
Death Process ◽  
Spectral Structure ◽  
First Passage ◽  
Completely Monotone ◽  
Spectral Representations ◽  
Absorbing States ◽  
Birth Death

Keilson [7] showed that for a birth-death process defined on non-negative integers with reflecting barrier at 0 the first-passage-time density from 0 to N (N to N + 1) has Pólya frequency of order infinity (is completely monotone). Brown and Chaganty [3] and Assaf et al. [1] studied the first-passage-time distribution for classes of discrete-time Markov chains and then produced the essentially same results as these through a uniformization. This paper addresses itself to an extension of Keilson's results to classes of Markov chains such as time-reversible Markov chains, skip-free Markov chains and birth-death processes with absorbing states. The extensions are due to the spectral representations of the infinitesimal generators governing these Markov chains. Explicit densities for those first-passage times are also given.

On passage and conditional passage times for Markov chains in continuous time

1988 ◽  
Vol 25 (02) ◽  
pp. 279-290 ◽  
Author(s):  
Masaaki Kijima
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
First Passage Time ◽  
Passage Time ◽  
Death Process ◽  
First Passage ◽  
Reversible Markov Chain ◽  
Irreducible Markov Chain ◽  
Passage Times

Let X(t) be a temporally homogeneous irreducible Markov chain in continuous time defined on . For k < i < j, let H = {k + 1, ···, j − 1} and let kTij ( jTik ) be the upward (downward) conditional first-passage time of X(t) from i to j(k) given no visit to . These conditional passage times are studied through first-passage times of a modified chain HX(t) constructed by making the set of states absorbing. It will be shown that the densities of kTij and jTik for any birth-death process are unimodal and the modes kmij ( jmik ) of the unimodal densities are non-increasing (non-decreasing) with respect to i. Some distribution properties of kTij and jTik for a time-reversible Markov chain are presented. Symmetry among kTij, jTik , and is also discussed, where , and are conditional passage times of the reversed process of X(t).

On conditional passage time structure of birth-death processes

1984 ◽  
Vol 21 (01) ◽  
pp. 10-21 ◽  
Author(s):  
Ushio Sumita
Keyword(s):  
Queueing System ◽  
First Passage Time ◽  
Arrival Rate ◽  
Passage Time ◽  
Death Process ◽  
Service Rate ◽  
Transition Rates ◽  
First Passage ◽  
Limiting Behavior ◽  
Birth Death

Let N(t) be a birth-death process on N = {0,1,2,· ··} governed by the transition rates λ n > 0 (n ≧ 0) and μ η > 0 (n ≧ 1). Let mTm be the conditional first-passage time from r to n, given no visit to m where m <r < n. The downward conditional first-passage time nTm is defined similarly. It will be shown that , for any λ n > 0 and μ η > 0. The limiting behavior of is considerably different from that of the ordinary first-passage time where, under certain conditions, exponentiality sets in as n →∞. We will prove that, when λ n → λ > 0 and μ η → μ > 0 as n → ∞with ρ = λ /μ < 1, one has as r → ∞where T BP(λ,μ) is the server busy period of an M/M/1 queueing system with arrival rate λand service rate μ.

On conditional passage time structure of birth-death processes

1984 ◽  
Vol 21 (1) ◽  
pp. 10-21 ◽  
Author(s):  
Ushio Sumita
Keyword(s):  
Queueing System ◽  
First Passage Time ◽  
Arrival Rate ◽  
Passage Time ◽  
Death Process ◽  
Service Rate ◽  
Transition Rates ◽  
First Passage ◽  
Limiting Behavior ◽  
Birth Death

Let N(t) be a birth-death process on N = {0,1,2,· ··} governed by the transition rates λn > 0 (n ≧ 0) and μ η > 0 (n ≧ 1). Let mTm be the conditional first-passage time from r to n, given no visit to m where m <r < n. The downward conditional first-passage time nTm is defined similarly. It will be shown that , for any λn > 0 and μ η > 0. The limiting behavior of is considerably different from that of the ordinary first-passage time where, under certain conditions, exponentiality sets in as n →∞. We will prove that, when λn → λ > 0 and μ η → μ > 0 as n → ∞with ρ = λ /μ < 1, one has as r → ∞where TBP(λ,μ) is the server busy period of an M/M/1 queueing system with arrival rate λand service rate μ.

On limiting behavior of ordinary and conditional first-passage times for a class of birth-death processes

1987 ◽  
Vol 24 (1) ◽  
pp. 235-240 ◽  
Author(s):  
U. Sumita
Keyword(s):  
First Passage Time ◽  
Arrival Rate ◽  
Passage Time ◽  
First Passage Times ◽  
Death Process ◽  
Service Rate ◽  
First Passage ◽  
Limiting Behavior ◽  
Passage Times ◽  
Birth Death

Let N(t) be a birth-death process on 𝒩= {0, 1, 2, ·· ·} governed by the transition rates λn > 0 (n ≧ 0) and μn > 0 (n ≧ 1) where λn → λ > 0 and μn → μ > 0 as n → ∞ and ρ = λ/μ. Let Tmn be the first-passage time of N(t) from m to n and define It is shown that, when converges in distribution to TBP(μ,λ) as n → ∞ where TΒΡ (μ,λ) is the server busy period of an M/M/1 queueing system with arrival rate μ and service rate λ. Correspondingly T0n/E[T0n] converges to 1 with probability 1 as n →∞. Of related interest is the conditional first-passage time mTrn of N(t) from r to n given no visit to m where m < r < n. As we shall see, the conditional first-passage time of N(t) can be viewed as an ordinary first-passage time of a modified birth-death process M(t) governed by where are generated from λn and μn. Furthermore it is shown that for and while for and This enables one to establish the relation between the limiting behavior of the ordinary first-passage times and that of the conditional first-passage times.

On limiting behavior of ordinary and conditional first-passage times for a class of birth-death processes

1987 ◽  
Vol 24 (01) ◽  
pp. 235-240 ◽  
Author(s):  
U. Sumita
Keyword(s):  
First Passage Time ◽  
Arrival Rate ◽  
Passage Time ◽  
First Passage Times ◽  
Death Process ◽  
Service Rate ◽  
First Passage ◽  
Limiting Behavior ◽  
Passage Times ◽  
Birth Death

LetN(t) be a birth-death process on 𝒩= {0, 1, 2, ·· ·} governed by the transition ratesλn&gt; 0 (n≧ 0) andμn&gt; 0 (n≧ 1) whereλn→λ&gt; 0 andμn→μ&gt; 0 asn→ ∞ andρ=λ/μ. LetTmnbe the first-passage time ofN(t) frommtonand defineIt is shown that, whenconverges in distribution toTBP(μ,λ)asn → ∞whereTΒΡ (μ,λ)is the server busy period of anM/M/1 queueing system with arrival rateμand service rateλ. CorrespondinglyT0n/E[T0n] converges to 1 with probability 1 asn→∞. Of related interest is the conditional first-passage timemTrnofN(t) from r tongiven no visit tomwherem &lt; r &lt; n.As we shall see, the conditional first-passage time ofN(t) can be viewed as an ordinary first-passage time of a modified birth-death processM(t) governed bywhereare generated fromλnandμn. Furthermore it is shown that forandwhile forandThis enables one to establish the relation between the limiting behavior of the ordinary first-passage times and that of the conditional first-passage times.

On passage and conditional passage times for Markov chains in continuous time

1988 ◽  
Vol 25 (2) ◽  
pp. 279-290 ◽  
Author(s):  
Masaaki Kijima
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
First Passage Time ◽  
Passage Time ◽  
Death Process ◽  
First Passage ◽  
Reversible Markov Chain ◽  
Irreducible Markov Chain ◽  
Passage Times

Let X(t) be a temporally homogeneous irreducible Markov chain in continuous time defined on . For k < i < j, let H = {k + 1, ···, j − 1} and let kTij (jTik) be the upward (downward) conditional first-passage time of X(t) from i to j(k) given no visit to . These conditional passage times are studied through first-passage times of a modified chain HX(t) constructed by making the set of states absorbing. It will be shown that the densities of kTij and jTik for any birth-death process are unimodal and the modes kmij (jmik) of the unimodal densities are non-increasing (non-decreasing) with respect to i. Some distribution properties of kTij and jTik for a time-reversible Markov chain are presented. Symmetry among kTij, jTik, and is also discussed, where , and are conditional passage times of the reversed process of X(t).

Spectral Theory for Skip-Free Markov Chains

1989 ◽  
Vol 3 (1) ◽  
pp. 77-88 ◽  
Author(s):  
Joseph Abate ◽  
Ward Whitt
Keyword(s):  
Markov Chains ◽  
Spectral Theory ◽  
First Passage Time ◽  
Passage Time ◽  
First Passage Times ◽  
Simple Relationship ◽  
First Passage ◽  
Birth And Death Processes ◽  
The Laplace Transform ◽  
Passage Times

The distribution of upward first passage times in skip-free Markov chains can be expressed solely in terms of the eigenvalues in the spectral representation, without performing a separate calculation to determine the eigenvectors. We provide insight into this result and skip-free Markov chains more generally by showing that part of the spectral theory developed for birth-and-death processes extends to skip-free chains. We show that the eigenvalues and eigenvectors of skip-free chains can be characterized in terms of recursively defined polynomials. Moreover, the Laplace transform of the upward first passage time from 0 to n is the reciprocal of the nth polynomial. This simple relationship holds because the Laplace transforms of the first passage times satisfy the same recursion as the polynomials except for a normalization.

Joint distributions of random variables and their integrals for certain birth-death and diffusion processes

1971 ◽  
Vol 3 (02) ◽  
pp. 339-352 ◽  
Author(s):  
J. Gani ◽  
D. R. Mcneil
Keyword(s):  
Diffusion Processes ◽  
First Passage Time ◽  
Integral Functional ◽  
Passage Time ◽  
Death Process ◽  
Traffic Jam ◽  
Joint Distributions ◽  
The Cost ◽  
And Storage ◽  
Birth Death

For the linear growth birth-death process with parameters λ n = nλ, μ n = nμ, Puri ((1966), (1968)) has investigated the joint distribution of the number X(t) of survivors in the process and the associated integral Y(t) = ∫0 t X(τ)dτ. In particular, he has obtained limiting results as t → ∞. Recently one of us (McNeil (1970)) has derived the distribution of the integral functional W x = ∫0 Tx g{X(τ)}dτ, where T x is the first passage time to the origin in a general birth-death process with X(0) = x and g(·) is an arbitrary function. Functionals of the form W x arise naturally in traffic and storage theory; for example W x may represent the total cost of a traffic jam, or the cost of storing a commodity until expiration of the stock. Moments of such functionals were found in the case of M/G/1 and GI/M/1 queues by Gaver (1969) and Daley (1969).

scholarly journals Sojourn time distributions in a Markovian G-queue with batch arrival and batch removal

1999 ◽  
Vol 12 (4) ◽  
pp. 339-356 ◽  
Author(s):  
Yang Woo Shin
Keyword(s):  
Markov Chains ◽  
Sojourn Time ◽  
First Passage Time ◽  
Passage Time ◽  
Batch Arrival ◽  
Single Server ◽  
Negative Customers ◽  
First Passage ◽  
Markovian Queue ◽  
Sojourn Time Distributions

We consider a single server Markovian queue with two types of customers; positive and negative, where positive customers arrive in batches and arrivals of negative customers remove positive customers in batches. Only positive customers form a queue and negative customers just reduce the system congestion by removing positive ones upon their arrivals. We derive the LSTs of sojourn time distributions for a single server Markovian queue with positive customers and negative customers by using the first passage time arguments for Markov chains.

Sign in / Sign up

Export Citation Format

Share Document