Computing approximate stationary distributions for discrete Markov processes with banded infinitesimal generators

We develop an algorithm for computing approximations to the stationary distribution of a discrete birth-and-death process, provided that the infinitesimal generator is a banded matrix. We begin by computing stationary distributions for processes whose infinitesimal generators are Hessenberg. Our derivation in this special case is different from the classical case but it leads to the same result. We then show how to extend these ideas to processes where the infinitesimal generator is banded (or half-banded) and to quasi-birth–death processes. Finally, we give an example of the application of this method to a nearly completely decomposable Markov chain to demonstrate the general applicability of the technique.

Download Full-text

On Truncation of the Matrix-Geometric Stationary Distributions

Mathematics ◽

10.3390/math7090798 ◽

2019 ◽

Vol 7 (9) ◽

pp. 798 ◽

Cited By ~ 1

Author(s):

Naumov ◽

Gaidamaka ◽

Samouylov

Keyword(s):

Finite Number ◽

Stationary Distribution ◽

Queueing Systems ◽

Death Process ◽

Birth And Death Process ◽

Loss System ◽

Stationary Distributions ◽

Open Network ◽

The Matrix

In this paper, we study queueing systems with an infinite and finite number of waiting places that can be modeled by a Quasi-Birth-and-Death process. We derive the conditions under which the stationary distribution for a loss system is a truncation of the stationary distribution of the Quasi-Birth-and-Death process and obtain the stationary distributions of both processes. We apply the obtained results to the analysis of a semi-open network in which a customer from an external queue replaces a customer leaving the system at the node from which the latter departed.

Download Full-text

Quasi-stationary distributions and one-dimensional circuit-switched networks

Journal of Applied Probability ◽

10.1017/s0021900200116821 ◽

1987 ◽

Vol 24 (04) ◽

pp. 965-977 ◽

Cited By ~ 4

Author(s):

Ilze Ziedins

Keyword(s):

Communication Networks ◽

Stationary Distribution ◽

Death Process ◽

Birth And Death Process ◽

Stationary Distributions ◽

Switched Networks ◽

One Dimensional ◽

Special Cases ◽

Quasi Stationary Distribution

We discuss the quasi-stationary distribution obtained when a simple birth and death process is conditioned on never exceeding K. An application of this model to one-dimensional circuit-switched communication networks is described, and some special cases examined.

Download Full-text

Ramaswami's duality and probabilistic algorithms for determining the rate matrix for a structured GI/M/1 Markov chain

The ANZIAM Journal ◽

10.1017/s1446181100009639 ◽

2005 ◽

Vol 46 (4) ◽

pp. 485-493

Author(s):

Emma Hunt

Keyword(s):

Markov Chain ◽

Fundamental Matrix ◽

Efficient Algorithms ◽

Death Process ◽

Birth And Death Process ◽

Probabilistic Algorithms ◽

Rate Matrix

AbstractWe show that Algorithm H* for the determination of the rate matrix of a block-GI/M/1 Markov chain is related by duality to Algorithm H for the determination of the fundamental matrix of a block-M/G/1 Markov chain. Duality is used to generate some efficient algorithms for finding the rate matrix in a quasi-birth-and-death process.

Download Full-text

Markov-modulated single-server queueing systems

Journal of Applied Probability ◽

10.1017/s0021900200107053 ◽

1994 ◽

Vol 31 (A) ◽

pp. 169-184

Author(s):

N. U. Prabhu ◽

L. C Tang

Keyword(s):

Markov Chain ◽

Queueing Systems ◽

Classical Case ◽

Single Server ◽

Discrete Time Markov Chain ◽

Joint Distributions ◽

Markov Random Walk ◽

Countable State ◽

Markov Modulated ◽

Special Case

We consider single-server queueing systems that are modulated by a discrete-time Markov chain on a countable state space. The underlying stochastic process is a Markov random walk (MRW) whose increments can be expressed as differences between service times and interarrival times. We derive the joint distributions of the waiting and idle times in the presence of the modulating Markov chain. Our approach is based on properties of the ladder sets associated with this MRW and its time-reversed counterpart. The special case of a Markov-modulated M/M/1 queueing system is then analysed and results analogous to the classical case are obtained.

Download Full-text

Quasi-stationary distributions of birth-and-death processes

Advances in Applied Probability ◽

10.1017/s0001867800031050 ◽

1978 ◽

Vol 10 (03) ◽

pp. 570-586 ◽

Cited By ~ 21

Author(s):

James A. Cavender

Keyword(s):

State Space ◽

Stationary Distribution ◽

Parameter Family ◽

Death Process ◽

Birth And Death Process ◽

Stationary Distributions ◽

Birth And Death Processes ◽

Finite State ◽

Finite State Space ◽

Quasi Stationary Distribution

Letqn(t) be the conditioned probability of finding a birth-and-death process in statenat timet,given that absorption into state 0 has not occurred by then. A family {q1(t),q2(t), · · ·} that is constant in time is a quasi-stationary distribution. If any exist, the quasi-stationary distributions comprise a one-parameter family related to quasi-stationary distributions of finite state-space approximations to the process.

Download Full-text

Quasi-stationary distributions and one-dimensional circuit-switched networks

Journal of Applied Probability ◽

10.2307/3214219 ◽

1987 ◽

Vol 24 (4) ◽

pp. 965-977 ◽

Cited By ~ 15

Author(s):

Ilze Ziedins

Keyword(s):

Communication Networks ◽

Stationary Distribution ◽

Death Process ◽

Birth And Death Process ◽

Stationary Distributions ◽

Switched Networks ◽

One Dimensional ◽

Special Cases ◽

Quasi Stationary Distribution

We discuss the quasi-stationary distribution obtained when a simple birth and death process is conditioned on never exceeding K. An application of this model to one-dimensional circuit-switched communication networks is described, and some special cases examined.

Download Full-text

Level phase independence for GI/M/1-type Markov chains

Journal of Applied Probability ◽

10.1239/jap/1014843078 ◽

2000 ◽

Vol 37 (4) ◽

pp. 984-998 ◽

Cited By ~ 6

Author(s):

Guy Latouche ◽

P. G. Taylor

Keyword(s):

Phase Transitions ◽

Markov Chains ◽

State Space ◽

Transition Probabilities ◽

The Other ◽

One Dimension ◽

Death Process ◽

Two Dimensional ◽

Birth And Death Process ◽

Special Case

GI/M/1-type Markov chains make up a class of two-dimensional Markov chains. One dimension is usually called the level, and the other is often called the phase. Transitions from states in level k are restricted to states in levels less than or equal to k+1. For given transition probabilities in the interior of the state space, we show that it is always possible to define the boundary transition probabilities in such a way that the level and phase are independent under the stationary distribution. We motivate our analysis by first considering the quasi-birth-and-death process special case in which transitions from any state are restricted to states in the same, or adjacent, levels.

Download Full-text

Quasi-stationary distributions of birth-and-death processes

Advances in Applied Probability ◽

10.2307/1426635 ◽

1978 ◽

Vol 10 (3) ◽

pp. 570-586 ◽

Cited By ~ 46

Author(s):

James A. Cavender

Keyword(s):

State Space ◽

Stationary Distribution ◽

Parameter Family ◽

Death Process ◽

Birth And Death Process ◽

Stationary Distributions ◽

Birth And Death Processes ◽

Finite State ◽

Finite State Space ◽

Quasi Stationary Distribution

Let qn(t) be the conditioned probability of finding a birth-and-death process in state n at time t, given that absorption into state 0 has not occurred by then. A family {q1(t), q2(t), · · ·} that is constant in time is a quasi-stationary distribution. If any exist, the quasi-stationary distributions comprise a one-parameter family related to quasi-stationary distributions of finite state-space approximations to the process.

Download Full-text

The recurrence and transience of two-dimensional linear birth and death processes

Advances in Applied Probability ◽

10.2307/1426423 ◽

1980 ◽

Vol 12 (3) ◽

pp. 615-639 ◽

Cited By ~ 2

Author(s):

J. Hutton

Keyword(s):

Markov Chain ◽

Linear Functions ◽

Death Process ◽

Two Dimensional ◽

Continuous Time Markov Chain ◽

Birth And Death Process ◽

Positive Probability ◽

Birth And Death Processes ◽

Imbedded Markov Chain ◽

Transient Case

A two-dimensional linear birth and death process is a continuous-time Markov chain Y(·) with state space (Z+)2 which can jump from the point (n, m) to one of its four neighbors, with rates that are linear functions of n and m. Criteria are extended for determining whether such a process has a positive probability or zero probability of escaping to infinity. In the transient case considered, the projections of the imbedded Markov chain {Xn} of the successive states visited by Y(·) on a suitable pair of orthonormal vectors v and w are shown to be regularly varying sequences with index 1. Specifically, (Xn, v)∽δn and (Xn, w)∽ kn/log n for positive constants δ and k.

Download Full-text