Hazard rate and reversed hazard rate monotonicities in continuous-time Markov chains

A continuous-time Markov chain on the non-negative integers is called skip-free to the right (left) if only unit increments to the right (left) are permitted. If a Markov chain is skip-free both to the right and to the left, it is called a birth–death process. Karlin and McGregor (1959) showed that if a continuous-time Markov chain is monotone in the sense of likelihood ratio ordering then it must be an (extended) birth–death process. This paper proves that if an irreducible Markov chain in continuous time is monotone in the sense of hazard rate (reversed hazard rate) ordering then it must be skip-free to the right (left). A birth–death process is then characterized as a continuous-time Markov chain that is monotone in the sense of both hazard rate and reversed hazard rate orderings. As an application, the first-passage-time distributions of such Markov chains are also studied.

Download Full-text

Quasi-limiting distributions of Markov chains that are skip-free to the left in continuous time

Journal of Applied Probability ◽

10.2307/3214761 ◽

1993 ◽

Vol 30 (3) ◽

pp. 509-517 ◽

Cited By ~ 18

Author(s):

Masaaki Kijima

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

Death Process ◽

Continuous Time Markov Chain ◽

Limiting Distributions ◽

Finite Approximations ◽

Death Case ◽

The Right ◽

Birth Death

A continuous-time Markov chain on the non-negative integers is called skip-free to the left (right) if the governing infinitesimal generator A = (aij) has the property that aij = 0 for j ≦ i ‒ 2 (i ≦ j – 2). If a Markov chain is skip-free both to the left and to the right, it is called a birth-death process. Quasi-limiting distributions of birth–death processes have been studied in detail in their own right and from the standpoint of finite approximations. In this paper, we generalize, to some extent, results for birth-death processes to Markov chains that are skip-free to the left in continuous time. In particular the decay parameter of skip-free Markov chains is shown to have a similar representation to the birth-death case and a result on convergence of finite quasi-limiting distributions is obtained.

Download Full-text

Quasi-limiting distributions of Markov chains that are skip-free to the left in continuous time

Journal of Applied Probability ◽

10.1017/s0021900200044260 ◽

1993 ◽

Vol 30 (03) ◽

pp. 509-517 ◽

Cited By ~ 6

Author(s):

Masaaki Kijima

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

Death Process ◽

Continuous Time Markov Chain ◽

Limiting Distributions ◽

Finite Approximations ◽

Death Case ◽

The Right ◽

Birth Death

A continuous-time Markov chain on the non-negative integers is called skip-free to the left (right) if the governing infinitesimal generator A = (aij ) has the property that aij = 0 for j ≦ i ‒ 2 (i ≦ j – 2). If a Markov chain is skip-free both to the left and to the right, it is called a birth-death process. Quasi-limiting distributions of birth–death processes have been studied in detail in their own right and from the standpoint of finite approximations. In this paper, we generalize, to some extent, results for birth-death processes to Markov chains that are skip-free to the left in continuous time. In particular the decay parameter of skip-free Markov chains is shown to have a similar representation to the birth-death case and a result on convergence of finite quasi-limiting distributions is obtained.

Download Full-text

On passage and conditional passage times for Markov chains in continuous time

Journal of Applied Probability ◽

10.1017/s0021900200040924 ◽

1988 ◽

Vol 25 (02) ◽

pp. 279-290 ◽

Cited By ~ 1

Author(s):

Masaaki Kijima

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

First Passage Time ◽

Passage Time ◽

Death Process ◽

First Passage ◽

Reversible Markov Chain ◽

Irreducible Markov Chain ◽

Passage Times

Let X(t) be a temporally homogeneous irreducible Markov chain in continuous time defined on . For k < i < j, let H = {k + 1, ···, j − 1} and let kTij ( jTik ) be the upward (downward) conditional first-passage time of X(t) from i to j(k) given no visit to . These conditional passage times are studied through first-passage times of a modified chain HX(t) constructed by making the set of states absorbing. It will be shown that the densities of kTij and jTik for any birth-death process are unimodal and the modes kmij ( jmik ) of the unimodal densities are non-increasing (non-decreasing) with respect to i. Some distribution properties of kTij and jTik for a time-reversible Markov chain are presented. Symmetry among kTij, jTik , and is also discussed, where , and are conditional passage times of the reversed process of X(t).

Download Full-text

THE DEVIATION MATRIX OF A CONTINUOUS-TIME MARKOV CHAIN

Probability in the Engineering and Informational Sciences ◽

10.1017/s0269964802163066 ◽

2002 ◽

Vol 16 (3) ◽

pp. 351-366 ◽

Cited By ~ 48

Author(s):

Pauline Coolen-Schrijner ◽

Erik A. van Doorn

Keyword(s):

Markov Chain ◽

Continuous Time ◽

Transition Probability ◽

Transition Probability Matrix ◽

Death Process ◽

Continuous Time Markov Chain ◽

Deviation Matrix ◽

The Matrix ◽

Birth Death

The deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix P(·) and ergodic matrix Π is the matrix D ≡ ∫0∞(P(t) − Π) dt. We give conditions for D to exist and discuss properties and a representation of D. The deviation matrix of a birth–death process is investigated in detail. We also describe a new application of deviation matrices by showing that a measure for the convergence to stationarity of a stochastically increasing Markov chain can be expressed in terms of the elements of the deviation matrix of the chain.

Download Full-text

On passage and conditional passage times for Markov chains in continuous time

Journal of Applied Probability ◽

10.2307/3214436 ◽

1988 ◽

Vol 25 (2) ◽

pp. 279-290 ◽

Cited By ~ 3

Author(s):

Masaaki Kijima

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

First Passage Time ◽

Passage Time ◽

Death Process ◽

First Passage ◽

Reversible Markov Chain ◽

Irreducible Markov Chain ◽

Passage Times

Let X(t) be a temporally homogeneous irreducible Markov chain in continuous time defined on . For k < i < j, let H = {k + 1, ···, j − 1} and let kTij (jTik) be the upward (downward) conditional first-passage time of X(t) from i to j(k) given no visit to . These conditional passage times are studied through first-passage times of a modified chain HX(t) constructed by making the set of states absorbing. It will be shown that the densities of kTij and jTik for any birth-death process are unimodal and the modes kmij (jmik) of the unimodal densities are non-increasing (non-decreasing) with respect to i. Some distribution properties of kTij and jTik for a time-reversible Markov chain are presented. Symmetry among kTij, jTik, and is also discussed, where , and are conditional passage times of the reversed process of X(t).

Download Full-text

A Numerical Approach for Evaluating the Time-Dependent Distribution of a Quasi Birth-Death Process

Methodology And Computing In Applied Probability ◽

10.1007/s11009-021-09882-6 ◽

2021 ◽

Author(s):

Michel Mandjes ◽

Birgit Sollie

Keyword(s):

Continuous Time ◽

Real Life ◽

Numerical Approach ◽

Time Dependent ◽

Death Process ◽

Continuous Time Markov Chain ◽

Likelihood Estimator ◽

Real Life Data ◽

The Matrix ◽

Birth Death

AbstractThis paper considers a continuous-time quasi birth-death (qbd) process, which informally can be seen as a birth-death process of which the parameters are modulated by an external continuous-time Markov chain. The aim is to numerically approximate the time-dependent distribution of the resulting bivariate Markov process in an accurate and efficient way. An approach based on the Erlangization principle is proposed and formally justified. Its performance is investigated and compared with two existing approaches: one based on numerical evaluation of the matrix exponential underlying the qbd process, and one based on the uniformization technique. It is shown that in many settings the approach based on Erlangization is faster than the other approaches, while still being highly accurate. In the last part of the paper, we demonstrate the use of the developed technique in the context of the evaluation of the likelihood pertaining to a time series, which can then be optimized over its parameters to obtain the maximum likelihood estimator. More specifically, through a series of examples with simulated and real-life data, we show how it can be deployed in model selection problems that involve the choice between a qbd and its non-modulated counterpart.

Download Full-text

Lumpability and marginalisability for continuous-time Markov chains

Journal of Applied Probability ◽

10.2307/3214762 ◽

1993 ◽

Vol 30 (3) ◽

pp. 518-528 ◽

Cited By ~ 31

Author(s):

Frank Ball ◽

Geoffrey F. Yeo

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Ion Channel ◽

Continuous Time ◽

Sufficient Conditions ◽

Continuous Time Markov Chain ◽

Continuous Time Markov Chains ◽

Probabilistic Proof ◽

Necessary And Sufficient ◽

Mutually Independent

We consider lumpability for continuous-time Markov chains and provide a simple probabilistic proof of necessary and sufficient conditions for strong lumpability, valid in circumstances not covered by known theory. We also consider the following marginalisability problem. Let {X{t)} = {(X1(t), X2(t), · ··, Xm(t))} be a continuous-time Markov chain. Under what conditions are the marginal processes {X1(t)}, {X2(t)}, · ··, {Xm(t)} also continuous-time Markov chains? We show that this is related to lumpability and, if no two of the marginal processes can jump simultaneously, then they are continuous-time Markov chains if and only if they are mutually independent. Applications to ion channel modelling and birth–death processes are discussed briefly.

Download Full-text

Quasi-ergodicity for non-homogeneous continuous-time Markov chains

Journal of Applied Probability ◽

10.2307/3214422 ◽

1989 ◽

Vol 26 (3) ◽

pp. 643-648 ◽

Cited By ~ 7

Author(s):

A. I. Zeifman

Keyword(s):

Differential Equation ◽

Markov Chain ◽

Markov Chains ◽

State Space ◽

Continuous Time ◽

Sufficient Conditions ◽

Continuous Time Markov Chain ◽

Countable State Space ◽

Continuous Time Markov Chains ◽

Countable State

We consider a non-homogeneous continuous-time Markov chain X(t) with countable state space. Definitions of uniform and strong quasi-ergodicity are introduced. The forward Kolmogorov system for X(t) is considered as a differential equation in the space of sequences l1. Sufficient conditions for uniform quasi-ergodicity are deduced from this equation. We consider conditions of uniform and strong ergodicity in the case of proportional intensities.

Download Full-text

A Note on Approximating Mean Occupation Times of Continuous-Time Markov Chains

Probability in the Engineering and Informational Sciences ◽

10.1017/s0269964800000796 ◽

1988 ◽

Vol 2 (2) ◽

pp. 267-268

Author(s):

Sheldon M. Ross

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

Transition Probabilities ◽

Random Variables ◽

Continuous Time Markov Chain ◽

Occupation Times ◽

Continuous Time Markov Chains ◽

Exponential Random Variables

In [1] an approach to approximate the transition probabilities and mean occupation times of a continuous-time Markov chain is presented. For the chain under consideration, let Pij(t) and Tij(t) denote respectively the probability that it is in state j at time t, and the total time spent in j by time t, in both cases conditional on the chain starting in state i. Also, let Y1,…, Yn be independent exponential random variables each with rate λ = n/t, which are also independent of the Markov chain.

Download Full-text