On the Problem of Establishing the Existence of Stationary Distributions for Continuous-Time Markov Chains

1993 ◽  
Vol 7 (4) ◽  
pp. 529-543 ◽  
Author(s):  
P. K. Pollett ◽  
P. G. Taylor
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Sufficient Condition ◽  
Transition Rates ◽  
Necessary And Sufficient Condition ◽  
Stationary Distributions ◽  
Continuous Time Markov Chains ◽  
Necessary And Sufficient ◽  
Networks Of Queues

We consider the problem of establishing the existence of stationary distributions for continuous-time Markov chains directly from the transition rates Q. Given an invariant probability distribution m for Q, we show that a necessary and sufficient condition for m to be a stationary distribution for the minimal process is that Q be regular. We provide sufficient conditions for the regularity of Q that are simple to verify in practice, thus allowing one to easily identify stationary distributions for a variety of models. To illustrate our results, we shall consider three classes of multidimensional Markov chains, namely, networks of queues with batch movements, semireversible queues, and partially balanced Markov processes.

Markovian couplings staying in arbitrary subsets of the state space

2002 ◽  
Vol 39 (01) ◽  
pp. 197-212 ◽  
Author(s):  
F. Javier López ◽  
Gerardo Sanz
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Transition Rates ◽  
Arbitrary Subset ◽  
State Spaces ◽  
Continuous Time Markov Chains ◽  
Countable State ◽  
Necessary And Sufficient

Let (X t ) and (Y t ) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (X t ) and (Y t ) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.

Markovian couplings staying in arbitrary subsets of the state space

2002 ◽  
Vol 39 (1) ◽  
pp. 197-212 ◽  
Author(s):  
F. Javier López ◽  
Gerardo Sanz
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Transition Rates ◽  
Arbitrary Subset ◽  
State Spaces ◽  
Continuous Time Markov Chains ◽  
Countable State ◽  
Necessary And Sufficient

Let (Xt) and (Yt) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (Xt) and (Yt) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.

Invariant measures for Q-processes when Q is not regular

1991 ◽  
Vol 23 (02) ◽  
pp. 277-292 ◽  
Author(s):  
P. K. Pollett
Keyword(s):  
Invariant Measure ◽  
Continuous Time ◽  
Invariant Measures ◽  
Sufficient Conditions ◽  
Sufficient Condition ◽  
Transition Rates ◽  
Major Result ◽  
Continuous Time Markov Chains ◽  
Simple Sufficient Condition ◽  
Necessary And Sufficient

The problem of determining invariant measures for continuous-time Markov chains directly from their transition rates is considered. The major result provides necessary and sufficient conditions for the existence of a unique ‘single-exit' chain with a specified invariant measure. This generalizes a result of Hou Chen-Ting and Chen Mufa that deals with symmetrically reversible chains. A simple sufficient condition for the existence of a unique honest chain for which the specified measure is invariant is also presented.

Invariant measures for Q-processes when Q is not regular

1991 ◽  
Vol 23 (2) ◽  
pp. 277-292 ◽  
Author(s):  
P. K. Pollett
Keyword(s):  
Invariant Measure ◽  
Continuous Time ◽  
Invariant Measures ◽  
Sufficient Conditions ◽  
Sufficient Condition ◽  
Transition Rates ◽  
Major Result ◽  
Continuous Time Markov Chains ◽  
Simple Sufficient Condition ◽  
Necessary And Sufficient

The problem of determining invariant measures for continuous-time Markov chains directly from their transition rates is considered. The major result provides necessary and sufficient conditions for the existence of a unique ‘single-exit' chain with a specified invariant measure. This generalizes a result of Hou Chen-Ting and Chen Mufa that deals with symmetrically reversible chains. A simple sufficient condition for the existence of a unique honest chain for which the specified measure is invariant is also presented.

On the relationship between µ-invariant measures and quasi-stationary distributions for continuous-time Markov chains

1993 ◽  
Vol 25 (01) ◽  
pp. 82-102
Author(s):  
M. G. Nair ◽  
P. K. Pollett
Keyword(s):  
Invariant Measure ◽  
Stationary Distribution ◽  
Continuous Time ◽  
Sufficient Condition ◽  
Transition Rates ◽  
Necessary And Sufficient Condition ◽  
Absorbing State ◽  
Necessary And Sufficient ◽  
Forward Equations ◽  
Quasi Stationary Distribution

In a recent paper, van Doorn (1991) explained how quasi-stationary distributions for an absorbing birth-death process could be determined from the transition rates of the process, thus generalizing earlier work of Cavender (1978). In this paper we shall show that many of van Doorn's results can be extended to deal with an arbitrary continuous-time Markov chain over a countable state space, consisting of an irreducible class, C, and an absorbing state, 0, which is accessible from C. Some of our results are extensions of theorems proved for honest chains in Pollett and Vere-Jones (1992). In Section 3 we prove that a probability distribution on C is a quasi-stationary distribution if and only if it is a µ-invariant measure for the transition function, P. We shall also show that if m is a quasi-stationary distribution for P, then a necessary and sufficient condition for m to be µ-invariant for Q is that P satisfies the Kolmogorov forward equations over C. When the remaining forward equations hold, the quasi-stationary distribution must satisfy a set of ‘residual equations' involving the transition rates into the absorbing state. The residual equations allow us to determine the value of µ for which the quasi-stationary distribution is µ-invariant for P. We also prove some more general results giving bounds on the values of µ for which a convergent measure can be a µ-subinvariant and then µ-invariant measure for P. The remainder of the paper is devoted to the question of when a convergent µ-subinvariant measure, m, for Q is a quasi-stationary distribution. Section 4 establishes a necessary and sufficient condition for m to be a quasi-stationary distribution for the minimal chain. In Section 5 we consider ‘single-exit' chains. We derive a necessary and sufficient condition for there to exist a process for which m is a quasi-stationary distribution. Under this condition all such processes can be specified explicitly through their resolvents. The results proved here allow us to conclude that the bounds for µ obtained in Section 3 are, in fact, tight. Finally, in Section 6, we illustrate our results by way of two examples: regular birth-death processes and a pure-birth process with absorption.

Lumpability and marginalisability for continuous-time Markov chains

1993 ◽  
Vol 30 (3) ◽  
pp. 518-528 ◽  
Author(s):  
Frank Ball ◽  
Geoffrey F. Yeo
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Ion Channel ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Continuous Time Markov Chains ◽  
Probabilistic Proof ◽  
Necessary And Sufficient ◽  
Mutually Independent

We consider lumpability for continuous-time Markov chains and provide a simple probabilistic proof of necessary and sufficient conditions for strong lumpability, valid in circumstances not covered by known theory. We also consider the following marginalisability problem. Let {X{t)} = {(X1(t), X2(t), · ··, Xm(t))} be a continuous-time Markov chain. Under what conditions are the marginal processes {X1(t)}, {X2(t)}, · ··, {Xm(t)} also continuous-time Markov chains? We show that this is related to lumpability and, if no two of the marginal processes can jump simultaneously, then they are continuous-time Markov chains if and only if they are mutually independent. Applications to ion channel modelling and birth–death processes are discussed briefly.

scholarly journals Positifitas dan Ketercapaian Sistem Linier Fractional Waktu Kontinu

CAUCHY ◽  
2011 ◽  
Vol 2 (1) ◽  
pp. 18
Author(s):  
Imam Fahcruddin
Keyword(s):  
Continuous Time ◽  
Sufficient Conditions ◽  
Sufficient Condition ◽  
Positive Systems ◽  
Necessary And Sufficient Condition ◽  
Fractional Systems ◽  
Necessary And Sufficient

<div class="standard"><a id="magicparlabel-2384">This paper studies a solution of the fractional continuous-time linier system. Necessary and sufficient condition were established for the internal and external positivity of fractional systems. Sufficient conditions are given for the reachability of fractional positive systems. </a></div>

On the identification of continuous-time Markov chains with a given invariant measure

1994 ◽  
Vol 31 (04) ◽  
pp. 897-910
Author(s):  
P. K. Pollett
Keyword(s):  
Invariant Measure ◽  
Continuous Time ◽  
Branching Process ◽  
Sufficient Condition ◽  
Necessary And Sufficient Condition ◽  
Birth Process ◽  
Continuous Time Markov Chains ◽  
Branching Process With Immigration ◽  
Birth Processes ◽  
Necessary And Sufficient

In [14] a necessary and sufficient condition was obtained for there to exist uniquely a Q-process with a specified invariant measure, under the assumption that Q is a stable, conservative, single-exit matrix. The purpose of this note is to demonstrate that, for an arbitrary stable and conservative q-matrix, the same condition suffices for the existence of a suitable Q-process, but that this process might not be unique. A range of examples is considered, including pure-birth processes, a birth process with catastrophes, birth-death processes and the Markov branching process with immigration.

Stochastic domination and Markovian couplings

2000 ◽  
Vol 32 (4) ◽  
pp. 1064-1076 ◽  
Author(s):  
F. Javier López ◽  
Servet Martínez ◽  
Gerardo Sanz
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Partially Ordered Set ◽  
Ordered Set ◽  
Sufficient Conditions ◽  
Stochastic Domination ◽  
Jackson Networks ◽  
Partially Ordered ◽  
Continuous Time Markov Chains ◽  
Necessary And Sufficient

For continuous-time Markov chains with semigroups P, P' taking values in a partially ordered set, such that P ≤ stP', we show the existence of an order-preserving Markovian coupling and give a way to construct it. From our proof, we also obtain the conditions of Brandt and Last for stochastic domination in terms of the associated intensity matrices. Our result is applied to get necessary and sufficient conditions for the existence of Markovian couplings between two Jackson networks.

On the relationship between µ-invariant measures and quasi-stationary distributions for continuous-time Markov chains

1993 ◽  
Vol 25 (1) ◽  
pp. 82-102 ◽  
Author(s):  
M. G. Nair ◽  
P. K. Pollett
Keyword(s):  
Invariant Measure ◽  
Stationary Distribution ◽  
Continuous Time ◽  
Sufficient Condition ◽  
Transition Rates ◽  
Necessary And Sufficient Condition ◽  
Absorbing State ◽  
Necessary And Sufficient ◽  
Forward Equations ◽  
Quasi Stationary Distribution

In a recent paper, van Doorn (1991) explained how quasi-stationary distributions for an absorbing birth-death process could be determined from the transition rates of the process, thus generalizing earlier work of Cavender (1978). In this paper we shall show that many of van Doorn's results can be extended to deal with an arbitrary continuous-time Markov chain over a countable state space, consisting of an irreducible class, C, and an absorbing state, 0, which is accessible from C. Some of our results are extensions of theorems proved for honest chains in Pollett and Vere-Jones (1992).In Section 3 we prove that a probability distribution on C is a quasi-stationary distribution if and only if it is a µ-invariant measure for the transition function, P. We shall also show that if m is a quasi-stationary distribution for P, then a necessary and sufficient condition for m to be µ-invariant for Q is that P satisfies the Kolmogorov forward equations over C. When the remaining forward equations hold, the quasi-stationary distribution must satisfy a set of ‘residual equations' involving the transition rates into the absorbing state. The residual equations allow us to determine the value of µ for which the quasi-stationary distribution is µ-invariant for P. We also prove some more general results giving bounds on the values of µ for which a convergent measure can be a µ-subinvariant and then µ-invariant measure for P. The remainder of the paper is devoted to the question of when a convergent µ-subinvariant measure, m, for Q is a quasi-stationary distribution. Section 4 establishes a necessary and sufficient condition for m to be a quasi-stationary distribution for the minimal chain. In Section 5 we consider ‘single-exit' chains. We derive a necessary and sufficient condition for there to exist a process for which m is a quasi-stationary distribution. Under this condition all such processes can be specified explicitly through their resolvents. The results proved here allow us to conclude that the bounds for µ obtained in Section 3 are, in fact, tight. Finally, in Section 6, we illustrate our results by way of two examples: regular birth-death processes and a pure-birth process with absorption.

Sign in / Sign up

Export Citation Format

Share Document