scholarly journals Property-Driven State-Space Coarsening for Continuous Time Markov Chains

2016 ◽  
pp. 3-18 ◽  
Author(s):  
Michalis Michaelides ◽  
Dimitrios Milios ◽  
Jane Hillston ◽  
Guido Sanguinetti
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Continuous Time Markov Chains

scholarly journals Two Approaches to the Construction of Perturbation Bounds for Continuous-Time Markov Chains

Mathematics ◽  
2020 ◽  
Vol 8 (2) ◽  
pp. 253 ◽  
Author(s):  
Alexander Zeifman ◽  
Victor Korolev ◽  
Yacov Satin
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Countable State Space ◽  
Perturbation Bounds ◽  
Queuing Models ◽  
Continuous Time Markov Chains ◽  
Quantitative Estimates ◽  
Countable State

This paper is largely a review. It considers two main methods used to study stability and to obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable state space. An approach is described to the construction of perturbation estimates for the main five classes of such chains associated with queuing models. Several specific models are considered for which the limit characteristics and perturbation bounds for admissible “perturbed” processes are calculated.

Quasi-ergodicity for non-homogeneous continuous-time Markov chains

1989 ◽  
Vol 26 (3) ◽  
pp. 643-648 ◽  
Author(s):  
A. I. Zeifman
Keyword(s):  
Differential Equation ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

We consider a non-homogeneous continuous-time Markov chain X(t) with countable state space. Definitions of uniform and strong quasi-ergodicity are introduced. The forward Kolmogorov system for X(t) is considered as a differential equation in the space of sequences l1. Sufficient conditions for uniform quasi-ergodicity are deduced from this equation. We consider conditions of uniform and strong ergodicity in the case of proportional intensities.

Markovian couplings staying in arbitrary subsets of the state space

2002 ◽  
Vol 39 (01) ◽  
pp. 197-212 ◽  
Author(s):  
F. Javier López ◽  
Gerardo Sanz
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Transition Rates ◽  
Arbitrary Subset ◽  
State Spaces ◽  
Continuous Time Markov Chains ◽  
Countable State ◽  
Necessary And Sufficient

Let (X t ) and (Y t ) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (X t ) and (Y t ) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.

scholarly journals Path integrals for continuous-time Markov chains

2002 ◽  
Vol 39 (4) ◽  
pp. 901-904 ◽  
Author(s):  
P. K. Pollett ◽  
V. T. Stefanov
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Path Integral ◽  
Continuous Time ◽  
Path Integrals ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

This note presents a method of evaluating the distribution of a path integral for Markov chains on a countable state space.

Markovian couplings staying in arbitrary subsets of the state space

2002 ◽  
Vol 39 (1) ◽  
pp. 197-212 ◽  
Author(s):  
F. Javier López ◽  
Gerardo Sanz
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Transition Rates ◽  
Arbitrary Subset ◽  
State Spaces ◽  
Continuous Time Markov Chains ◽  
Countable State ◽  
Necessary And Sufficient

Let (Xt) and (Yt) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (Xt) and (Yt) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.

Quasi-ergodicity for non-homogeneous continuous-time Markov chains

1989 ◽  
Vol 26 (03) ◽  
pp. 643-648 ◽  
Author(s):  
A. I. Zeifman
Keyword(s):  
Differential Equation ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

We consider a non-homogeneous continuous-time Markov chain X(t) with countable state space. Definitions of uniform and strong quasi-ergodicity are introduced. The forward Kolmogorov system for X(t) is considered as a differential equation in the space of sequences l 1 . Sufficient conditions for uniform quasi-ergodicity are deduced from this equation. We consider conditions of uniform and strong ergodicity in the case of proportional intensities.

scholarly journals Path integrals for continuous-time Markov chains

2002 ◽  
Vol 39 (04) ◽  
pp. 901-904 ◽  
Author(s):  
P. K. Pollett ◽  
V. T. Stefanov
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Path Integral ◽  
Continuous Time ◽  
Path Integrals ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

This note presents a method of evaluating the distribution of a path integral for Markov chains on a countable state space.

The expected time until absorption when absorption is not certain

1998 ◽  
Vol 35 (04) ◽  
pp. 812-823 ◽  
Author(s):  
D. M. Walker
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Death Process ◽  
Birth And Death Process ◽  
Expected Time ◽  
Absorbing State ◽  
Continuous Time Markov Chains ◽  
Birth Death

This paper considers continuous-time Markov chains whose state space consists of an irreducible class, 𝒞, and an absorbing state which is accessible from 𝒞. The purpose is to provide a way to determine the expected time to absorption conditional on such time being finite, in the case where absorption occurs with probability less than 1. The results are illustrated by applications to the general birth and death process and the linear birth, death and catastrophe process.

Sign in / Sign up

Export Citation Format

Share Document