scholarly journals A Note on Error Bounds for Approximating Transition Probabilities in Continuous-time Markov Chains

1988 ◽  
Vol 2 (4) ◽  
pp. 471-474 ◽  
Author(s):  
Nico M. van Dijk
Keyword(s):  
Markov Chains ◽  
Error Bound ◽  
Error Bounds ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Practical Interest ◽  
Occupation Times ◽  
Continuous Time Markov Chains ◽  
Elegant Method ◽  
Comparison Relation

Recently, Ross [1] proposed an elegant method of approximating transition probabilities and mean occupation times in continuous-time Markov chains based upon recursively inspecting the process at exponential times. The method turned out to be amazingly efficient for the examples investigated. However, no formal rough error bound was provided. Any error bound even though robust is of practical interest in engineering (e.g., for determining truncation criteria or setting up an experiment). This note primarily aims to show that by a simple and standard comparison relation a rough error bound of the method is secured. Also, some alternative approximations are inspected.

Approximating Transition Probabilities and Mean Occupation Times in Continuous-Time Markov Chains

1987 ◽  
Vol 1 (3) ◽  
pp. 251-264 ◽  
Author(s):  
Sheldon M. Ross
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Random Time ◽  
Occupation Times ◽  
New Approach ◽  
Continuous Time Markov Chains ◽  
The Mean ◽  
Mean Time

In this paper we propose a new approach for estimating the transition probabilities and mean occupation times of continuous-time Markov chains. Our approach is to approximate the probability of being in a state (or the mean time already spent in a state) at time t by the probability of being in that state (or the mean time already spent in that state) at a random time that is gamma distributed with mean t.

A Note on Approximating Mean Occupation Times of Continuous-Time Markov Chains

1988 ◽  
Vol 2 (2) ◽  
pp. 267-268
Author(s):  
Sheldon M. Ross
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Random Variables ◽  
Continuous Time Markov Chain ◽  
Occupation Times ◽  
Continuous Time Markov Chains ◽  
Exponential Random Variables

In [1] an approach to approximate the transition probabilities and mean occupation times of a continuous-time Markov chain is presented. For the chain under consideration, let Pij(t) and Tij(t) denote respectively the probability that it is in state j at time t, and the total time spent in j by time t, in both cases conditional on the chain starting in state i. Also, let Y1,…, Yn be independent exponential random variables each with rate λ = n/t, which are also independent of the Markov chain.

A description of the long-term behaviour of absorbing continuous-time Markov chains using a centre manifold

1990 ◽  
Vol 22 (1) ◽  
pp. 111-128 ◽  
Author(s):  
P. K. Pollett ◽  
A. J. Roberts
Keyword(s):  
Differential Equations ◽  
Markov Chains ◽  
Invariant Manifold ◽  
Markov Processes ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Centre Manifold ◽  
Continuous Time Markov Chains ◽  
Linear Differential

We use the notion of an invariant manifold to describe the long-term behaviour of absorbing continuous-time Markov processes with a denumerable infinity of states. We show that there exists an invariant manifold for the forward differential equations and we are able to describe the evolution of the state probabilities on this manifold. Our approach gives rise to a new method for calculating conditional limiting distributions, one which is also appropriate for dealing with processes whose transition probabilities satisfy a system of non-linear differential equations.

A description of the long-term behaviour of absorbing continuous-time Markov chains using a centre manifold

1990 ◽  
Vol 22 (01) ◽  
pp. 111-128 ◽  
Author(s):  
P. K. Pollett ◽  
A. J. Roberts
Keyword(s):  
Differential Equations ◽  
Markov Chains ◽  
Invariant Manifold ◽  
Markov Processes ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Centre Manifold ◽  
Continuous Time Markov Chains ◽  
Linear Differential

We use the notion of an invariant manifold to describe the long-term behaviour of absorbing continuous-time Markov processes with a denumerable infinity of states. We show that there exists an invariant manifold for the forward differential equations and we are able to describe the evolution of the state probabilities on this manifold. Our approach gives rise to a new method for calculating conditional limiting distributions, one which is also appropriate for dealing with processes whose transition probabilities satisfy a system of non-linear differential equations.

An Error-Bound Theorem for Approximate Markov Chains

1992 ◽  
Vol 6 (3) ◽  
pp. 413-424 ◽  
Author(s):  
Nico M. Van Dijk
Keyword(s):  
Steady State ◽  
Markov Chains ◽  
Error Bound ◽  
Error Bounds ◽  
Practical Interest ◽  
Approximate Model ◽  
Communication Model ◽  
Uniform Bound ◽  
Bound Theorem

Recently an error-bound theorem was reported to conclude analytic error bounds for approximate Markov chains. The theorem required a uniform bound for marginal expectations of the approximate model. This note will relax this bound to steady state rather than marginal expectations as of practical interest(i) to simplify verification and/or(ii) to obtain a more accurate error bound.A communication model is studied in detail to support the results.

Sign in / Sign up

Export Citation Format

Share Document