scholarly journals Subgeometric Ergodicity Analysis of Continuous-Time Markov Chains under Random-Time State-Dependent Lyapunov Drift Conditions

2014 ◽  
Vol 2014 ◽  
pp. 1-5
Author(s):  
Mokaedi V. Lekgari
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Random Time ◽  
Lyapunov Analysis ◽  
Continuous Time Markov Chains ◽  
State Dependent ◽  
Drift Conditions

We investigate random-time state-dependent Foster-Lyapunov analysis on subgeometric rate ergodicity of continuous-time Markov chains (CTMCs). We are mainly concerned with making use of the available results on deterministic state-dependent drift conditions for CTMCs and on random-time state-dependent drift conditions for discrete-time Markov chains and transferring them to CTMCs.

Markov chains and generalized continued fractions

1992 ◽  
Vol 29 (04) ◽  
pp. 838-849 ◽  
Author(s):  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Wide Class ◽  
Stochastic Models ◽  
Continuous Time ◽  
Continued Fractions ◽  
Invariant Measures ◽  
Continuous Time Markov Chains ◽  
Generalized Continued Fractions

This paper deals with a class of discrete-time Markov chains for which the invariant measures can be expressed in terms of generalized continued fractions. The representation covers a wide class of stochastic models and is well suited for numerical applications. The results obtained can easily be extended to continuous-time Markov chains.

Approximating Transition Probabilities and Mean Occupation Times in Continuous-Time Markov Chains

1987 ◽  
Vol 1 (3) ◽  
pp. 251-264 ◽  
Author(s):  
Sheldon M. Ross
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Random Time ◽  
Occupation Times ◽  
New Approach ◽  
Continuous Time Markov Chains ◽  
The Mean ◽  
Mean Time

In this paper we propose a new approach for estimating the transition probabilities and mean occupation times of continuous-time Markov chains. Our approach is to approximate the probability of being in a state (or the mean time already spent in a state) at time t by the probability of being in that state (or the mean time already spent in that state) at a random time that is gamma distributed with mean t.

Conditions for strong ergodicity using intensity matrices

1988 ◽  
Vol 25 (1) ◽  
pp. 34-42 ◽  
Author(s):  
Jean Johnson ◽  
Dean Isaacson
Keyword(s):  
Markov Chains ◽  
Rate Of Convergence ◽  
Discrete Time ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Limiting Behavior ◽  
Strong Ergodicity ◽  
Continuous Time Markov Chains

Sufficient conditions for strong ergodicity of discrete-time non-homogeneous Markov chains have been given in several papers. Conditions have been given using the left eigenvectors ψn of Pn(ψ nPn = ψ n) and also using the limiting behavior of Pn. In this paper we consider the analogous results in the case of continuous-time Markov chains where one uses the intensity matrices Q(t) instead of P(s, t). A bound on the rate of convergence of certain strongly ergodic chains is also given.

Markov chains and generalized continued fractions

1992 ◽  
Vol 29 (4) ◽  
pp. 838-849 ◽  
Author(s):  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Wide Class ◽  
Stochastic Models ◽  
Continuous Time ◽  
Continued Fractions ◽  
Invariant Measures ◽  
Continuous Time Markov Chains ◽  
Generalized Continued Fractions

This paper deals with a class of discrete-time Markov chains for which the invariant measures can be expressed in terms of generalized continued fractions. The representation covers a wide class of stochastic models and is well suited for numerical applications. The results obtained can easily be extended to continuous-time Markov chains.

Discrete time methods for simulating continuous time Markov chains

1976 ◽  
Vol 8 (04) ◽  
pp. 772-788 ◽  
Author(s):  
Arie Hordijk ◽  
Donald L. Iglehart ◽  
Rolf Schassberger
Keyword(s):  
Central Limit Theorem ◽  
Limit Theorem ◽  
Markov Chains ◽  
Confidence Intervals ◽  
Discrete Time ◽  
Continuous Time ◽  
Statistical Efficiency ◽  
Numerical Examples ◽  
The Central Limit Theorem ◽  
Continuous Time Markov Chains

This paper discusses several problems which arise when the regenerative method is used to analyse simulations of Markov chains. The first problem involves calculating the variance constant which appears in the central limit theorem used to obtain confidence intervals. Knowledge of this constant is very helpful in evaluating simulation methodologies. The second problem is to devise a method for simulating continuous time Markov chains without having to generate exponentially distributed holding times. Several methods are presented and compared. Numerical examples are given to illustrate the computional and statistical efficiency of these methods.

scholarly journals Subgeometric Ergodicity under Random-Time State-Dependent Drift Conditions

2014 ◽  
Vol 2014 ◽  
pp. 1-5
Author(s):  
Mokaedi V. Lekgari
Keyword(s):  
Markov Chains ◽  
Stochastic Networks ◽  
Random Time ◽  
Stopping Times ◽  
Lyapunov Techniques ◽  
State Dependent ◽  
Drift Conditions ◽  
The Stability

Motivated by possible applications of Lyapunov techniques in the stability of stochastic networks, subgeometric ergodicity of Markov chains is investigated. In a nutshell, in this study we take a look atf-ergodic general Markov chains, subgeometrically ergodic at rater, when the random-time Foster-Lyapunov drift conditions on a set of stopping times are satisfied.

Strong ergodicity for continuous-time Markov chains

1978 ◽  
Vol 15 (4) ◽  
pp. 699-706 ◽  
Author(s):  
Dean Isaacson ◽  
Barry Arnold
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Strong Ergodicity ◽  
Continuous Time Markov Chains

The concept of strong ergodicity for discrete-time homogeneous Markov chains has been characterized in several ways (Dobrushin (1956), Lin (1975), Isaacson and Tweedie (1978)). In this paper the characterization using mean visit times (Huang and Isaacson (1977)) is extended to continuous-time Markov chains. From this it follows that for a certain subclass of continuous-time Markov chains, X(t), is strongly ergodic if and only if the associated embedded chain is Cesaro strongly ergodic.

scholarly journals Networks—B

2021 ◽  
pp. 93-113
Author(s):  
Jean Walrand
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Time Reversal ◽  
Continuous Time ◽  
Queueing Networks ◽  
Product Form ◽  
Continuous Time Markov Chains

AbstractThis chapter provides the derivations of the results in the previous chapter. It also develops the theory of continuous-time Markov chains.Section 6.1 proves the results on the spreading of rumors. Section 6.2 presents the theory of continuous-time Markov chains that are used to model queueing networks, among many other applications. That section explains the relationships between continuous-time and related discrete-time Markov chains. Sections 6.3 and 6.4 prove the results about product-form networks by using a time-reversal argument.

Discrete time methods for simulating continuous time Markov chains

1976 ◽  
Vol 8 (4) ◽  
pp. 772-788 ◽  
Author(s):  
Arie Hordijk ◽  
Donald L. Iglehart ◽  
Rolf Schassberger
Keyword(s):  
Central Limit Theorem ◽  
Limit Theorem ◽  
Markov Chains ◽  
Confidence Intervals ◽  
Discrete Time ◽  
Continuous Time ◽  
Statistical Efficiency ◽  
Numerical Examples ◽  
The Central Limit Theorem ◽  
Continuous Time Markov Chains

This paper discusses several problems which arise when the regenerative method is used to analyse simulations of Markov chains. The first problem involves calculating the variance constant which appears in the central limit theorem used to obtain confidence intervals. Knowledge of this constant is very helpful in evaluating simulation methodologies. The second problem is to devise a method for simulating continuous time Markov chains without having to generate exponentially distributed holding times. Several methods are presented and compared. Numerical examples are given to illustrate the computional and statistical efficiency of these methods.

Sign in / Sign up

Export Citation Format

Share Document