Markov chains and generalized continued fractions

1992 ◽  
Vol 29 (04) ◽  
pp. 838-849 ◽  
Author(s):  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Wide Class ◽  
Stochastic Models ◽  
Continuous Time ◽  
Continued Fractions ◽  
Invariant Measures ◽  
Continuous Time Markov Chains ◽  
Generalized Continued Fractions

This paper deals with a class of discrete-time Markov chains for which the invariant measures can be expressed in terms of generalized continued fractions. The representation covers a wide class of stochastic models and is well suited for numerical applications. The results obtained can easily be extended to continuous-time Markov chains.

Markov chains and generalized continued fractions

1992 ◽  
Vol 29 (4) ◽  
pp. 838-849 ◽  
Author(s):  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Wide Class ◽  
Stochastic Models ◽  
Continuous Time ◽  
Continued Fractions ◽  
Invariant Measures ◽  
Continuous Time Markov Chains ◽  
Generalized Continued Fractions

This paper deals with a class of discrete-time Markov chains for which the invariant measures can be expressed in terms of generalized continued fractions. The representation covers a wide class of stochastic models and is well suited for numerical applications. The results obtained can easily be extended to continuous-time Markov chains.

Conditions for strong ergodicity using intensity matrices

1988 ◽  
Vol 25 (1) ◽  
pp. 34-42 ◽  
Author(s):  
Jean Johnson ◽  
Dean Isaacson
Keyword(s):  
Markov Chains ◽  
Rate Of Convergence ◽  
Discrete Time ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Limiting Behavior ◽  
Strong Ergodicity ◽  
Continuous Time Markov Chains

Sufficient conditions for strong ergodicity of discrete-time non-homogeneous Markov chains have been given in several papers. Conditions have been given using the left eigenvectors ψn of Pn(ψ nPn = ψ n) and also using the limiting behavior of Pn. In this paper we consider the analogous results in the case of continuous-time Markov chains where one uses the intensity matrices Q(t) instead of P(s, t). A bound on the rate of convergence of certain strongly ergodic chains is also given.

scholarly journals Subgeometric Ergodicity Analysis of Continuous-Time Markov Chains under Random-Time State-Dependent Lyapunov Drift Conditions

2014 ◽  
Vol 2014 ◽  
pp. 1-5
Author(s):  
Mokaedi V. Lekgari
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Random Time ◽  
Lyapunov Analysis ◽  
Continuous Time Markov Chains ◽  
State Dependent ◽  
Drift Conditions

We investigate random-time state-dependent Foster-Lyapunov analysis on subgeometric rate ergodicity of continuous-time Markov chains (CTMCs). We are mainly concerned with making use of the available results on deterministic state-dependent drift conditions for CTMCs and on random-time state-dependent drift conditions for discrete-time Markov chains and transferring them to CTMCs.

Discrete time methods for simulating continuous time Markov chains

1976 ◽  
Vol 8 (04) ◽  
pp. 772-788 ◽  
Author(s):  
Arie Hordijk ◽  
Donald L. Iglehart ◽  
Rolf Schassberger
Keyword(s):  
Central Limit Theorem ◽  
Limit Theorem ◽  
Markov Chains ◽  
Confidence Intervals ◽  
Discrete Time ◽  
Continuous Time ◽  
Statistical Efficiency ◽  
Numerical Examples ◽  
The Central Limit Theorem ◽  
Continuous Time Markov Chains

This paper discusses several problems which arise when the regenerative method is used to analyse simulations of Markov chains. The first problem involves calculating the variance constant which appears in the central limit theorem used to obtain confidence intervals. Knowledge of this constant is very helpful in evaluating simulation methodologies. The second problem is to devise a method for simulating continuous time Markov chains without having to generate exponentially distributed holding times. Several methods are presented and compared. Numerical examples are given to illustrate the computional and statistical efficiency of these methods.

Strong ergodicity for continuous-time Markov chains

1978 ◽  
Vol 15 (4) ◽  
pp. 699-706 ◽  
Author(s):  
Dean Isaacson ◽  
Barry Arnold
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Strong Ergodicity ◽  
Continuous Time Markov Chains

The concept of strong ergodicity for discrete-time homogeneous Markov chains has been characterized in several ways (Dobrushin (1956), Lin (1975), Isaacson and Tweedie (1978)). In this paper the characterization using mean visit times (Huang and Isaacson (1977)) is extended to continuous-time Markov chains. From this it follows that for a certain subclass of continuous-time Markov chains, X(t), is strongly ergodic if and only if the associated embedded chain is Cesaro strongly ergodic.

scholarly journals Networks—B

2021 ◽  
pp. 93-113
Author(s):  
Jean Walrand
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Time Reversal ◽  
Continuous Time ◽  
Queueing Networks ◽  
Product Form ◽  
Continuous Time Markov Chains

AbstractThis chapter provides the derivations of the results in the previous chapter. It also develops the theory of continuous-time Markov chains.Section 6.1 proves the results on the spreading of rumors. Section 6.2 presents the theory of continuous-time Markov chains that are used to model queueing networks, among many other applications. That section explains the relationships between continuous-time and related discrete-time Markov chains. Sections 6.3 and 6.4 prove the results about product-form networks by using a time-reversal argument.

Discrete time methods for simulating continuous time Markov chains

1976 ◽  
Vol 8 (4) ◽  
pp. 772-788 ◽  
Author(s):  
Arie Hordijk ◽  
Donald L. Iglehart ◽  
Rolf Schassberger
Keyword(s):  
Central Limit Theorem ◽  
Limit Theorem ◽  
Markov Chains ◽  
Confidence Intervals ◽  
Discrete Time ◽  
Continuous Time ◽  
Statistical Efficiency ◽  
Numerical Examples ◽  
The Central Limit Theorem ◽  
Continuous Time Markov Chains

This paper discusses several problems which arise when the regenerative method is used to analyse simulations of Markov chains. The first problem involves calculating the variance constant which appears in the central limit theorem used to obtain confidence intervals. Knowledge of this constant is very helpful in evaluating simulation methodologies. The second problem is to devise a method for simulating continuous time Markov chains without having to generate exponentially distributed holding times. Several methods are presented and compared. Numerical examples are given to illustrate the computional and statistical efficiency of these methods.

scholarly journals Symmetry, Integrable Chain Models and Stochastic Processes

1998 ◽  
Vol 10 (06) ◽  
pp. 723-750 ◽  
Author(s):  
Sergio Albeverio ◽  
Shao-Ming Fei
Keyword(s):  
Markov Chains ◽  
Stochastic Processes ◽  
Discrete Time ◽  
Integrable Systems ◽  
Continuous Time ◽  
Integrable Models ◽  
Algebraic Structures ◽  
Transition Matrices ◽  
Continuous Time Markov Chains ◽  
Exactly Solvable

A general way to construct chain models with certain Lie algebraic or quantum Lie algebraic symmetries is presented. These symmetric models give rise to series of integrable systems. As an example the chain models with An symmetry and the related Temperley–Lieb algebraic structures and representations are discussed. It is shown that corresponding to these An symmetric integrable chain models there are exactly solvable stationary discrete-time (resp. continuous-time) Markov chains with transition matrices (resp. intensity matrices) having spectra which coincide with the ones of the corresponding integrable models.

Strong ergodicity for continuous-time Markov chains

1978 ◽  
Vol 15 (04) ◽  
pp. 699-706 ◽  
Author(s):  
Dean Isaacson ◽  
Barry Arnold
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Strong Ergodicity ◽  
Continuous Time Markov Chains

The concept of strong ergodicity for discrete-time homogeneous Markov chains has been characterized in several ways (Dobrushin (1956), Lin (1975), Isaacson and Tweedie (1978)). In this paper the characterization using mean visit times (Huang and Isaacson (1977)) is extended to continuous-time Markov chains. From this it follows that for a certain subclass of continuous-time Markov chains, X(t), is strongly ergodic if and only if the associated embedded chain is Cesaro strongly ergodic.

Sign in / Sign up

Export Citation Format

Share Document