Error bounds for augmented truncation approximations of Markov chains via the perturbation method

2018 ◽  
Vol 50 (2) ◽  
pp. 645-669 ◽  
Author(s):  
Yuanyuan Liu ◽  
Wendi Li
Keyword(s):  
Markov Chains ◽  
Perturbation Method ◽  
Error Bounds ◽  
Continuous Time ◽  
Stochastic Matrix ◽  
Probability Vector ◽  
The Poisson Equation ◽  
Residual Matrix ◽  
Recurrent Markov Chain ◽  
Positive Recurrent

AbstractLetPbe the transition matrix of a positive recurrent Markov chain on the integers with invariant probability vectorπT, and let(n)P̃ be a stochastic matrix, formed by augmenting the entries of the (n+ 1) x (n+ 1) northwest corner truncation ofParbitrarily, with invariant probability vector(n)πT. We derive computableV-norm bounds on the error betweenπTand(n)πTin terms of the perturbation method from three different aspects: the Poisson equation, the residual matrix, and the norm ergodicity coefficient, which we prove to be effective by showing that they converge to 0 asntends to ∞ under suitable conditions. We illustrate our results through several examples. Comparing our error bounds with the ones of Tweedie (1998), we see that our bounds are more applicable and accurate. Moreover, we also consider possible extensions of our results to continuous-time Markov chains.

Truncation approximations of invariant measures for Markov chains

1998 ◽  
Vol 35 (03) ◽  
pp. 517-536 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Transition Matrix ◽  
Invariant Measures ◽  
Invariant Distribution ◽  
Special Cases ◽  
Recurrent Markov Chain ◽  
Positive Recurrent

Let P be the transition matrix of a positive recurrent Markov chain on the integers, with invariant distribution π. If (n) P denotes the n x n ‘northwest truncation’ of P, it is known that approximations to π(j)/π(0) can be constructed from (n) P, but these are known to converge to the probability distribution itself in special cases only. We show that such convergence always occurs for three further general classes of chains, geometrically ergodic chains, stochastically monotone chains, and those dominated by stochastically monotone chains. We show that all ‘finite’ perturbations of stochastically monotone chains can be considered to be dominated by such chains, and thus the results hold for a much wider class than is first apparent. In the cases of uniformly ergodic chains, and chains dominated by irreducible stochastically monotone chains, we find practical bounds on the accuracy of the approximations.

Geometric bounds on iterative approximations for nearly completely decomposable Markov chains

1990 ◽  
Vol 27 (03) ◽  
pp. 521-529 ◽  
Author(s):  
Guy Louchard ◽  
Guy Latouche
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Perturbation Analysis ◽  
Stochastic Matrix ◽  
Stationary Probability ◽  
Probability Vector ◽  
Finite Markov Chain ◽  
Stationary Probability Vector

We consider a finite Markov chain with nearly-completely decomposable stochastic matrix. We determine bounds for the error, when the stationary probability vector is approximated via a perturbation analysis.

scholarly journals A Note on Error Bounds for Approximating Transition Probabilities in Continuous-time Markov Chains

1988 ◽  
Vol 2 (4) ◽  
pp. 471-474 ◽  
Author(s):  
Nico M. van Dijk
Keyword(s):  
Markov Chains ◽  
Error Bound ◽  
Error Bounds ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Practical Interest ◽  
Occupation Times ◽  
Continuous Time Markov Chains ◽  
Elegant Method ◽  
Comparison Relation

Recently, Ross [1] proposed an elegant method of approximating transition probabilities and mean occupation times in continuous-time Markov chains based upon recursively inspecting the process at exponential times. The method turned out to be amazingly efficient for the examples investigated. However, no formal rough error bound was provided. Any error bound even though robust is of practical interest in engineering (e.g., for determining truncation criteria or setting up an experiment). This note primarily aims to show that by a simple and standard comparison relation a rough error bound of the method is secured. Also, some alternative approximations are inspected.

Geometric bounds on iterative approximations for nearly completely decomposable Markov chains

1990 ◽  
Vol 27 (3) ◽  
pp. 521-529 ◽  
Author(s):  
Guy Louchard ◽  
Guy Latouche
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Perturbation Analysis ◽  
Stochastic Matrix ◽  
Stationary Probability ◽  
Probability Vector ◽  
Finite Markov Chain ◽  
Stationary Probability Vector

We consider a finite Markov chain with nearly-completely decomposable stochastic matrix. We determine bounds for the error, when the stationary probability vector is approximated via a perturbation analysis.

Truncation approximations of invariant measures for Markov chains

1998 ◽  
Vol 35 (3) ◽  
pp. 517-536 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Transition Matrix ◽  
Invariant Measures ◽  
Invariant Distribution ◽  
Special Cases ◽  
Recurrent Markov Chain ◽  
Positive Recurrent

Let P be the transition matrix of a positive recurrent Markov chain on the integers, with invariant distribution π. If (n)P denotes the n x n ‘northwest truncation’ of P, it is known that approximations to π(j)/π(0) can be constructed from (n)P, but these are known to converge to the probability distribution itself in special cases only. We show that such convergence always occurs for three further general classes of chains, geometrically ergodic chains, stochastically monotone chains, and those dominated by stochastically monotone chains. We show that all ‘finite’ perturbations of stochastically monotone chains can be considered to be dominated by such chains, and thus the results hold for a much wider class than is first apparent. In the cases of uniformly ergodic chains, and chains dominated by irreducible stochastically monotone chains, we find practical bounds on the accuracy of the approximations.

Characterization of the conditional stationary distribution in Markov chains via systems of linear inequalities

2020 ◽  
Vol 52 (4) ◽  
pp. 1249-1283
Author(s):  
Masatoshi Kimura ◽  
Tetsuya Takine
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Continuous Time ◽  
State Transition ◽  
Convex Polytope ◽  
Linear Inequalities ◽  
Systems Of Linear Inequalities ◽  
Free Sets ◽  
Practical Implications

AbstractThis paper considers ergodic, continuous-time Markov chains $\{X(t)\}_{t \in (\!-\infty,\infty)}$ on $\mathbb{Z}^+=\{0,1,\ldots\}$ . For an arbitrarily fixed $N \in \mathbb{Z}^+$ , we study the conditional stationary distribution $\boldsymbol{\pi}(N)$ given the Markov chain being in $\{0,1,\ldots,N\}$ . We first characterize $\boldsymbol{\pi}(N)$ via systems of linear inequalities and identify simplices that contain $\boldsymbol{\pi}(N)$ , by examining the $(N+1) \times (N+1)$ northwest corner block of the infinitesimal generator $\textbf{\textit{Q}}$ and the subset of the first $N+1$ states whose members are directly reachable from at least one state in $\{N+1,N+2,\ldots\}$ . These results are closely related to the augmented truncation approximation (ATA), and we provide some practical implications for the ATA. Next we consider an extension of the above results, using the $(K+1) \times (K+1)$ ( $K > N$ ) northwest corner block of $\textbf{\textit{Q}}$ and the subset of the first $K+1$ states whose members are directly reachable from at least one state in $\{K+1,K+2,\ldots\}$ . Furthermore, we introduce new state transition structures called (K, N)-skip-free sets, using which we obtain the minimum convex polytope that contains $\boldsymbol{\pi}(N)$ .

Sign in / Sign up

Export Citation Format

Share Document