scholarly journals Potential analysis for positive recurrent Markov chains with asymptotically zero drift: Power-type asymptotics

2013 ◽  
Vol 123 (8) ◽  
pp. 3027-3051 ◽  
Author(s):  
Denis Denisov ◽  
Dmitry Korshunov ◽  
Vitali Wachtel
Keyword(s):  
Markov Chains ◽  
Zero Drift ◽  
Power Type ◽  
Potential Analysis ◽  
Positive Recurrent

Truncation approximations of invariant measures for Markov chains

1998 ◽  
Vol 35 (03) ◽  
pp. 517-536 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Transition Matrix ◽  
Invariant Measures ◽  
Invariant Distribution ◽  
Special Cases ◽  
Recurrent Markov Chain ◽  
Positive Recurrent

Let P be the transition matrix of a positive recurrent Markov chain on the integers, with invariant distribution π. If (n) P denotes the n x n ‘northwest truncation’ of P, it is known that approximations to π(j)/π(0) can be constructed from (n) P, but these are known to converge to the probability distribution itself in special cases only. We show that such convergence always occurs for three further general classes of chains, geometrically ergodic chains, stochastically monotone chains, and those dominated by stochastically monotone chains. We show that all ‘finite’ perturbations of stochastically monotone chains can be considered to be dominated by such chains, and thus the results hold for a much wider class than is first apparent. In the cases of uniformly ergodic chains, and chains dominated by irreducible stochastically monotone chains, we find practical bounds on the accuracy of the approximations.

Single-shelf library-type Markov chains with infinitely many books

1977 ◽  
Vol 14 (02) ◽  
pp. 298-308 ◽  
Author(s):  
Peter R. Nelson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Natural Order ◽  
Order Book ◽  
Exit Boundary ◽  
One Step ◽  
Step Transition ◽  
The Right ◽  
Positive Recurrent

In a single-shelf library having infinitely many books B 1 , B 2 , …, the probability of selecting each book is assumed known. Books are removed one at a time and replaced in position k prior to the next removal. Books are moved either to the right or the left as is necessary to vacate position k. Those arrangements of books where after some finite position all the books are in natural order (book i occupies position i) are considered as states in an infinite Markov chain. When k > 1, we show that the chain can never be positive recurrent. When k = 1, we find the limits of ratios of one-step transition probabilities; and when k = 1 and the chain is transient, we find the Martin exit boundary.

Error bounds for augmented truncation approximations of Markov chains via the perturbation method

2018 ◽  
Vol 50 (2) ◽  
pp. 645-669 ◽  
Author(s):  
Yuanyuan Liu ◽  
Wendi Li
Keyword(s):  
Markov Chains ◽  
Perturbation Method ◽  
Error Bounds ◽  
Continuous Time ◽  
Stochastic Matrix ◽  
Probability Vector ◽  
The Poisson Equation ◽  
Residual Matrix ◽  
Recurrent Markov Chain ◽  
Positive Recurrent

AbstractLetPbe the transition matrix of a positive recurrent Markov chain on the integers with invariant probability vectorπT, and let(n)P̃ be a stochastic matrix, formed by augmenting the entries of the (n+ 1) x (n+ 1) northwest corner truncation ofParbitrarily, with invariant probability vector(n)πT. We derive computableV-norm bounds on the error betweenπTand(n)πTin terms of the perturbation method from three different aspects: the Poisson equation, the residual matrix, and the norm ergodicity coefficient, which we prove to be effective by showing that they converge to 0 asntends to ∞ under suitable conditions. We illustrate our results through several examples. Comparing our error bounds with the ones of Tweedie (1998), we see that our bounds are more applicable and accurate. Moreover, we also consider possible extensions of our results to continuous-time Markov chains.

scholarly journals On the Dual Relationship Between Markov Chains of GI/M/1 and M/G/1 Type

2010 ◽  
Vol 42 (1) ◽  
pp. 210-225 ◽  
Author(s):  
P. G. Taylor ◽  
B. Van Houdt
Keyword(s):  
Markov Chains ◽  
Renewal Process ◽  
Markov Renewal Process ◽  
Dual Processes ◽  
Design Of Algorithms ◽  
Type Process ◽  
The Matrix ◽  
Positive Recurrent ◽  
Matrix Transforms ◽  
Phd Thesis

In 1990, Ramaswami proved that, given a Markov renewal process of M/G/1 type, it is possible to construct a Markov renewal process of GI/M/1 type such that the matrix transforms G(z, s) for the M/G/1-type process and R(z, s) for the GI/M/1-type process satisfy a duality relationship. In his 1996 PhD thesis, Bright used time reversal arguments to show that it is possible to define a different dual for positive-recurrent and transient processes of M/G/1 type and GI/M/1 type. Here we compare the properties of the Ramaswami and Bright dual processes and show that the Bright dual has desirable properties that can be exploited in the design of algorithms for the analysis of Markov chains of GI/M/1 type and M/G/1 type.

Infinite block-structured transition matrices and their properties

1998 ◽  
Vol 30 (2) ◽  
pp. 365-384 ◽  
Author(s):  
Yiqiang Q. Zhao ◽  
Wei Li ◽  
W. John Braun
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Block Structure ◽  
Sufficient Conditions ◽  
Characteristic Functions ◽  
Scalar Case ◽  
Transition Matrices ◽  
Positive Recurrent ◽  
Infinite State ◽  
Block Structured

In this paper, we study Markov chains with infinite state block-structured transition matrices, whose states are partitioned into levels according to the block structure, and various associated measures. Roughly speaking, these measures involve first passage times or expected numbers of visits to certain levels without hitting other levels. They are very important and often play a key role in the study of a Markov chain. Necessary and/or sufficient conditions are obtained for a Markov chain to be positive recurrent, recurrent, or transient in terms of these measures. Results are obtained for general irreducible Markov chains as well as those with transition matrices possessing some block structure. We also discuss the decomposition or the factorization of the characteristic equations of these measures. In the scalar case, we locate the zeros of these characteristic functions and therefore use these zeros to characterize a Markov chain. Examples and various remarks are given to illustrate some of the results.

scholarly journals Renewal theory for transient Markov chains with asymptotically zero drift

2020 ◽  
Vol 373 (10) ◽  
pp. 7253-7286
Author(s):  
Denis Denisov ◽  
Dmitry Korshunov ◽  
Vitali Wachtel
Keyword(s):  
Markov Chains ◽  
Renewal Theory ◽  
Zero Drift

Single-shelf library-type Markov chains with infinitely many books

1977 ◽  
Vol 14 (2) ◽  
pp. 298-308 ◽  
Author(s):  
Peter R. Nelson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Natural Order ◽  
Order Book ◽  
Exit Boundary ◽  
One Step ◽  
Step Transition ◽  
The Right ◽  
Positive Recurrent

In a single-shelf library having infinitely many books B1, B2, …, the probability of selecting each book is assumed known. Books are removed one at a time and replaced in position k prior to the next removal. Books are moved either to the right or the left as is necessary to vacate position k. Those arrangements of books where after some finite position all the books are in natural order (book i occupies position i) are considered as states in an infinite Markov chain. When k > 1, we show that the chain can never be positive recurrent. When k = 1, we find the limits of ratios of one-step transition probabilities; and when k = 1 and the chain is transient, we find the Martin exit boundary.

A note on acceptance rate criteria for CLTS for Metropolis–Hastings algorithms

1999 ◽  
Vol 36 (04) ◽  
pp. 1210-1217 ◽  
Author(s):  
G. O. Roberts
Keyword(s):  
Markov Chains ◽  
Central Limit ◽  
Limit Theorems ◽  
Acceptance Rate ◽  
Ergodic Averages ◽  
Current State ◽  
Rejection Probability ◽  
Independence Sampler ◽  
Positive Recurrent ◽  
Metropolis Hastings Algorithms

This paper considers positive recurrent Markov chains where the probability of remaining in the current state is arbitrarily close to 1. Specifically, conditions are given which ensure the non-existence of central limit theorems for ergodic averages of functionals of the chain. The results are motivated by applications for Metropolis–Hastings algorithms which are constructed in terms of a rejection probability (where a rejection involves remaining at the current state). Two examples for commonly used algorithms are given, for the independence sampler and the Metropolis-adjusted Langevin algorithm. The examples are rather specialized, although, in both cases, the problems which arise are typical of problems commonly occurring for the particular algorithm being used.

scholarly journals Markov chains with heavy-tailed increments and asymptotically zero drift

2019 ◽  
Vol 24 (0) ◽  
Author(s):  
Nicholas Georgiou ◽  
Mikhail V. Menshikov ◽  
Dimitri Petritis ◽  
Andrew R. Wade
Keyword(s):  
Markov Chains ◽  
Zero Drift ◽  
Heavy Tailed
Sign in / Sign up

Export Citation Format

Share Document