scholarly journals A note on the geometric ergodicity of a Markov chain

1989 ◽  
Vol 21 (3) ◽  
pp. 702-704 ◽  
Author(s):  
K. S. Chan

It is known that if an irreducible and aperiodic Markov chain satisfies a ‘drift' condition in terms of a non-negative measurable function g(x), it is geometrically ergodic. See, e.g. Nummelin (1984), p. 90. We extend the analysis to show that the distance between the nth-step transition probability and the invariant probability measure is bounded above by ρ n(a + bg(x)) for some constants a, b> 0 and ρ < 1. The result is then applied to obtain convergence rates to the invariant probability measures for an autoregressive process and a random walk on a half line.

1989 ◽  
Vol 21 (03) ◽  
pp. 702-704 ◽  
Author(s):  
K. S. Chan

It is known that if an irreducible and aperiodic Markov chain satisfies a ‘drift' condition in terms of a non-negative measurable function g(x), it is geometrically ergodic. See, e.g. Nummelin (1984), p. 90. We extend the analysis to show that the distance between the nth-step transition probability and the invariant probability measure is bounded above by ρ n (a + bg(x)) for some constants a, b&gt; 0 and ρ &lt; 1. The result is then applied to obtain convergence rates to the invariant probability measures for an autoregressive process and a random walk on a half line.


2004 ◽  
Vol 41 (03) ◽  
pp. 778-790
Author(s):  
Zhenting Hou ◽  
Yuanyuan Liu

This paper investigates the rate of convergence to the probability distribution of the embedded M/G/1 and GI/M/n queues. We introduce several types of ergodicity including l-ergodicity, geometric ergodicity, uniformly polynomial ergodicity and strong ergodicity. The usual method to prove ergodicity of a Markov chain is to check the existence of a Foster–Lyapunov function or a drift condition, while here we analyse the generating function of the first return probability directly and obtain practical criteria. Moreover, the method can be extended to M/G/1- and GI/M/1-type Markov chains.


1983 ◽  
Vol 20 (3) ◽  
pp. 482-504 ◽  
Author(s):  
C. Cocozza-Thivent ◽  
C. Kipnis ◽  
M. Roussignol

We investigate how the property of null-recurrence is preserved for Markov chains under a perturbation of the transition probability. After recalling some useful criteria in terms of the one-step transition nucleus we present two methods to determine barrier functions, one in terms of taboo potentials for the unperturbed Markov chain, and the other based on Taylor's formula.


2018 ◽  
Vol 55 (3) ◽  
pp. 862-886 ◽  
Author(s):  
F. Alberto Grünbaum ◽  
Manuel D. de la Iglesia

Abstract We consider upper‒lower (UL) (and lower‒upper (LU)) factorizations of the one-step transition probability matrix of a random walk with the state space of nonnegative integers, with the condition that both upper and lower triangular matrices in the factorization are also stochastic matrices. We provide conditions on the free parameter of the UL factorization in terms of certain continued fractions such that this stochastic factorization is possible. By inverting the order of the factors (also known as a Darboux transformation) we obtain a new family of random walks where it is possible to state the spectral measures in terms of a Geronimus transformation. We repeat this for the LU factorization but without a free parameter. Finally, we apply our results in two examples; the random walk with constant transition probabilities, and the random walk generated by the Jacobi orthogonal polynomials. In both situations we obtain urn models associated with all the random walks in question.


1989 ◽  
Vol 26 (4) ◽  
pp. 757-766 ◽  
Author(s):  
Ram Lal ◽  
U. Narayan Bhat

In a correlated random walk (CRW) the probabilities of movement in the positive and negative direction are given by the transition probabilities of a Markov chain. The walk can be represented as a Markov chain if we use a bivariate state space, with the location of the particle and the direction of movement as the two variables. In this paper we derive explicit results for the following characteristics of the walk directly from its transition probability matrix: (i) n -step transition probabilities for the unrestricted CRW, (ii) equilibrium distribution for the CRW restricted on one side, and (iii) equilibrium distribution and first-passage characteristics for the CRW restricted on both sides (i.e., with finite state space).


1992 ◽  
Vol 22 (2) ◽  
pp. 217-223 ◽  
Author(s):  
Heikki Bonsdorff

AbstractUnder certain conditions, a Bonus-Malus system can be interpreted as a Markov chain whose n-step transition probabilities converge to a limit probability distribution. In this paper, the rate of the convergence is studied by means of the eigenvalues of the transition probability matrix of the Markov chain.


1989 ◽  
Vol 26 (04) ◽  
pp. 757-766 ◽  
Author(s):  
Ram Lal ◽  
U. Narayan Bhat

In a correlated random walk (CRW) the probabilities of movement in the positive and negative direction are given by the transition probabilities of a Markov chain. The walk can be represented as a Markov chain if we use a bivariate state space, with the location of the particle and the direction of movement as the two variables. In this paper we derive explicit results for the following characteristics of the walk directly from its transition probability matrix: (i) n -step transition probabilities for the unrestricted CRW, (ii) equilibrium distribution for the CRW restricted on one side, and (iii) equilibrium distribution and first-passage characteristics for the CRW restricted on both sides (i.e., with finite state space).


2015 ◽  
Vol 36 (4) ◽  
pp. 1247-1259
Author(s):  
LI PENG ◽  
TETURO KAMAE

Let $1,-1,-1,1,-1,1,1,-1,-1,1,1,\ldots$ be the $\{-1,1\}$-valued Thue–Morse sequence. Its correlation dimension is $D_{2}$, satisfying $$\begin{eqnarray}\mathop{\sum }_{k=0}^{K-1}|{\it\gamma}(k)|^{2}\asymp K^{1-D_{2}}\end{eqnarray}$$ in the sense that the ratio between the left- and right-hand sides is bounded away from 0 and $\infty$ as $K\rightarrow \infty$, where ${\it\gamma}$ is the correlation function; its value is known [Zaks, Pikovsky and Kurths. On the correlation dimension of the spectral measure for the Thue–Morse sequence. J. Stat. Phys.88(5/6) (1997), 1387–1392] to be $$\begin{eqnarray}D_{2}=1-\log \frac{1+\sqrt{17}}{4}\bigg/\log 2=0.64298\ldots .\end{eqnarray}$$ Under its spectral measure ${\it\mu}$ on $[0,1)$, consider the transformation $T$ with $Tx=2x$ ($\text{mod}~1$). It is shown to be of Kolmogorov type having entropy at least $D_{2}\log 2$. Moreover, a random walk is defined by $T^{-1}$ which has the transition probability $$\begin{eqnarray}P_{1}((1/2)x+(1/2)j\mid x)=(1/2)(1-\cos ({\it\pi}(x+j)))\quad (j=0,1).\end{eqnarray}$$ It is proved that this random walk is mixing and ${\it\mu}$ is the unique stationary measure. Moreover, $$\begin{eqnarray}\lim _{N\rightarrow \infty }\int P_{N}((x-{\it\varepsilon},x+{\it\varepsilon})|x)\,d{\it\mu}(x)\asymp {\it\varepsilon}^{D_{2}}\quad (\text{as}~{\it\varepsilon}\rightarrow 0),\end{eqnarray}$$ where $P_{N}(\cdot \mid \cdot )$ is the $N$-step transition probability.


1977 ◽  
Vol 14 (03) ◽  
pp. 621-625
Author(s):  
A. O. Pittenger

Suppose a physical process is modelled by a Markov chain with transition probability on S 1 ∪ S 2, S 1 denoting the transient states and S 2 a set of absorbing states. If v denotes the output distribution on S 2, the question arises as to what input distributions (of raw materials) on S 1 produce v. In this note we give an alternative to the formulation of Ray and Margo [2] and reduce the problem to one system of linear inequalities. An application to random walk is given and the equiprobability case examined in detail.


2017 ◽  
Vol 54 (2) ◽  
pp. 638-654 ◽  
Author(s):  
K. Kamatani

Abstract We describe the ergodic properties of some Metropolis–Hastings algorithms for heavy-tailed target distributions. The results of these algorithms are usually analyzed under a subgeometric ergodic framework, but we prove that the mixed preconditioned Crank–Nicolson (MpCN) algorithm has geometric ergodicity even for heavy-tailed target distributions. This useful property comes from the fact that, under a suitable transformation, the MpCN algorithm becomes a random-walk Metropolis algorithm.


Sign in / Sign up

Export Citation Format

Share Document