scholarly journals Spectral Theory for Weakly Reversible Markov Chains

2012 ◽  
Vol 49 (01) ◽  
pp. 245-265 ◽  
Author(s):  
Achim Wübker
Keyword(s):  
Markov Chains ◽  
Spectral Gap ◽  
Transition Probabilities ◽  
Markov Operator ◽  
Spectral Gaps ◽  
Reversible Markov Chains ◽  
General State Space ◽  
Stationary Transition ◽  
Positive Recurrent ◽  
Necessary And Sufficient

The theory of L 2-spectral gaps for reversible Markov chains has been studied by many authors. In this paper we consider positive recurrent general state space Markov chains with stationary transition probabilities. Replacing the assumption of reversibility with a weaker assumption, we still obtain a simple necessary and sufficient condition for the spectral gap property of the associated Markov operator in terms of the isoperimetric constant. We show that this result can be applied to a large class of Markov chains, including those that are related to positive recurrent finite-range random walks on Z.

scholarly journals Spectral Theory for Weakly Reversible Markov Chains

2012 ◽  
Vol 49 (1) ◽  
pp. 245-265 ◽  
Author(s):  
Achim Wübker
Keyword(s):  
Markov Chains ◽  
Spectral Gap ◽  
Transition Probabilities ◽  
Markov Operator ◽  
Spectral Gaps ◽  
Reversible Markov Chains ◽  
General State Space ◽  
Stationary Transition ◽  
Positive Recurrent ◽  
Necessary And Sufficient

The theory of L2-spectral gaps for reversible Markov chains has been studied by many authors. In this paper we consider positive recurrent general state space Markov chains with stationary transition probabilities. Replacing the assumption of reversibility with a weaker assumption, we still obtain a simple necessary and sufficient condition for the spectral gap property of the associated Markov operator in terms of the isoperimetric constant. We show that this result can be applied to a large class of Markov chains, including those that are related to positive recurrent finite-range random walks on Z.

Single-shelf library-type Markov chains with infinitely many books

1977 ◽  
Vol 14 (02) ◽  
pp. 298-308 ◽  
Author(s):  
Peter R. Nelson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Natural Order ◽  
Order Book ◽  
Exit Boundary ◽  
One Step ◽  
Step Transition ◽  
The Right ◽  
Positive Recurrent

In a single-shelf library having infinitely many books B 1 , B 2 , …, the probability of selecting each book is assumed known. Books are removed one at a time and replaced in position k prior to the next removal. Books are moved either to the right or the left as is necessary to vacate position k. Those arrangements of books where after some finite position all the books are in natural order (book i occupies position i) are considered as states in an infinite Markov chain. When k > 1, we show that the chain can never be positive recurrent. When k = 1, we find the limits of ratios of one-step transition probabilities; and when k = 1 and the chain is transient, we find the Martin exit boundary.

Reversibility of Markov chains with applications to storage models

1985 ◽  
Vol 22 (01) ◽  
pp. 123-137 ◽  
Author(s):  
Hideo Ōsawa
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Risk Model ◽  
Sufficient Conditions ◽  
Single Server ◽  
General State ◽  
Insurance Risk ◽  
General State Space ◽  
Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

The robustness of positive recurrence and recurrence of Markov chains under perturbations of the transition probabilities

1975 ◽  
Vol 12 (04) ◽  
pp. 744-752 ◽  
Author(s):  
Richard L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Transition Probabilities ◽  
General State ◽  
Transition Matrices ◽  
Positive Recurrence ◽  
Markov Chain Models ◽  
Countable Space ◽  
General State Space

In many Markov chain models, the immediate characteristic of importance is the positive recurrence of the chain. In this note we investigate whether positivity, and also recurrence, are robust properties of Markov chains when the transition laws are perturbed. The chains we consider are on a fairly general state space : when specialised to a countable space, our results are essentially that, if the transition matrices of two irreducible chains coincide on all but a finite number of columns, then positivity of one implies positivity of both; whilst if they coincide on all but a finite number of rows and columns, recurrence of one implies recurrence of both. Examples are given to show that these results (and their general analogues) cannot in general be strengthened.

Reversibility of Markov chains with applications to storage models

1985 ◽  
Vol 22 (1) ◽  
pp. 123-137 ◽  
Author(s):  
Hideo Ōsawa
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Risk Model ◽  
Sufficient Conditions ◽  
Single Server ◽  
General State ◽  
Insurance Risk ◽  
General State Space ◽  
Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

Markov Chains with Stationary Transition Probabilities

1961 ◽  
Vol 45 (354) ◽  
pp. 362
Author(s):  
D. G. Kendall ◽  
Kai Lai Chung
Keyword(s):  
Markov Chains ◽  
Transition Probabilities ◽  
Stationary Transition

Limit theorems for sums of a sequence of random variables defined on a Markov chain

1977 ◽  
Vol 14 (03) ◽  
pp. 614-620
Author(s):  
David B. Wolfson
Keyword(s):  
Limit Theorems ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Renewal Theory ◽  
Strong Mixing ◽  
Mixing Condition ◽  
The Matrix ◽  
Stationary Transition ◽  
Positive Recurrent ◽  
Markov Renewal Theory

Let {(Jn, Xn),n≧ 0} be the standardJ–Xprocess of Markov renewal theory. Suppose {Jn,n≧ 0} is irreducible, aperiodic and positive recurrent. It is shown using the strong mixing condition, that ifconverges in distribution, wherean, bn>0 (bn→∞) are real constants, then the limit lawFmust be stable. SupposeQ(x) = {PijHi(x)} is the semi-Markov matrix of {(JnXn),n≧ 0}. Then then-fold convolution,Q∗n(bnx + anbn), converges in distribution toF(x)Π if and only ifconverges in distribution toF. Π is the matrix of stationary transition probabilities of {Jn,n≧ 0}. Sufficient conditions on theHi's are given for the convergence of the sequence of semi-Markov matrices toF(x)Π, whereFis stable.

Sign in / Sign up

Export Citation Format

Share Document