Recurrence and invariant measure of Markov chains in double-infinite random environments

2001 ◽  
Vol 44 (10) ◽  
pp. 1294-1299 ◽  
Author(s):  
Yingqiu Li
Keyword(s):  
Invariant Measure ◽  
Markov Chains ◽  
Random Environments

Stability and approximation of invariant measures of Markov chains in random environments

2015 ◽  
Vol 16 (01) ◽  
pp. 1650003 ◽  
Author(s):  
Gary Froyland ◽  
Cecilia González-Tokman
Keyword(s):  
Random Walk ◽  
Invariant Measure ◽  
Markov Chains ◽  
Random Environment ◽  
Numerical Scheme ◽  
Invariant Measures ◽  
Numerical Approach ◽  
Convergence In Probability ◽  
Random Environments ◽  
Finite State

We consider finite-state Markov chains driven by stationary ergodic invertible processes representing random environments. Our main result is that the invariant measures of Markov chains in random environments (MCREs) are stable under a wide variety of perturbations. We prove stability in the sense of convergence in probability of the invariant measure of the perturbed MCRE to the original invariant measure. We also develop a new numerical scheme to construct rigorous approximations of the invariant measures, which converge in probability as the resolution of the scheme increases. This numerical approach is illustrated with an example of a random walk in a random environment.

On determining absorption probabilities for Markov chains in random environments

1981 ◽  
Vol 13 (2) ◽  
pp. 369-387 ◽  
Author(s):  
Richard D. Bourgin ◽  
Robert Cogburn
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Random Environment ◽  
Efficient Method ◽  
General Framework ◽  
Random Environments ◽  
Extinction Probabilities ◽  
Environmental Process ◽  
Branching Chains ◽  
Absorption Probabilities

The general framework of a Markov chain in a random environment is presented and the problem of determining extinction probabilities is discussed. An efficient method for determining absorption probabilities and criteria for certain absorption are presented in the case that the environmental process is a two-state Markov chain. These results are then applied to birth and death, queueing and branching chains in random environments.

Invariant measures for Markov chains with no irreducibility assumptions

1988 ◽  
Vol 25 (A) ◽  
pp. 275-285 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Probability Theory ◽  
Invariant Measure ◽  
Markov Chains ◽  
Invariant Measures ◽  
Random Coefficient ◽  
General Function ◽  
Autoregressive Processes ◽  
Detailed Consideration ◽  
Positive Recurrence ◽  
Countable Space

Foster's criterion for positive recurrence of irreducible countable space Markov chains is one of the oldest tools in applied probability theory. In various papers in JAP and AAP it has been shown that, under extensions of irreducibility such as ϕ -irreducibility, analogues of and generalizations of Foster's criterion give conditions for the existence of an invariant measure π for general space chains, and for π to have a finite f-moment ∫π (dy)f(y), where f is a general function. In the case f ≡ 1 these cover the question of finiteness of π itself. In this paper we show that the same conditions imply the same conclusions without any irreducibility assumptions; Foster's criterion forces sufficient and appropriate regularity on the space automatically. The proofs involve detailed consideration of the structure of the minimal subinvariant measures of the chain. The results are applied to random coefficient autoregressive processes in order to illustrate the need to remove irreducibility conditions if possible.

Moments for stationary and quasi-stationary distributions of markov chains

1985 ◽  
Vol 22 (01) ◽  
pp. 148-155 ◽  
Author(s):  
E. Seneta ◽  
R. L. Tweedie
Keyword(s):  
Invariant Measure ◽  
Markov Chains ◽  
Stationary Distributions ◽  
Limit Distributions ◽  
Sufficient Set ◽  
Watson Process ◽  
Necessary And Sufficient ◽  
Conditional Limit ◽  
Finite Moments

A necessary and sufficient set of conditions is given for the finiteness of a general moment of the R -invariant measure of an R -recurrent substochastic matrix. The conditions are conceptually related to Foster's theorem. The result extends that of [8], and is illustratively applied to the single and multitype subcritical Galton–Watson process to find conditions for Yaglom-type conditional limit distributions to have finite moments.

Quasi-stationary distributions for Markov chains on a general state space

1974 ◽  
Vol 11 (4) ◽  
pp. 726-741 ◽  
Author(s):  
Richard. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Invariant Measure ◽  
Markov Chains ◽  
State Space ◽  
Finiteness Condition ◽  
General State ◽  
Stationary Distributions ◽  
General State Space ◽  
Countably Infinite ◽  
Stationary Behaviour

The quasi-stationary behaviour of a Markov chain which is φ-irreducible when restricted to a subspace of a general state space is investigated. It is shown that previous work on the case where the subspace is finite or countably infinite can be extended to general chains, and the existence of certain quasi-stationary limits as honest distributions is equivalent to the restricted chain being R-positive with the unique R-invariant measure satisfying a certain finiteness condition.

A ratio limit theorem for (sub) Markov chains on {1,2, …} with bounded jumps

1995 ◽  
Vol 27 (3) ◽  
pp. 652-691 ◽  
Author(s):  
Harry Kesten
Keyword(s):  
Harmonic Function ◽  
Invariant Measure ◽  
Probability Measure ◽  
Markov Chains ◽  
Positive Matrices ◽  
Ratio Limit Theorems ◽  
Ratio Limit ◽  
A Chain ◽  
Image Position ◽  
Quasi Stationary Distribution

We consider positive matrices Q, indexed by {1,2, …}. Assume that there exists a constant 1 L < ∞ and sequences u1< u2< · ·· and d1d2< · ·· such that Q(i, j) = 0 whenever i < ur < ur + L < j or i > dr + L > dr > j for some r. If Q satisfies some additional uniform irreducibility and aperiodicity assumptions, then for s > 0, Q has at most one positive s-harmonic function and at most one s-invariant measure µ. We use this result to show that if Q is also substochastic, then it has the strong ratio limit property, that is for a suitable R and some R–1-harmonic function f and R–1-invariant measure µ. Under additional conditions µ can be taken as a probability measure on {1,2, …} and exists. An example shows that this limit may fail to exist if Q does not satisfy the restrictions imposed above, even though Q may have a minimal normalized quasi-stationary distribution (i.e. a probability measure µ for which R–1µ = µQ).The results have an immediate interpretation for Markov chains on {0,1,2, …} with 0 as an absorbing state. They give ratio limit theorems for such a chain, conditioned on not yet being absorbed at 0 by time n.

Invariant measures for Markov chains with no irreducibility assumptions

1988 ◽  
Vol 25 (A) ◽  
pp. 275-285 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Probability Theory ◽  
Invariant Measure ◽  
Markov Chains ◽  
Invariant Measures ◽  
Random Coefficient ◽  
General Function ◽  
Autoregressive Processes ◽  
Detailed Consideration ◽  
Positive Recurrence ◽  
Countable Space

Foster's criterion for positive recurrence of irreducible countable space Markov chains is one of the oldest tools in applied probability theory. In various papers in JAP and AAP it has been shown that, under extensions of irreducibility such as ϕ -irreducibility, analogues of and generalizations of Foster's criterion give conditions for the existence of an invariant measure π for general space chains, and for π to have a finite f-moment ∫π (dy)f(y), where f is a general function. In the case f ≡ 1 these cover the question of finiteness of π itself.In this paper we show that the same conditions imply the same conclusions without any irreducibility assumptions; Foster's criterion forces sufficient and appropriate regularity on the space automatically. The proofs involve detailed consideration of the structure of the minimal subinvariant measures of the chain.The results are applied to random coefficient autoregressive processes in order to illustrate the need to remove irreducibility conditions if possible.

Sign in / Sign up

Export Citation Format

Share Document