Rare event analysis of the state frequencies of a large number of Markov chains

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

Download Full-text

A New Algorithm for Computing the Ergodic Probability Vector for Large Markov Chains

Probability in the Engineering and Informational Sciences ◽

10.1017/s0269964800001479 ◽

1990 ◽

Vol 4 (1) ◽

pp. 89-116 ◽

Cited By ~ 15

Author(s):

Ushlo Sumita ◽

Maria Rieders

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

The State ◽

Special Structure ◽

Probability Vector ◽

Rank Reduction ◽

Replacement Process ◽

Novel Algorithm

A novel algorithm is developed which computes the ergodic probability vector for large Markov chains. Decomposing the state space into lumps, the algorithm generates a replacement process on each lump, where any exit from a lump is instantaneously replaced at some state in that lump. The replacement distributions are constructed recursively in such a way that, in the limit, the ergodic probability vector for a replacement process on one lump will be proportional to the ergodic probability vector of the original Markov chain restricted to that lump. Inverse matrices computed in the algorithm are of size (M – 1), where M is the number of lumps, thereby providing a substantial rank reduction. When a special structure is present, the procedure for generating the replacement distributions can be simplified. The relevance of the new algorithm to the aggregation-disaggregation algorithm of Takahashi [29] is also discussed.

Download Full-text

Lumpability for non-irreducible finite markov chains

Journal of Applied Probability ◽

10.1017/s002190020002876x ◽

1984 ◽

Vol 21 (03) ◽

pp. 567-574 ◽

Cited By ~ 2

Author(s):

Atef M. Abdel-Moneim ◽

Frederick W. Leysieffer

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Discrete Time ◽

The State ◽

Discrete Time Markov Chain ◽

Finite Markov Chains

Conditions under which a function of a finite, discrete-time Markov chain, X(t), is again Markov are given, when X(t) is not irreducible. These conditions are given in terms of an interrelationship between two partitions of the state space of X(t), the partition induced by the minimal essential classes of X(t) and the partition with respect to which lumping is to be considered.

Download Full-text

On computer-intensive simulation and estimation methods for rare-event analysis in epidemic models

Statistics in Medicine ◽

10.1002/sim.6596 ◽

2015 ◽

Vol 34 (28) ◽

pp. 3696-3713

Author(s):

Stéphan Clémençon ◽

Anthony Cousien ◽

Miraine Dávila Felipe ◽

Viet Chi Tran

Keyword(s):

Rare Event ◽

Epidemic Models ◽

Estimation Methods ◽

Event Analysis

Download Full-text

Conductance bounds on the L 2 convergence rate of Metropolis algorithms on unbounded state spaces

Advances in Applied Probability ◽

10.1017/s0001867800012957 ◽

2004 ◽

Vol 36 (01) ◽

pp. 243-266

Author(s):

Søren F. Jarner ◽

Wai Kong Yuen

Keyword(s):

Convergence Rate ◽

Markov Chains ◽

Spectral Gap ◽

Unbounded Domains ◽

Decomposition Theorem ◽

Metropolis Algorithm ◽

The State ◽

Bounded Domains ◽

Target Density ◽

State Spaces

In this paper we derive bounds on the conductance and hence on the spectral gap of a Metropolis algorithm with a monotone, log-concave target density on an interval of ℝ. We show that the minimal conductance set has measure ½ and we use this characterization to bound the conductance in terms of the conductance of the algorithm restricted to a smaller domain. Whereas previous work on conductance has resulted in good bounds for Markov chains on bounded domains, this is the first conductance bound applicable to unbounded domains. We then show how this result can be combined with the state-decomposition theorem of Madras and Randall (2002) to bound the spectral gap of Metropolis algorithms with target distributions with monotone, log-concave tails on ℝ.

Download Full-text