Necessary and sufficient conditions for recurrence and transience of Markov chains, in terms of inequalities

For an aperiodic, irreducible Markov chain with the non-negative integers as state space it is shown that the existence of a solution to in which yi → ∞is necessary and sufficient for recurrence, and the existence of a bounded solution to the same inequalities, with yk < yo, · · ·, yN–1 for some k ≧ N, is necessary and sufficient for transience.

Download Full-text

Necessary and sufficient conditions for recurrence and transience of Markov chains, in terms of inequalities

Journal of Applied Probability ◽

10.1017/s002190020002619x ◽

1978 ◽

Vol 15 (04) ◽

pp. 848-851 ◽

Cited By ~ 7

Author(s):

Jean-François Mertens ◽

Ester Samuel-Cahn ◽

Shmuel Zamir

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Bounded Solution ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Existence Of A Solution ◽

Necessary And Sufficient ◽

Irreducible Markov Chain

For an aperiodic, irreducible Markov chain with the non-negative integers as state space it is shown that the existence of a solution to in which yi → ∞is necessary and sufficient for recurrence, and the existence of a bounded solution to the same inequalities, with yk < y o, · · ·, yN –1 for some k ≧ N, is necessary and sufficient for transience.

Download Full-text

Markov Chains Conditioned Never to Wait Too Long at the Origin

Journal of Applied Probability ◽

10.1017/s0021900200005891 ◽

2009 ◽

Vol 46 (03) ◽

pp. 812-826

Author(s):

Saul Jacka

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Sufficient Conditions ◽

Weak Limit ◽

The State ◽

Coin Tossing ◽

First Time ◽

Vague Convergence ◽

Irreducible Markov Chain

Motivated by Feller's coin-tossing problem, we consider the problem of conditioning an irreducible Markov chain never to wait too long at 0. Denoting by τ the first time that the chain,X, waits for at least one unit of time at the origin, we consider conditioning the chain on the event (τ›T). We show that there is a weak limit asT→∞ in the cases where either the state space is finite orXis transient. We give sufficient conditions for the existence of a weak limit in other cases and show that we have vague convergence to a defective limit if the time to hit zero has a lighter tail than τ and τ is subexponential.

Download Full-text

Limiting second moments for transient states of Markov chains

Journal of Applied Probability ◽

10.1017/s0021900200096108 ◽

1973 ◽

Vol 10 (04) ◽

pp. 891-894

Author(s):

H. P. Wynn

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Covariance Matrix ◽

Discrete Time ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Time Points ◽

Transient States ◽

Second Moments ◽

Necessary And Sufficient

The set of transient states of a Markov chain is considered as a system. If numbers of arrivals to the system at discrete time points have constant mean and covariance matrix then there is a limiting distribution of numbers in the states. Necessary and sufficient conditions are given for this distribution to yield zero correlations between states.

Download Full-text

On the construction problem for single-exit Markov chains

Bulletin of the Australian Mathematical Society ◽

10.1017/s0004972700029282 ◽

1991 ◽

Vol 43 (3) ◽

pp. 439-450 ◽

Cited By ~ 8

Author(s):

P.K. Pollett

Keyword(s):

Invariant Measure ◽

Markov Chains ◽

State Space ◽

Existence And Uniqueness ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Construction Problem ◽

Necessary And Sufficient

I shall consider the following problem: given a stable, conservative, single-exit q-matrix, Q, over an irreducible state-space S and a μ-subinvariant measure, m, for Q, determine all Q-processes for which m is a μ-invariant measure. I shall provide necessary and sufficient conditions for the existence and uniqueness of such a process.

Download Full-text

Markov Chains Conditioned Never to Wait Too Long at the Origin

Journal of Applied Probability ◽

10.1239/jap/1253279853 ◽

2009 ◽

Vol 46 (3) ◽

pp. 812-826

Author(s):

Saul Jacka

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Sufficient Conditions ◽

Weak Limit ◽

The State ◽

Coin Tossing ◽

First Time ◽

Vague Convergence ◽

Irreducible Markov Chain

Motivated by Feller's coin-tossing problem, we consider the problem of conditioning an irreducible Markov chain never to wait too long at 0. Denoting by τ the first time that the chain, X, waits for at least one unit of time at the origin, we consider conditioning the chain on the event (τ›T). We show that there is a weak limit as T→∞ in the cases where either the state space is finite or X is transient. We give sufficient conditions for the existence of a weak limit in other cases and show that we have vague convergence to a defective limit if the time to hit zero has a lighter tail than τ and τ is subexponential.

Download Full-text

Limiting second moments for transient states of Markov chains

Journal of Applied Probability ◽

10.2307/3212393 ◽

1973 ◽

Vol 10 (4) ◽

pp. 891-894 ◽

Cited By ~ 2

Author(s):

H. P. Wynn

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Covariance Matrix ◽

Discrete Time ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Time Points ◽

Transient States ◽

Second Moments ◽

Necessary And Sufficient

Download Full-text

The Condition of Regularity in Simple Markov Chains

Australian Journal of Physics ◽

10.1071/ph560387 ◽

1956 ◽

Vol 9 (3) ◽

pp. 387

Author(s):

J Gani

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Stochastic Matrix ◽

Sufficient Conditions ◽

Latent Root ◽

Necessary And Sufficient Conditions ◽

A Chain ◽

Necessary And Sufficient

The paper generalizes a proof, and outlines an alternative to it, for the well-known theorem on the conditions of regularity in a simple Markov chain; this is that the necessary and sufficient conditions for a chain to be regular are that the latent root 1 of the stochastic matrix for the chain must be simple, and the remaining roots have moduli less than 1.

Download Full-text

Some sufficient conditions for non-ergodicity of markov chains

Journal of Applied Probability ◽

10.1017/s0021900200029065 ◽

1985 ◽

Vol 22 (01) ◽

pp. 138-147 ◽

Cited By ~ 1

Author(s):

Wojciech Szpankowski

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Lyapunov Functions ◽

Sufficient Conditions

Some sufficient conditions for non-ergodicity are given for a Markov chain with denumerable state space. These conditions generalize Foster's results, in that unbounded Lyapunov functions are considered. Our criteria directly extend the conditions obtained in Kaplan (1979), in the sense that a class of Lyapunov functions is studied. Applications are presented through some examples; in particular, sufficient conditions for non-ergodicity of a multidimensional Markov chain are given.

Download Full-text

Reversibility of Markov chains with applications to storage models

Journal of Applied Probability ◽

10.1017/s0021900200029053 ◽

1985 ◽

Vol 22 (01) ◽

pp. 123-137 ◽

Cited By ~ 2

Author(s):

Hideo Ōsawa

Keyword(s):

Markov Chains ◽

State Space ◽

Discrete Time ◽

Risk Model ◽

Sufficient Conditions ◽

Single Server ◽

General State ◽

Insurance Risk ◽

General State Space ◽

Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

Download Full-text