Transience and recurrence of an interesting Markov chain

This note studies the natural extension to the countable case of a chain considered by Hendricks (1972) and gives necessary and sufficient conditions for transience, null recurrence and positive recurrence.

Download Full-text

The Condition of Regularity in Simple Markov Chains

Australian Journal of Physics ◽

10.1071/ph560387 ◽

1956 ◽

Vol 9 (3) ◽

pp. 387

Author(s):

J Gani

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Stochastic Matrix ◽

Sufficient Conditions ◽

Latent Root ◽

Necessary And Sufficient Conditions ◽

A Chain ◽

Necessary And Sufficient

The paper generalizes a proof, and outlines an alternative to it, for the well-known theorem on the conditions of regularity in a simple Markov chain; this is that the necessary and sufficient conditions for a chain to be regular are that the latent root 1 of the stochastic matrix for the chain must be simple, and the remaining roots have moduli less than 1.

Download Full-text

The M/M/1 queue with mass exodus and mass arrivals when empty

Journal of Applied Probability ◽

10.1017/s0021900200100816 ◽

1997 ◽

Vol 34 (01) ◽

pp. 192-207 ◽

Cited By ~ 12

Author(s):

Anyue Chen ◽

Eric Renshaw

Keyword(s):

Sufficient Conditions ◽

High Order ◽

Necessary And Sufficient Conditions ◽

Positive Recurrence ◽

Powerful Technique ◽

Resolvent Approach ◽

Necessary And Sufficient ◽

Direct Attack

An M/M/1 queue is subject to mass exodus at rate β and mass immigration at rate when idle. A general resolvent approach is used to derive occupation probabilities and high-order moments. This powerful technique is not only considerably easier to apply than a standard direct attack on the forward p.g.f. equation, but it also implicitly yields necessary and sufficient conditions for recurrence, positive recurrence and transience.

Download Full-text

Convex Necessary and Sufficient Conditions for Density Safety Constraints in Markov Chain Synthesis

IEEE Transactions on Automatic Control ◽

10.1109/tac.2015.2400712 ◽

2015 ◽

Vol 60 (10) ◽

pp. 2813-2818 ◽

Cited By ~ 13

Author(s):

Behcet Acikmese ◽

Nazli Demir ◽

Matthew W. Harris

Keyword(s):

Markov Chain ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Safety Constraints ◽

Chain Synthesis ◽

Necessary And Sufficient

Download Full-text

Stability of maxima of random variables defined on a Markov chain

Advances in Applied Probability ◽

10.2307/1426000 ◽

1972 ◽

Vol 4 (2) ◽

pp. 285-295 ◽

Cited By ~ 13

Author(s):

Sidney I. Resnick

Keyword(s):

Markov Chain ◽

Regularity Condition ◽

Sufficient Conditions ◽

Random Variables ◽

Necessary And Sufficient Conditions ◽

Finite Markov Chain ◽

Finite Product ◽

Normalizing Constants ◽

Necessary And Sufficient

Consider maxima Mn of a sequence of random variables defined on a finite Markov chain. Necessary and sufficient conditions for the existence of normalizing constants Bn such that are given. The problem can be reduced to studying maxima of i.i.d. random variables drawn from a finite product of distributions πi=1mHi(x). The effect of each factor Hi(x) on the behavior of maxima from πi=1mHi is analyzed. Under a mild regularity condition, Bn can be chosen to be the maximum of the m quantiles of order (1 - n-1) of the H's.

Download Full-text

Necessary and sufficient conditions for recurrence and transience of Markov chains, in terms of inequalities

Journal of Applied Probability ◽

10.2307/3213440 ◽

1978 ◽

Vol 15 (4) ◽

pp. 848-851 ◽

Cited By ~ 22

Author(s):

Jean-François Mertens ◽

Ester Samuel-Cahn ◽

Shmuel Zamir

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Bounded Solution ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Existence Of A Solution ◽

Necessary And Sufficient ◽

Irreducible Markov Chain

For an aperiodic, irreducible Markov chain with the non-negative integers as state space it is shown that the existence of a solution to in which yi → ∞is necessary and sufficient for recurrence, and the existence of a bounded solution to the same inequalities, with yk < yo, · · ·, yN–1 for some k ≧ N, is necessary and sufficient for transience.

Download Full-text

Recurrence classification for a family of non-linear storage models

Probability and Mathematical Statistics ◽

10.19195/0208-4147.37.2.8 ◽

2018 ◽

Vol 37 (2) ◽

pp. 337-353

Author(s):

Peter W. Glynn ◽

Sanatan Rai ◽

John E. Glynn

Keyword(s):

Discrete Time ◽

Power Law ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Storage Model ◽

Positive Recurrence ◽

Non Linear ◽

Necessary And Sufficient ◽

Recurrence Classification

RECURRENCE CLASSIFICATION FOR A FAMILY OF NON-LINEAR STORAGE MODELSNecessary and sufficient conditions for positive recurrence of a discrete-time non-linear storage model with power law dynamics arederived. In addition, necessary and sufficient conditions for finiteness of p-th stationary moments are obtained for this class of models.

Download Full-text

Stability in a non-homogeneous Markov chain model in manpower systems

Journal of Applied Probability ◽

10.1017/s0021900200034264 ◽

1981 ◽

Vol 18 (04) ◽

pp. 924-930 ◽

Cited By ~ 4

Author(s):

P.-C. G. Vassiliou

Keyword(s):

Markov Chain ◽

Markov Chain Model ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Homogeneous Markov Chain ◽

Initial Structure ◽

Chain Model ◽

Limiting Behaviour ◽

Necessary And Sufficient

Necessary and sufficient conditions for stability, imposed firstly on the initial structure and the sequence of recruitment, and secondly on the initial structure and the sequence of expansion are provided in forms of two theorems. Also the limiting behaviour of the expected relative grade sizes is studied if we drop the conditions for stability imposed on the initial structure and keep the same sequence of expansion. Finally we examine the limiting behaviour of the expected grade sizes if we drop the assumption of a continuously expanding system.

Download Full-text

Hendricks libraries and Tsetlin piles

Advances in Applied Probability ◽

10.1017/s0001867800036697 ◽

1982 ◽

Vol 14 (01) ◽

pp. 37-55 ◽

Cited By ~ 1

Author(s):

Jacques-Edouard Dies

Keyword(s):

Markov Chains ◽

Large Class ◽

Special Class ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Stationary Measure ◽

Sufficient Condition ◽

Positive Recurrence ◽

Necessary And Sufficient

In order to study the transience of Hendricks libraries, we introduce and study a special class of Markov chains, the Tsetlin d-piles, generalizing Tsetlin libraries and briefly defined as follows: a 1-pile is a Tsetlin library and a d-pile is a Tsetlin library where each book is replaced by a (d − 1)-pile. We give a stationary measure of these chains and establish the necessary and sufficient conditions for positive recurrence and transience. Finally, the study of d-piles allows us to determine a sufficient condition for transience of quite a large class of Hendricks libraries.

Download Full-text

Limiting second moments for transient states of Markov chains

Journal of Applied Probability ◽

10.1017/s0021900200096108 ◽

1973 ◽

Vol 10 (04) ◽

pp. 891-894

Author(s):

H. P. Wynn

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Covariance Matrix ◽

Discrete Time ◽

Sufficient Conditions ◽

Necessary And Sufficient Conditions ◽

Time Points ◽

Transient States ◽

Second Moments ◽

Necessary And Sufficient

The set of transient states of a Markov chain is considered as a system. If numbers of arrivals to the system at discrete time points have constant mean and covariance matrix then there is a limiting distribution of numbers in the states. Necessary and sufficient conditions are given for this distribution to yield zero correlations between states.

Download Full-text