Single-shelf library-type Markov chains with infinitely many books

1977 ◽  
Vol 14 (02) ◽  
pp. 298-308 ◽  
Author(s):  
Peter R. Nelson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Natural Order ◽  
Order Book ◽  
Exit Boundary ◽  
One Step ◽  
Step Transition ◽  
The Right ◽  
Positive Recurrent

In a single-shelf library having infinitely many books B 1 , B 2 , …, the probability of selecting each book is assumed known. Books are removed one at a time and replaced in position k prior to the next removal. Books are moved either to the right or the left as is necessary to vacate position k. Those arrangements of books where after some finite position all the books are in natural order (book i occupies position i) are considered as states in an infinite Markov chain. When k > 1, we show that the chain can never be positive recurrent. When k = 1, we find the limits of ratios of one-step transition probabilities; and when k = 1 and the chain is transient, we find the Martin exit boundary.

Single-shelf library-type Markov chains with infinitely many books

1977 ◽  
Vol 14 (2) ◽  
pp. 298-308 ◽  
Author(s):  
Peter R. Nelson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Natural Order ◽  
Order Book ◽  
Exit Boundary ◽  
One Step ◽  
Step Transition ◽  
The Right ◽  
Positive Recurrent

In a single-shelf library having infinitely many books B1, B2, …, the probability of selecting each book is assumed known. Books are removed one at a time and replaced in position k prior to the next removal. Books are moved either to the right or the left as is necessary to vacate position k. Those arrangements of books where after some finite position all the books are in natural order (book i occupies position i) are considered as states in an infinite Markov chain. When k > 1, we show that the chain can never be positive recurrent. When k = 1, we find the limits of ratios of one-step transition probabilities; and when k = 1 and the chain is transient, we find the Martin exit boundary.

On non-dissipative Markov chains

1957 ◽  
Vol 53 (4) ◽  
pp. 825-835 ◽  
Author(s):  
J. G. Mauldon
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Usual Convention ◽  
One Step ◽  
Step Transition ◽  
Image Position

Consider a Markov chain with an enumerable infinity of states, labelled 0, 1, 2, …, whose one-step transition probabilities pij are independent of time. ThenI write and, departing slightly from the usual convention,Then it is known ((1), pp. 324–34, or (6)) that the limits πij always exist, and that

Stabilité de la récurrence nulle pour certaines chaines de Markov perturbées

1983 ◽  
Vol 20 (3) ◽  
pp. 482-504 ◽  
Author(s):  
C. Cocozza-Thivent ◽  
C. Kipnis ◽  
M. Roussignol
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probability ◽  
The Other ◽  
Barrier Functions ◽  
One Step ◽  
Null Recurrence ◽  
Step Transition ◽  
The One ◽  
Taylor's Formula

We investigate how the property of null-recurrence is preserved for Markov chains under a perturbation of the transition probability. After recalling some useful criteria in terms of the one-step transition nucleus we present two methods to determine barrier functions, one in terms of taboo potentials for the unperturbed Markov chain, and the other based on Taylor's formula.

Algebraic characterization of infinite Markov chains where movement to the right is limited to one step

1977 ◽  
Vol 14 (04) ◽  
pp. 740-747 ◽  
Author(s):  
Ester Samuel-Cahn ◽  
Shmuel Zamir
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Necessary Conditions ◽  
Sufficient Conditions ◽  
Algebraic Characterization ◽  
Transient States ◽  
One Step ◽  
The Right

We consider an infinite Markov chain with states E 0, E 1, …, such that E 1, E 2, … is not closed, and for i ≧ 1 movement to the right is limited by one step. Simple algebraic characterizations are given for persistency of all states, and, if E 0 is absorbing, simple expressions are given for the probabilities of staying forever among the transient states. Examples are furnished, and simple necessary conditions and sufficient conditions for the above characterizations are given.

On the fluctuation of stochastically monotone Markov chains and some applications

1983 ◽  
Vol 20 (1) ◽  
pp. 178-184 ◽  
Author(s):  
Harry Cohn
Keyword(s):  
Markov Chains ◽  
Transition Probabilities ◽  
Branching Processes ◽  
Random Variables ◽  
One Step ◽  
Type Property ◽  
Step Transition ◽  
Varying Environment

A Borel–Cantelli-type property in terms of one-step transition probabilities is given for events like {|Xn+1| > a + ε, |Xn|≦a}, a and ε being two positive numbers. Applications to normed sums of i.i.d. random variables with infinite mean and branching processes in varying environment with or without immigration are derived.

The departure process from the GI/G/1 Queue

1969 ◽  
Vol 6 (3) ◽  
pp. 704-707 ◽  
Author(s):  
Thomas L. Vlach ◽  
Ralph L. Disney
Keyword(s):  
Markov Chain ◽  
Markov Process ◽  
Transition Probabilities ◽  
Two Dimensional ◽  
Departure Process ◽  
Dimensional State Space ◽  
Semi Markov Process ◽  
One Step ◽  
Step Transition ◽  
The One

The departure process from the GI/G/1 queue is shown to be a semi-Markov process imbedded at departure points with a two-dimensional state space. Transition probabilities for this process are defined and derived from the distributions of the arrival and service processes. The one step transition probabilities and a stationary distribution are obtained for the imbedded two-dimensional Markov chain.

Algebraic characterization of infinite Markov chains where movement to the right is limited to one step

1977 ◽  
Vol 14 (4) ◽  
pp. 740-747 ◽  
Author(s):  
Ester Samuel-Cahn ◽  
Shmuel Zamir
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Necessary Conditions ◽  
Sufficient Conditions ◽  
Algebraic Characterization ◽  
Transient States ◽  
One Step ◽  
The Right

We consider an infinite Markov chain with states E0, E1, …, such that E1, E2, … is not closed, and for i ≧ 1 movement to the right is limited by one step. Simple algebraic characterizations are given for persistency of all states, and, if E0 is absorbing, simple expressions are given for the probabilities of staying forever among the transient states. Examples are furnished, and simple necessary conditions and sufficient conditions for the above characterizations are given.

On the fluctuation of stochastically monotone Markov chains and some applications

1983 ◽  
Vol 20 (01) ◽  
pp. 178-184 ◽  
Author(s):  
Harry Cohn
Keyword(s):  
Markov Chains ◽  
Transition Probabilities ◽  
Branching Processes ◽  
Random Variables ◽  
One Step ◽  
Type Property ◽  
Step Transition ◽  
Varying Environment

A Borel–Cantelli-type property in terms of one-step transition probabilities is given for events like {|Xn +1| > a + ε, |Xn|≦a}, a and ε being two positive numbers. Applications to normed sums of i.i.d. random variables with infinite mean and branching processes in varying environment with or without immigration are derived.

The departure process from the GI/G/1 Queue

1969 ◽  
Vol 6 (03) ◽  
pp. 704-707 ◽  
Author(s):  
Thomas L. Vlach ◽  
Ralph L. Disney
Keyword(s):  
Markov Chain ◽  
Markov Process ◽  
Transition Probabilities ◽  
Two Dimensional ◽  
Departure Process ◽  
Dimensional State Space ◽  
Semi Markov Process ◽  
One Step ◽  
Step Transition ◽  
The One

The departure process from the GI/G/1 queue is shown to be a semi-Markov process imbedded at departure points with a two-dimensional state space. Transition probabilities for this process are defined and derived from the distributions of the arrival and service processes. The one step transition probabilities and a stationary distribution are obtained for the imbedded two-dimensional Markov chain.

scholarly journals A limit theorem for Markov chains with continuous state space

1963 ◽  
Vol 3 (3) ◽  
pp. 351-358 ◽  
Author(s):  
P. D. Finch
Keyword(s):  
State Space ◽  
Transition Probabilities ◽  
Borel Subset ◽  
Homogeneous Markov Chain ◽  
Continuous State Space ◽  
Continuous State ◽  
One Step ◽  
Step Transition ◽  
Image Position ◽  
The Right

Let R denote the set of real numbers, B the σ-field of all Borel subsets of R. A homogeneous Markov Chain with state space a Borel subset Ω of R is a sequence {an}, n≧ 0, of random variables, taking values in Ω, with one-step transition probabilities P(1) (ξ, A) defined by for each choice of ξ, ξ0, …, ξn−1 in ω and all Borel subsets A of ω The fact that the right-hand side of (1.1) does not depend on the ξi, 0 ≧ i > n, is of course the Markovian property, the non-dependence on n is the homogeneity of the chain.

Sign in / Sign up

Export Citation Format

Share Document