On non-dissipative Markov chains

1957 ◽  
Vol 53 (4) ◽  
pp. 825-835 ◽  
Author(s):  
J. G. Mauldon
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Usual Convention ◽  
One Step ◽  
Step Transition ◽  
Image Position

Consider a Markov chain with an enumerable infinity of states, labelled 0, 1, 2, …, whose one-step transition probabilities pij are independent of time. ThenI write and, departing slightly from the usual convention,Then it is known ((1), pp. 324–34, or (6)) that the limits πij always exist, and that

Single-shelf library-type Markov chains with infinitely many books

1977 ◽  
Vol 14 (02) ◽  
pp. 298-308 ◽  
Author(s):  
Peter R. Nelson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Natural Order ◽  
Order Book ◽  
Exit Boundary ◽  
One Step ◽  
Step Transition ◽  
The Right ◽  
Positive Recurrent

In a single-shelf library having infinitely many books B 1 , B 2 , …, the probability of selecting each book is assumed known. Books are removed one at a time and replaced in position k prior to the next removal. Books are moved either to the right or the left as is necessary to vacate position k. Those arrangements of books where after some finite position all the books are in natural order (book i occupies position i) are considered as states in an infinite Markov chain. When k > 1, we show that the chain can never be positive recurrent. When k = 1, we find the limits of ratios of one-step transition probabilities; and when k = 1 and the chain is transient, we find the Martin exit boundary.

Single-shelf library-type Markov chains with infinitely many books

1977 ◽  
Vol 14 (2) ◽  
pp. 298-308 ◽  
Author(s):  
Peter R. Nelson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Natural Order ◽  
Order Book ◽  
Exit Boundary ◽  
One Step ◽  
Step Transition ◽  
The Right ◽  
Positive Recurrent

In a single-shelf library having infinitely many books B1, B2, …, the probability of selecting each book is assumed known. Books are removed one at a time and replaced in position k prior to the next removal. Books are moved either to the right or the left as is necessary to vacate position k. Those arrangements of books where after some finite position all the books are in natural order (book i occupies position i) are considered as states in an infinite Markov chain. When k > 1, we show that the chain can never be positive recurrent. When k = 1, we find the limits of ratios of one-step transition probabilities; and when k = 1 and the chain is transient, we find the Martin exit boundary.

Stabilité de la récurrence nulle pour certaines chaines de Markov perturbées

1983 ◽  
Vol 20 (3) ◽  
pp. 482-504 ◽  
Author(s):  
C. Cocozza-Thivent ◽  
C. Kipnis ◽  
M. Roussignol
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probability ◽  
The Other ◽  
Barrier Functions ◽  
One Step ◽  
Null Recurrence ◽  
Step Transition ◽  
The One ◽  
Taylor's Formula

We investigate how the property of null-recurrence is preserved for Markov chains under a perturbation of the transition probability. After recalling some useful criteria in terms of the one-step transition nucleus we present two methods to determine barrier functions, one in terms of taboo potentials for the unperturbed Markov chain, and the other based on Taylor's formula.

On a class of non-homogeneous Markov chains

1982 ◽  
Vol 92 (3) ◽  
pp. 527-534 ◽  
Author(s):  
Harry Cohn
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Homogeneous Markov Chain ◽  
Image Position

AbstractSuppose that {Xn} is a countable non-homogeneous Markov chain andIf converges for any i, l, m, j with , thenwhenever lim , whereas if converges, thenwhere and . The behaviour of transition probabilities between various groups of states is studied and criteria for recurrence and transience are given.

On the fluctuation of stochastically monotone Markov chains and some applications

1983 ◽  
Vol 20 (1) ◽  
pp. 178-184 ◽  
Author(s):  
Harry Cohn
Keyword(s):  
Markov Chains ◽  
Transition Probabilities ◽  
Branching Processes ◽  
Random Variables ◽  
One Step ◽  
Type Property ◽  
Step Transition ◽  
Varying Environment

A Borel–Cantelli-type property in terms of one-step transition probabilities is given for events like {|Xn+1| > a + ε, |Xn|≦a}, a and ε being two positive numbers. Applications to normed sums of i.i.d. random variables with infinite mean and branching processes in varying environment with or without immigration are derived.

The departure process from the GI/G/1 Queue

1969 ◽  
Vol 6 (3) ◽  
pp. 704-707 ◽  
Author(s):  
Thomas L. Vlach ◽  
Ralph L. Disney
Keyword(s):  
Markov Chain ◽  
Markov Process ◽  
Transition Probabilities ◽  
Two Dimensional ◽  
Departure Process ◽  
Dimensional State Space ◽  
Semi Markov Process ◽  
One Step ◽  
Step Transition ◽  
The One

The departure process from the GI/G/1 queue is shown to be a semi-Markov process imbedded at departure points with a two-dimensional state space. Transition probabilities for this process are defined and derived from the distributions of the arrival and service processes. The one step transition probabilities and a stationary distribution are obtained for the imbedded two-dimensional Markov chain.

The Galton-Watson process with mean one

1967 ◽  
Vol 4 (3) ◽  
pp. 489-495 ◽  
Author(s):  
E. Seneta
Keyword(s):  
Markov Chain ◽  
Transition Probabilities ◽  
Watson Process ◽  
Step Transition ◽  
Image Position ◽  
Number Of Individuals

Let Zn be the number of individuals in the n-th generation of a simple (discrete, one-type) Galton-Watson process, descended from a single ancestor. Write and assume, as usual, that 0 < F(0) < 1. It is well known that the p.g.f. of Zn is Fn(s), the n-th functional iterate of F(s), and that if the process is construed as a Markov chain on the states 0, 1, 2,…, then its n-step transition probabilities are given by

The Galton-Watson process with mean one

1967 ◽  
Vol 4 (03) ◽  
pp. 489-495 ◽  
Author(s):  
E. Seneta
Keyword(s):  
Markov Chain ◽  
Transition Probabilities ◽  
Watson Process ◽  
Step Transition ◽  
Image Position ◽  
Number Of Individuals

Let Zn be the number of individuals in the n-th generation of a simple (discrete, one-type) Galton-Watson process, descended from a single ancestor. Write and assume, as usual, that 0 &lt; F(0) &lt; 1. It is well known that the p.g.f. of Zn is Fn (s), the n-th functional iterate of F(s), and that if the process is construed as a Markov chain on the states 0, 1, 2,…, then its n-step transition probabilities are given by

Occupation times for two state Markov chains

1971 ◽  
Vol 8 (02) ◽  
pp. 381-390 ◽  
Author(s):  
P. J. Pedler
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Random Variables ◽  
Time Parameter ◽  
Conditional Probabilities ◽  
Occupation Times ◽  
The Matrix ◽  
Image Position

Consider first a Markov chain with two ergodic states E 1 and E 2, and discrete time parameter set {0, 1, 2, ···, n}. Define the random variables Z 0, Z 1, Z 2, ···, Zn by then the conditional probabilities for k = 1,2,···, n, are independent of k. Thus the matrix of transition probabilities is

On the fluctuation of stochastically monotone Markov chains and some applications

1983 ◽  
Vol 20 (01) ◽  
pp. 178-184 ◽  
Author(s):  
Harry Cohn
Keyword(s):  
Markov Chains ◽  
Transition Probabilities ◽  
Branching Processes ◽  
Random Variables ◽  
One Step ◽  
Type Property ◽  
Step Transition ◽  
Varying Environment

A Borel–Cantelli-type property in terms of one-step transition probabilities is given for events like {|Xn +1| &gt; a + ε, |Xn|≦a}, a and ε being two positive numbers. Applications to normed sums of i.i.d. random variables with infinite mean and branching processes in varying environment with or without immigration are derived.

Sign in / Sign up

Export Citation Format

Share Document