Markov Chains with a Finite Number of States

2012 ◽  
pp. 67-84
Author(s):  
Leonid Koralov ◽  
Yakov G. Sinai
Keyword(s):  
Markov Chains ◽  
Finite Number

The robustness of positive recurrence and recurrence of Markov chains under perturbations of the transition probabilities

1975 ◽  
Vol 12 (04) ◽  
pp. 744-752 ◽  
Author(s):  
Richard L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Transition Probabilities ◽  
General State ◽  
Transition Matrices ◽  
Positive Recurrence ◽  
Markov Chain Models ◽  
Countable Space ◽  
General State Space

In many Markov chain models, the immediate characteristic of importance is the positive recurrence of the chain. In this note we investigate whether positivity, and also recurrence, are robust properties of Markov chains when the transition laws are perturbed. The chains we consider are on a fairly general state space : when specialised to a countable space, our results are essentially that, if the transition matrices of two irreducible chains coincide on all but a finite number of columns, then positivity of one implies positivity of both; whilst if they coincide on all but a finite number of rows and columns, recurrence of one implies recurrence of both. Examples are given to show that these results (and their general analogues) cannot in general be strengthened.

General transition probabilities for finite Markov chains

1964 ◽  
Vol 60 (1) ◽  
pp. 83-91 ◽  
Author(s):  
Marcel F. Neuts
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Initial State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

We consider a stationary discrete-time Markov chain with a finite number m of possible states which we designate by 1,…,m. We assume that at time t = 0 the process is in an initial state i with probability (i = 1,…, m) and such that and .

KOLMOGOROV MATRIX, AND A CONTINUOUS MARKOV CHAIN WITH A FINITE NUMBER OF STATES

2018 ◽  
pp. 503-509
Author(s):  
Anatoly Ivanovich Perov
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Almost Periodic ◽  
Periodic Coefficients ◽  
Constant Coefficients ◽  
Averaged Systems

In terms of ergodicity of averaged systems with constant coefficients (and Kolmogorov matrix), the signs of ergodicity of continuous Markov chains with a finite number of States with periodic and almost periodic coefficients are indicated.

The robustness of positive recurrence and recurrence of Markov chains under perturbations of the transition probabilities

1975 ◽  
Vol 12 (4) ◽  
pp. 744-752 ◽  
Author(s):  
Richard L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Transition Probabilities ◽  
General State ◽  
Transition Matrices ◽  
Positive Recurrence ◽  
Markov Chain Models ◽  
Countable Space ◽  
General State Space

In many Markov chain models, the immediate characteristic of importance is the positive recurrence of the chain. In this note we investigate whether positivity, and also recurrence, are robust properties of Markov chains when the transition laws are perturbed. The chains we consider are on a fairly general state space : when specialised to a countable space, our results are essentially that, if the transition matrices of two irreducible chains coincide on all but a finite number of columns, then positivity of one implies positivity of both; whilst if they coincide on all but a finite number of rows and columns, recurrence of one implies recurrence of both. Examples are given to show that these results (and their general analogues) cannot in general be strengthened.

Criteria and Methods for Reliable Three-Dimensional Image Reconstruction

1974 ◽  
Vol 32 ◽  
pp. 330-331
Author(s):  
R. A. Crowther
Keyword(s):  
Image Reconstruction ◽  
Finite Number ◽  
Density Distribution ◽  
Electron Micrographs ◽  
Mathematical Problem ◽  
Three Dimensional ◽  
Ideal Case ◽  
Dimensional Image ◽  
General Object ◽  
The Ideal

The reconstruction of a three-dimensional image of a specimen from a set of electron micrographs reduces, under certain assumptions about the imaging process in the microscope, to the mathematical problem of reconstructing a density distribution from a set of its plane projections.In the absence of noise we can formulate a purely geometrical criterion, which, for a general object, fixes the resolution attainable from a given finite number of views in terms of the size of the object. For simplicity we take the ideal case of projections collected by a series of m equally spaced tilts about a single axis.

Markov models—Markov chains

2019 ◽  
Vol 16 (8) ◽  
pp. 663-664 ◽  
Author(s):  
Jasleen K. Grewal ◽  
Martin Krzywinski ◽  
Naomi Altman
Keyword(s):  
Markov Chains ◽  
Markov Models
Sign in / Sign up

Export Citation Format

Share Document