scholarly journals Continuity properties of a factor of Markov chains

2016 ◽  
Vol 53 (1) ◽  
pp. 216-230 ◽  
Author(s):  
Walter A. F. de Carvalho ◽  
Sandro Gallo ◽  
Nancy L. Garcia
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Discontinuous Transition ◽  
The Past ◽  
Continuity Properties

Abstract Starting from a Markov chain with a finite or a countable infinite alphabet, we consider the chain obtained when all but one symbol are indistinguishable for the practitioner. We study conditions on the transition matrix of the Markov chain ensuring that the image chain has continuous or discontinuous transition probabilities with respect to the past.

scholarly journals On the Embedding Problem for Discrete-Time Markov Chains

2013 ◽  
Vol 50 (04) ◽  
pp. 918-930 ◽  
Author(s):  
Marie-Anne Guerry
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Embedding Problem ◽  
Time Intervals ◽  
Square Roots ◽  
Probability Matrices

When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. This characterization is referred to in the literature as the embedding problem for discrete time Markov chains. The probability matrix which has probability root(s) is called embeddable. In this paper for two-state Markov chains, necessary and sufficient conditions for embeddability are formulated and the probability square roots of the transition matrix are presented in analytic form. In finding conditions for the existence of probability square roots for (k x k) transition matrices, properties of row-normalized matrices are examined. Besides the existence of probability square roots, the uniqueness of these solutions is discussed: In the case of nonuniqueness, a procedure is introduced to identify a transition matrix that takes into account the specificity of the concrete context. In the case of nonexistence of a probability root, the concept of an approximate probability root is introduced as a solution of an optimization problem related to approximate nonnegative matrix factorization.

scholarly journals On the Embedding Problem for Discrete-Time Markov Chains

2013 ◽  
Vol 50 (4) ◽  
pp. 918-930 ◽  
Author(s):  
Marie-Anne Guerry
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Embedding Problem ◽  
Time Intervals ◽  
Square Roots ◽  
Probability Matrices

When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. This characterization is referred to in the literature as the embedding problem for discrete time Markov chains. The probability matrix which has probability root(s) is called embeddable.In this paper for two-state Markov chains, necessary and sufficient conditions for embeddability are formulated and the probability square roots of the transition matrix are presented in analytic form. In finding conditions for the existence of probability square roots for (kxk) transition matrices, properties of row-normalized matrices are examined. Besides the existence of probability square roots, the uniqueness of these solutions is discussed: In the case of nonuniqueness, a procedure is introduced to identify a transition matrix that takes into account the specificity of the concrete context. In the case of nonexistence of a probability root, the concept of an approximate probability root is introduced as a solution of an optimization problem related to approximate nonnegative matrix factorization.

scholarly journals A Bayesian model for binary Markov chains

2004 ◽  
Vol 2004 (8) ◽  
pp. 421-429 ◽  
Author(s):  
Souad Assoudou ◽  
Belkheir Essebbar
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chains ◽  
Bayesian Estimation ◽  
Bayesian Model ◽  
Transition Probabilities ◽  
Simulated Data ◽  
Bayesian Estimator ◽  
Jeffreys Prior ◽  
Data Set

This note is concerned with Bayesian estimation of the transition probabilities of a binary Markov chain observed from heterogeneous individuals. The model is founded on the Jeffreys' prior which allows for transition probabilities to be correlated. The Bayesian estimator is approximated by means of Monte Carlo Markov chain (MCMC) techniques. The performance of the Bayesian estimates is illustrated by analyzing a small simulated data set.

scholarly journals Infinitely divisible random transition probabilities with application to dependent Markov chains

1984 ◽  
Vol 25 (4) ◽  
pp. 463-472
Author(s):  
Peter L. Chesson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
White Noise ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Transition Rates ◽  
Infinitely Divisible ◽  
Random Transition ◽  
Transition Probability Matrices ◽  
Probability Matrices

AbstractRandom transition probability matrices with stationary independent factors define “white noise” environment processes for Markov chains. Two examples are considered in detail. Such environment processes can be used to construct several Markov chains which are dependent, have the same transition probabilities and are jointly a Markov chain. Transition rates for such processes are evaluated. These results have application to the study of animal movements.

Perturbation theory and finite Markov chains

1968 ◽  
Vol 5 (2) ◽  
pp. 401-413 ◽  
Author(s):  
Paul J. Schweitzer
Keyword(s):  
Perturbation Theory ◽  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Fundamental Matrix ◽  
Transition Probabilities ◽  
Semi Group ◽  
Partial Derivatives ◽  
Finite Markov Chains ◽  
Derivatives Of

A perturbation formalism is presented which shows how the stationary distribution and fundamental matrix of a Markov chain containing a single irreducible set of states change as the transition probabilities vary. Expressions are given for the partial derivatives of the stationary distribution and fundamental matrix with respect to the transition probabilities. Semi-group properties of the generators of transformations from one Markov chain to another are investigated. It is shown that a perturbation formalism exists in the multiple subchain case if and only if the change in the transition probabilities does not alter the number of, or intermix the various subchains. The formalism is presented when this condition is satisfied.

Truncation approximations of invariant measures for Markov chains

1998 ◽  
Vol 35 (03) ◽  
pp. 517-536 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Transition Matrix ◽  
Invariant Measures ◽  
Invariant Distribution ◽  
Special Cases ◽  
Recurrent Markov Chain ◽  
Positive Recurrent

Let P be the transition matrix of a positive recurrent Markov chain on the integers, with invariant distribution π. If (n) P denotes the n x n ‘northwest truncation’ of P, it is known that approximations to π(j)/π(0) can be constructed from (n) P, but these are known to converge to the probability distribution itself in special cases only. We show that such convergence always occurs for three further general classes of chains, geometrically ergodic chains, stochastically monotone chains, and those dominated by stochastically monotone chains. We show that all ‘finite’ perturbations of stochastically monotone chains can be considered to be dominated by such chains, and thus the results hold for a much wider class than is first apparent. In the cases of uniformly ergodic chains, and chains dominated by irreducible stochastically monotone chains, we find practical bounds on the accuracy of the approximations.

scholarly journals A note on repeated sequences in Markov chains

1987 ◽  
Vol 19 (03) ◽  
pp. 739-742 ◽  
Author(s):  
J. D. Biggins
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Renewal Process ◽  
Transition Matrix ◽  
Repeated Sequences ◽  
Markov Renewal Process ◽  
Conditional Mean ◽  
Sojourn Times

If (non-overlapping) repeats of specified sequences of states in a Markov chain are considered, the result is a Markov renewal process. Formulae somewhat simpler than those given in Biggins and Cannings (1987) are derived which can be used to obtain the transition matrix and conditional mean sojourn times in this process.

Single-shelf library-type Markov chains with infinitely many books

1977 ◽  
Vol 14 (02) ◽  
pp. 298-308 ◽  
Author(s):  
Peter R. Nelson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Natural Order ◽  
Order Book ◽  
Exit Boundary ◽  
One Step ◽  
Step Transition ◽  
The Right ◽  
Positive Recurrent

In a single-shelf library having infinitely many books B 1 , B 2 , …, the probability of selecting each book is assumed known. Books are removed one at a time and replaced in position k prior to the next removal. Books are moved either to the right or the left as is necessary to vacate position k. Those arrangements of books where after some finite position all the books are in natural order (book i occupies position i) are considered as states in an infinite Markov chain. When k > 1, we show that the chain can never be positive recurrent. When k = 1, we find the limits of ratios of one-step transition probabilities; and when k = 1 and the chain is transient, we find the Martin exit boundary.

scholarly journals Hitting Time and Inverse Problems for Markov Chains

2008 ◽  
Vol 45 (03) ◽  
pp. 640-649
Author(s):  
Victor de la Peña ◽  
Henryk Gzyl ◽  
Patrick McDonald
Keyword(s):  
Markov Chain ◽  
Inverse Problems ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Hitting Time ◽  
First Hitting Time ◽  
Joint Distributions ◽  
Finite Set

Let W n be a simple Markov chain on the integers. Suppose that X n is a simple Markov chain on the integers whose transition probabilities coincide with those of W n off a finite set. We prove that there is an M > 0 such that the Markov chain W n and the joint distributions of the first hitting time and first hitting place of X n started at the origin for the sets {-M, M} and {-(M + 1), (M + 1)} algorithmically determine the transition probabilities of X n .

The robustness of positive recurrence and recurrence of Markov chains under perturbations of the transition probabilities

1975 ◽  
Vol 12 (04) ◽  
pp. 744-752 ◽  
Author(s):  
Richard L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Transition Probabilities ◽  
General State ◽  
Transition Matrices ◽  
Positive Recurrence ◽  
Markov Chain Models ◽  
Countable Space ◽  
General State Space

In many Markov chain models, the immediate characteristic of importance is the positive recurrence of the chain. In this note we investigate whether positivity, and also recurrence, are robust properties of Markov chains when the transition laws are perturbed. The chains we consider are on a fairly general state space : when specialised to a countable space, our results are essentially that, if the transition matrices of two irreducible chains coincide on all but a finite number of columns, then positivity of one implies positivity of both; whilst if they coincide on all but a finite number of rows and columns, recurrence of one implies recurrence of both. Examples are given to show that these results (and their general analogues) cannot in general be strengthened.

Sign in / Sign up

Export Citation Format

Share Document