Renewal-type behavior of absorption times in Markov chains

1994 ◽  
Vol 26 (4) ◽  
pp. 988-1005 ◽  
Author(s):  
Bernard Van Cutsem ◽  
Bernard Ycart
Keyword(s):  
Asymptotic Behavior ◽  
Markov Chain ◽  
Limit Theorem ◽  
Markov Chains ◽  
Transition Matrix ◽  
Renewal Theory ◽  
Stochastic Comparison ◽  
Absorption Time ◽  
Starting Point ◽  
Lower Triangular

This paper studies the absorption time of an integer-valued Markov chain with a lower-triangular transition matrix. The main results concern the asymptotic behavior of the absorption time when the starting point tends to infinity (asymptotics of moments and central limit theorem). They are obtained using stochastic comparison for Markov chains and the classical theorems of renewal theory. Applications to the description of large random chains of partitions and large random ordered partitions are given.

Renewal-type behavior of absorption times in Markov chains

1994 ◽  
Vol 26 (04) ◽  
pp. 988-1005 ◽  
Author(s):  
Bernard Van Cutsem ◽  
Bernard Ycart
Keyword(s):  
Asymptotic Behavior ◽  
Markov Chain ◽  
Limit Theorem ◽  
Markov Chains ◽  
Transition Matrix ◽  
Renewal Theory ◽  
Stochastic Comparison ◽  
Absorption Time ◽  
Starting Point ◽  
Lower Triangular

This paper studies the absorption time of an integer-valued Markov chain with a lower-triangular transition matrix. The main results concern the asymptotic behavior of the absorption time when the starting point tends to infinity (asymptotics of moments and central limit theorem). They are obtained using stochastic comparison for Markov chains and the classical theorems of renewal theory. Applications to the description of large random chains of partitions and large random ordered partitions are given.

Truncation approximations of invariant measures for Markov chains

1998 ◽  
Vol 35 (03) ◽  
pp. 517-536 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Transition Matrix ◽  
Invariant Measures ◽  
Invariant Distribution ◽  
Special Cases ◽  
Recurrent Markov Chain ◽  
Positive Recurrent

Let P be the transition matrix of a positive recurrent Markov chain on the integers, with invariant distribution π. If (n) P denotes the n x n ‘northwest truncation’ of P, it is known that approximations to π(j)/π(0) can be constructed from (n) P, but these are known to converge to the probability distribution itself in special cases only. We show that such convergence always occurs for three further general classes of chains, geometrically ergodic chains, stochastically monotone chains, and those dominated by stochastically monotone chains. We show that all ‘finite’ perturbations of stochastically monotone chains can be considered to be dominated by such chains, and thus the results hold for a much wider class than is first apparent. In the cases of uniformly ergodic chains, and chains dominated by irreducible stochastically monotone chains, we find practical bounds on the accuracy of the approximations.

scholarly journals A note on repeated sequences in Markov chains

1987 ◽  
Vol 19 (03) ◽  
pp. 739-742 ◽  
Author(s):  
J. D. Biggins
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Renewal Process ◽  
Transition Matrix ◽  
Repeated Sequences ◽  
Markov Renewal Process ◽  
Conditional Mean ◽  
Sojourn Times

If (non-overlapping) repeats of specified sequences of states in a Markov chain are considered, the result is a Markov renewal process. Formulae somewhat simpler than those given in Biggins and Cannings (1987) are derived which can be used to obtain the transition matrix and conditional mean sojourn times in this process.

Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons

2000 ◽  
Vol 37 (03) ◽  
pp. 795-806 ◽  
Author(s):  
Laurent Truffet
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Stochastic Comparison ◽  
Homogeneous Markov Chain ◽  
Monotone Functions ◽  
Stochastic Comparisons ◽  
Reduction Techniques ◽  
Ordered State

We propose in this paper two methods to compute Markovian bounds for monotone functions of a discrete time homogeneous Markov chain evolving in a totally ordered state space. The main interest of such methods is to propose algorithms to simplify analysis of transient characteristics such as the output process of a queue, or sojourn time in a subset of states. Construction of bounds are based on two kinds of results: well-known results on stochastic comparison between Markov chains with the same state space; and the fact that in some cases a function of Markov chain is again a homogeneous Markov chain but with smaller state space. Indeed, computation of bounds uses knowledge on the whole initial model. However, only part of this data is necessary at each step of the algorithms.

Compound Poisson limit theorems for Markov chains

1997 ◽  
Vol 34 (1) ◽  
pp. 24-34 ◽  
Author(s):  
Shoou-Ren Hsiau
Keyword(s):  
Markov Chain ◽  
Limit Theorem ◽  
Markov Chains ◽  
Limit Theorems ◽  
Compound Poisson ◽  
Poisson Limit

This paper establishes a compound Poisson limit theorem for the sum of a sequence of multi-state Markov chains. Our theorem generalizes an earlier one by Koopman for the two-state Markov chain. Moreover, a similar approach is used to derive a limit theorem for the sum of the k th-order two-state Markov chain.

scholarly journals On the Embedding Problem for Discrete-Time Markov Chains

2013 ◽  
Vol 50 (04) ◽  
pp. 918-930 ◽  
Author(s):  
Marie-Anne Guerry
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Embedding Problem ◽  
Time Intervals ◽  
Square Roots ◽  
Probability Matrices

When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. This characterization is referred to in the literature as the embedding problem for discrete time Markov chains. The probability matrix which has probability root(s) is called embeddable. In this paper for two-state Markov chains, necessary and sufficient conditions for embeddability are formulated and the probability square roots of the transition matrix are presented in analytic form. In finding conditions for the existence of probability square roots for (k x k) transition matrices, properties of row-normalized matrices are examined. Besides the existence of probability square roots, the uniqueness of these solutions is discussed: In the case of nonuniqueness, a procedure is introduced to identify a transition matrix that takes into account the specificity of the concrete context. In the case of nonexistence of a probability root, the concept of an approximate probability root is introduced as a solution of an optimization problem related to approximate nonnegative matrix factorization.

scholarly journals Light-tailed asymptotics of stationary probability vectors of Markov chains of GI/G/1 type

2005 ◽  
Vol 37 (04) ◽  
pp. 1075-1093 ◽  
Author(s):  
Quan-Lin Li ◽  
Yiqiang Q. Zhao
Keyword(s):  
Asymptotic Behavior ◽  
Markov Chain ◽  
Markov Chains ◽  
Asymptotic Analysis ◽  
Positive Solution ◽  
Generating Function ◽  
Spectral Properties ◽  
Stationary Probability ◽  
Probability Vector ◽  
Stationary Probability Vector

In this paper, we consider the asymptotic behavior of stationary probability vectors of Markov chains of GI/G/1 type. The generating function of the stationary probability vector is explicitly expressed by theR-measure. This expression of the generating function is more convenient for the asymptotic analysis than those in the literature. TheRG-factorization of both the repeating row and the Wiener-Hopf equations for the boundary row are used to provide necessary spectral properties. The stationary probability vector of a Markov chain of GI/G/1 type is shown to be light tailed if the blocks of the repeating row and the blocks of the boundary row are light tailed. We derive two classes of explicit expression for the asymptotic behavior, the geometric tail, and the semigeometric tail, based on the repeating row, the boundary row, or the minimal positive solution of a crucial equation involved in the generating function, and discuss the singularity classes of the stationary probability vector.

scholarly journals Estimating the Entropy Rate of Finite Markov Chains With Application to Behavior Studies

2019 ◽  
Vol 44 (3) ◽  
pp. 282-308 ◽  
Author(s):  
Brian G. Vegetabile ◽  
Stephanie A. Stout-Oswald ◽  
Elysia Poggi Davis ◽  
Tallie Z. Baram ◽  
Hal S. Stern
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Markov Processes ◽  
Simulation Study ◽  
Transition Matrix ◽  
Sliding Window ◽  
Entropy Rate ◽  
Homogeneous Markov Chain ◽  
The Third ◽  
Finite Markov Chains

Predictability of behavior is an important characteristic in many fields including biology, medicine, marketing, and education. When a sequence of actions performed by an individual can be modeled as a stationary time-homogeneous Markov chain the predictability of the individual’s behavior can be quantified by the entropy rate of the process. This article compares three estimators of the entropy rate of finite Markov processes. The first two methods directly estimate the entropy rate through estimates of the transition matrix and stationary distribution of the process. The third method is related to the sliding-window Lempel–Ziv compression algorithm. The methods are compared via a simulation study and in the context of a study of interactions between mothers and their children.

Asymptotic behavior of the distribution of the absorption time of a Markov chain

Cybernetics ◽  
1974 ◽  
Vol 8 (2) ◽  
pp. 193-196
Author(s):  
V. S. Korolyuk ◽  
I. P. Penev ◽  
A. F. Turbin
Keyword(s):  
Asymptotic Behavior ◽  
Markov Chain ◽  
Absorption Time

scholarly journals Light-tailed asymptotics of stationary probability vectors of Markov chains of GI/G/1 type

2005 ◽  
Vol 37 (4) ◽  
pp. 1075-1093 ◽  
Author(s):  
Quan-Lin Li ◽  
Yiqiang Q. Zhao
Keyword(s):  
Asymptotic Behavior ◽  
Markov Chain ◽  
Markov Chains ◽  
Asymptotic Analysis ◽  
Positive Solution ◽  
Generating Function ◽  
Spectral Properties ◽  
Stationary Probability ◽  
Probability Vector ◽  
Stationary Probability Vector

In this paper, we consider the asymptotic behavior of stationary probability vectors of Markov chains of GI/G/1 type. The generating function of the stationary probability vector is explicitly expressed by the R-measure. This expression of the generating function is more convenient for the asymptotic analysis than those in the literature. The RG-factorization of both the repeating row and the Wiener-Hopf equations for the boundary row are used to provide necessary spectral properties. The stationary probability vector of a Markov chain of GI/G/1 type is shown to be light tailed if the blocks of the repeating row and the blocks of the boundary row are light tailed. We derive two classes of explicit expression for the asymptotic behavior, the geometric tail, and the semigeometric tail, based on the repeating row, the boundary row, or the minimal positive solution of a crucial equation involved in the generating function, and discuss the singularity classes of the stationary probability vector.

Sign in / Sign up

Export Citation Format

Share Document