A NOTE ON RÉNYI'S ENTROPY RATE FOR TIME-INHOMOGENEOUS MARKOV CHAINS

2018 ◽  
Vol 33 (4) ◽  
pp. 579-590
Author(s):  
Wenxi Li ◽  
Zhongzhi Wang
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Entropy Rate ◽  
Transition Matrices ◽  
Frobenius Theorem ◽  
Primitive Matrix ◽  
Inhomogeneous Markov Chain ◽  
Renyi’S Entropy ◽  
Renyi’S Divergence

AbstractIn this note, we use the Perron–Frobenius theorem to obtain the Rényi's entropy rate for a time-inhomogeneous Markov chain whose transition matrices converge to a primitive matrix. As direct corollaries, we also obtain the Rényi's entropy rate for asymptotic circular Markov chain and the Rényi's divergence rate between two time-inhomogeneous Markov chains.

The robustness of positive recurrence and recurrence of Markov chains under perturbations of the transition probabilities

1975 ◽  
Vol 12 (04) ◽  
pp. 744-752 ◽  
Author(s):  
Richard L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Transition Probabilities ◽  
General State ◽  
Transition Matrices ◽  
Positive Recurrence ◽  
Markov Chain Models ◽  
Countable Space ◽  
General State Space

In many Markov chain models, the immediate characteristic of importance is the positive recurrence of the chain. In this note we investigate whether positivity, and also recurrence, are robust properties of Markov chains when the transition laws are perturbed. The chains we consider are on a fairly general state space : when specialised to a countable space, our results are essentially that, if the transition matrices of two irreducible chains coincide on all but a finite number of columns, then positivity of one implies positivity of both; whilst if they coincide on all but a finite number of rows and columns, recurrence of one implies recurrence of both. Examples are given to show that these results (and their general analogues) cannot in general be strengthened.

scholarly journals Stochastic and substochastic solutions for infinite-state Markov chains with applications to matrix-analytic methods

2008 ◽  
Vol 40 (04) ◽  
pp. 1157-1173
Author(s):  
Winfried K. Grassmann ◽  
Javad Tavakoli
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Matrices ◽  
Matrix Analytic Methods ◽  
Analytic Methods ◽  
Finite Queues ◽  
Qbd Processes ◽  
Stochastic Solution ◽  
Number Of Visits ◽  
Infinite State

This paper deals with censoring of infinite-state banded Markov chains. Censoring involves reducing the time spent in states outside a certain set of states to 0 without affecting the number of visits within this set. We show that, if all states are transient, there is, besides the standard censored Markov chain, a nonstandard censored Markov chain which is stochastic. Both the stochastic and the substochastic solutions are found by censoring a sequence of finite transition matrices. If all matrices in the sequence are stochastic, the stochastic solution arises in the limit, whereas the substochastic solution arises if the matrices in the sequence are substochastic. We also show that, if the Markov chain is recurrent, the only solution is the stochastic solution. Censoring is particularly fruitful when applied to quasi-birth-and-death (QBD) processes. It turns out that key matrices in such processes are not unique, a fact that has been observed by several authors. We note that the stochastic solution is important for the analysis of finite queues.

Infinite block-structured transition matrices and their properties

1998 ◽  
Vol 30 (2) ◽  
pp. 365-384 ◽  
Author(s):  
Yiqiang Q. Zhao ◽  
Wei Li ◽  
W. John Braun
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Block Structure ◽  
Sufficient Conditions ◽  
Characteristic Functions ◽  
Scalar Case ◽  
Transition Matrices ◽  
Positive Recurrent ◽  
Infinite State ◽  
Block Structured

In this paper, we study Markov chains with infinite state block-structured transition matrices, whose states are partitioned into levels according to the block structure, and various associated measures. Roughly speaking, these measures involve first passage times or expected numbers of visits to certain levels without hitting other levels. They are very important and often play a key role in the study of a Markov chain. Necessary and/or sufficient conditions are obtained for a Markov chain to be positive recurrent, recurrent, or transient in terms of these measures. Results are obtained for general irreducible Markov chains as well as those with transition matrices possessing some block structure. We also discuss the decomposition or the factorization of the characteristic equations of these measures. In the scalar case, we locate the zeros of these characteristic functions and therefore use these zeros to characterize a Markov chain. Examples and various remarks are given to illustrate some of the results.

scholarly journals An Entropy Rate Theorem for a Hidden Inhomogeneous Markov Chain

2017 ◽  
Vol 8 (1) ◽  
pp. 19-26
Author(s):  
Yao Qi-feng ◽  
Dong Yun ◽  
Wang Zhong-Zhi
Keyword(s):  
Stochastic Process ◽  
Markov Chain ◽  
Hidden Markov ◽  
The Other ◽  
Entropy Rate ◽  
Hidden Markov Chain ◽  
Mild Conditions ◽  
Ergodic Properties ◽  
Inhomogeneous Markov Chain ◽  
Flexible Application

Objective: The main object of our study is to extend some entropy rate theorems to a Hidden Inhomogeneous Markov Chain (HIMC) and establish an entropy rate theorem under some mild conditions. Introduction: A hidden inhomogeneous Markov chain contains two different stochastic processes; one is an inhomogeneous Markov chain whose states are hidden and the other is a stochastic process whose states are observable. Materials and Methods: The proof of theorem requires some ergodic properties of an inhomogeneous Markov chain, and the flexible application of the properties of norm and the bounded conditions of series are also indispensable. Results: This paper presents an entropy rate theorem for an HIMC under some mild conditions and two corollaries for a hidden Markov chain and an inhomogeneous Markov chain. Conclusion: Under some mild conditions, the entropy rates of an inhomogeneous Markov chains, a hidden Markov chain and an HIMC are similar and easy to calculate.

scholarly journals Estimating the Entropy Rate of Finite Markov Chains With Application to Behavior Studies

2019 ◽  
Vol 44 (3) ◽  
pp. 282-308 ◽  
Author(s):  
Brian G. Vegetabile ◽  
Stephanie A. Stout-Oswald ◽  
Elysia Poggi Davis ◽  
Tallie Z. Baram ◽  
Hal S. Stern
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Markov Processes ◽  
Simulation Study ◽  
Transition Matrix ◽  
Sliding Window ◽  
Entropy Rate ◽  
Homogeneous Markov Chain ◽  
The Third ◽  
Finite Markov Chains

Predictability of behavior is an important characteristic in many fields including biology, medicine, marketing, and education. When a sequence of actions performed by an individual can be modeled as a stationary time-homogeneous Markov chain the predictability of the individual’s behavior can be quantified by the entropy rate of the process. This article compares three estimators of the entropy rate of finite Markov processes. The first two methods directly estimate the entropy rate through estimates of the transition matrix and stationary distribution of the process. The third method is related to the sliding-window Lempel–Ziv compression algorithm. The methods are compared via a simulation study and in the context of a study of interactions between mothers and their children.

scholarly journals Stochastic and substochastic solutions for infinite-state Markov chains with applications to matrix-analytic methods

2008 ◽  
Vol 40 (4) ◽  
pp. 1157-1173 ◽  
Author(s):  
Winfried K. Grassmann ◽  
Javad Tavakoli
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Matrices ◽  
Matrix Analytic Methods ◽  
Analytic Methods ◽  
Finite Queues ◽  
Qbd Processes ◽  
Stochastic Solution ◽  
Number Of Visits ◽  
Infinite State

This paper deals with censoring of infinite-state banded Markov chains. Censoring involves reducing the time spent in states outside a certain set of states to 0 without affecting the number of visits within this set. We show that, if all states are transient, there is, besides the standard censored Markov chain, a nonstandard censored Markov chain which is stochastic. Both the stochastic and the substochastic solutions are found by censoring a sequence of finite transition matrices. If all matrices in the sequence are stochastic, the stochastic solution arises in the limit, whereas the substochastic solution arises if the matrices in the sequence are substochastic. We also show that, if the Markov chain is recurrent, the only solution is the stochastic solution. Censoring is particularly fruitful when applied to quasi-birth-and-death (QBD) processes. It turns out that key matrices in such processes are not unique, a fact that has been observed by several authors. We note that the stochastic solution is important for the analysis of finite queues.

The robustness of positive recurrence and recurrence of Markov chains under perturbations of the transition probabilities

1975 ◽  
Vol 12 (4) ◽  
pp. 744-752 ◽  
Author(s):  
Richard L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Transition Probabilities ◽  
General State ◽  
Transition Matrices ◽  
Positive Recurrence ◽  
Markov Chain Models ◽  
Countable Space ◽  
General State Space

In many Markov chain models, the immediate characteristic of importance is the positive recurrence of the chain. In this note we investigate whether positivity, and also recurrence, are robust properties of Markov chains when the transition laws are perturbed. The chains we consider are on a fairly general state space : when specialised to a countable space, our results are essentially that, if the transition matrices of two irreducible chains coincide on all but a finite number of columns, then positivity of one implies positivity of both; whilst if they coincide on all but a finite number of rows and columns, recurrence of one implies recurrence of both. Examples are given to show that these results (and their general analogues) cannot in general be strengthened.

Coefficients of ergodicity for stochastically monotone Markov chains

1992 ◽  
Vol 29 (4) ◽  
pp. 850-860 ◽  
Author(s):  
G. Ch. Pflug ◽  
W. Schachermayer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Probabilistic Interpretation ◽  
Speed Of Convergence ◽  
Transition Matrices ◽  
Wasserstein Metric ◽  
Ergodic Markov Chain ◽  
Finite State ◽  
Second Eigenvalue ◽  
Finite State Space

In this paper we show that to each distance d defined on the finite state space S of a strongly ergodic Markov chain there corresponds a coefficient ρd of ergodicity based on the Wasserstein metric. For a class of stochastically monotone transition matrices P, the infimum over all such coefficients is given by the spectral radius of P – R, where R = limkPk and is attained. This result has a probabilistic interpretation of a control of the speed of convergence of by the metric d and is linked to the second eigenvalue of P.

Non-abelian free group actions: Markov processes, the Abramov–Rohlin formula and Yuzvinskii’s formula

2009 ◽  
Vol 30 (6) ◽  
pp. 1629-1663 ◽  
Author(s):  
LEWIS BOWEN
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Markov Processes ◽  
Free Group ◽  
Skew Product ◽  
Addition Formula ◽  
Transition Matrices ◽  
Classical Formula ◽  
Algebraic Actions ◽  
Free Group Actions

AbstractThis paper introduces Markov chains and processes over non-abelian free groups and semigroups. We prove a formula for the f-invariant of a Markov chain over a free group in terms of transition matrices that parallels the classical formula for the entropy a Markov chain. Applications include free group analogues of the Abramov–Rohlin formula for skew-product actions and Yuzvinskii’s addition formula for algebraic actions.

Sign in / Sign up

Export Citation Format

Share Document