A geometric invariant in weak lumpability of finite Markov chains

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

Download Full-text

Weak lumpability in finite Markov chains

Journal of Applied Probability ◽

10.1017/s0021900200037190 ◽

1982 ◽

Vol 19 (03) ◽

pp. 685-691 ◽

Cited By ~ 4

Author(s):

Atef M. Abdel-moneim ◽

Frederick W. Leysieffer

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Transition Probability ◽

Linear Equations ◽

Transition Probability Matrix ◽

Finite Markov Chain ◽

Systems Of Linear Equations ◽

Finite Markov Chains ◽

The Given

Criteria are given to determine whether a given finite Markov chain can be lumped weakly with respect to a given partition of its state space. These conditions are given in terms of solution classes of systems of linear equations associated with the transition probability matrix of the Markov chain and the given partition.

Download Full-text

Weak lumpability in finite Markov chains

Journal of Applied Probability ◽

10.2307/3213528 ◽

1982 ◽

Vol 19 (3) ◽

pp. 685-691 ◽

Cited By ~ 16

Author(s):

Atef M. Abdel-moneim ◽

Frederick W. Leysieffer

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Transition Probability ◽

Linear Equations ◽

Transition Probability Matrix ◽

Finite Markov Chain ◽

Systems Of Linear Equations ◽

Finite Markov Chains ◽

The Given

Criteria are given to determine whether a given finite Markov chain can be lumped weakly with respect to a given partition of its state space. These conditions are given in terms of solution classes of systems of linear equations associated with the transition probability matrix of the Markov chain and the given partition.

Download Full-text

Estimating the Entropy Rate of Finite Markov Chains With Application to Behavior Studies

Journal of Educational and Behavioral Statistics ◽

10.3102/1076998618822540 ◽

2019 ◽

Vol 44 (3) ◽

pp. 282-308 ◽

Cited By ~ 8

Author(s):

Brian G. Vegetabile ◽

Stephanie A. Stout-Oswald ◽

Elysia Poggi Davis ◽

Tallie Z. Baram ◽

Hal S. Stern

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Markov Processes ◽

Simulation Study ◽

Transition Matrix ◽

Sliding Window ◽

Entropy Rate ◽

Homogeneous Markov Chain ◽

The Third ◽

Finite Markov Chains

Predictability of behavior is an important characteristic in many fields including biology, medicine, marketing, and education. When a sequence of actions performed by an individual can be modeled as a stationary time-homogeneous Markov chain the predictability of the individual’s behavior can be quantified by the entropy rate of the process. This article compares three estimators of the entropy rate of finite Markov processes. The first two methods directly estimate the entropy rate through estimates of the transition matrix and stationary distribution of the process. The third method is related to the sliding-window Lempel–Ziv compression algorithm. The methods are compared via a simulation study and in the context of a study of interactions between mothers and their children.

Download Full-text

General transition probabilities for finite Markov chains

Mathematical Proceedings of the Cambridge Philosophical Society ◽

10.1017/s0305004100037488 ◽

1964 ◽

Vol 60 (1) ◽

pp. 83-91 ◽

Cited By ~ 5

Author(s):

Marcel F. Neuts

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Finite Number ◽

Discrete Time ◽

Transition Probabilities ◽

Initial State ◽

Discrete Time Markov Chain ◽

Finite Markov Chains

We consider a stationary discrete-time Markov chain with a finite number m of possible states which we designate by 1,…,m. We assume that at time t = 0 the process is in an initial state i with probability (i = 1,…, m) and such that and .

Download Full-text

The square root law in the embedding detection problem for Markov chains with unknown matrix of transition probabilities

Discrete Mathematics and Applications ◽

10.1515/dma-2019-0007 ◽

2019 ◽

Vol 29 (1) ◽

pp. 59-68

Author(s):

Artem V. Volgin

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Transition Probabilities ◽

Transition Probability ◽

Classical Model ◽

Transition Probability Matrix ◽

Square Root ◽

Detection Problem ◽

Asymptotic Growth

Abstract We consider the classical model of embeddings in a simple binary Markov chain with unknown transition probability matrix. We obtain conditions on the asymptotic growth of lengths of the original and embedded sequences sufficient for the consistency of the proposed statistical embedding detection test.

Download Full-text

Predicting the probability of transforming different classes of monthly droughts in Iran

10.21203/rs.3.rs-240328/v1 ◽

2021 ◽

Author(s):

Peyman Mahmoudi ◽

Allahbakhsh Rigi

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Drought Index ◽

First Passage Time ◽

Transition Probability ◽

Severe Drought ◽

Homogeneous Markov Chain ◽

Effective Drought Index ◽

Daily Precipitation Data ◽

Drought Classes

Abstract The main objective of this study was to predict the transition probability of different drought classes by applying Homogenous and non- Homogenous Markov chain models. The daily precipitation data of 40 synoptic stations in Iran, for a period of 35 years (1983–2018), was used to access the study objectives. The Effective Drought Index (EDI) was applied to categorize Iran’s droughts. With the implementation of cluster analysis on the daily values of effective drought index (EDI), it was observed that Iran can be divided into five separate regions based on the behavior of the time series of the studied stations. The spatial mean of the effective drought index (EDI) of each region was also calculated. After forming the transition frequency matrix, the dependent and correlated test of data was conducted via chi-square test. The results of this test confirmed the assumption that the various drought classes are correlated in five studied regions. Eventually, after adjusting the transition probability matrix for the studied regions, the homogenous and non-homogenous Markov chains were modeled and Markov characteristics of droughts were extracted including various class probabilities of drought severity, the average expected residence time in each drought class, the expected first passage time from various classes of droughts to the wet classes, and the short-term prediction of various drought classes. Regarding these climate areas, the results showed that the probability of each category is reduced as the severity of drought increases from poor drought category to severe and very severe drought. In the non-homogeneous Markov chain, the probability of each category of drought for winter, spring, and fall indicated that the probability of weak drought category is more than other categories. Since the obtained anticipating results are dependent on the early months, they were more accurate than those of the homogeneous Markov chain. In general, both Markov chains showed favorable results that can be very useful for water resource planners.

Download Full-text

On some mixing times for nonreversible finite Markov chains

Journal of Applied Probability ◽

10.1017/jpr.2017.21 ◽

2017 ◽

Vol 54 (2) ◽

pp. 627-637 ◽

Cited By ~ 3

Author(s):

Lu-Jing Huang ◽

Yong-Hua Mao

Keyword(s):

Markov Chains ◽

Transition Probability ◽

Transition Probability Matrix ◽

Affirmative Answer ◽

Hitting Time ◽

Mixing Times ◽

Reversible Transition ◽

Commute Time ◽

Finite Markov Chains

Abstract By adding a vorticity matrix to the reversible transition probability matrix, we show that the commute time and average hitting time are smaller than that of the original reversible one. In particular, we give an affirmative answer to a conjecture of Aldous and Fill (2002). Further quantitive properties are also studied for the nonreversible finite Markov chains.

Download Full-text

Markovian bounds on functions of finite Markov chains

Advances in Applied Probability ◽

10.1017/s0001867800010910 ◽

2001 ◽

Vol 33 (2) ◽

pp. 505-519 ◽

Cited By ~ 1

Author(s):

James Ledoux ◽

Laurent Truffet

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Discrete Time ◽

Sufficient Conditions ◽

Coupling Method ◽

Geometric Invariant ◽

Discrete Time Markov Chain ◽

Finite Markov Chains ◽

Stochastic Majorization ◽

Necessary And Sufficient

In this paper, we obtain Markovian bounds on a function of a homogeneous discrete time Markov chain. For deriving such bounds, we use well-known results on stochastic majorization of Markov chains and the Rogers–Pitman lumpability criterion. The proposed method of comparison between functions of Markov chains is not equivalent to generalized coupling method of Markov chains, although we obtain same kind of majorization. We derive necessary and sufficient conditions for existence of our Markovian bounds. We also discuss the choice of the geometric invariant related to the lumpability condition that we use.

Download Full-text