scholarly journals Linear dynamics for the state vector of Markov chain functions

2004 ◽  
Vol 36 (4) ◽  
pp. 1198-1211 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chain ◽  
State Vector ◽  
Probability Distributions ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
The State ◽  
Linear Dynamics ◽  
Deterministic Function ◽  
Finite State ◽  
Definition Of

Let (φ(Xn))n be a function of a finite-state Markov chain (Xn)n. In this article, we investigate the conditions under which the random variables φ(n) have the same distribution as Yn (for every n), where (Yn)n is a Markov chain with fixed transition probability matrix. In other words, for a deterministic function φ, we investigate the conditions under which (Xn)n is weakly lumpable for the state vector. We show that the set of all probability distributions of X0, such that (Xn)n is weakly lumpable for the state vector, can be finitely generated. The connections between our definition of lumpability and the usual one (i.e. as the proportional dynamics property) are discussed.

scholarly journals Linear dynamics for the state vector of Markov chain functions

2004 ◽  
Vol 36 (04) ◽  
pp. 1198-1211
Author(s):  
James Ledoux
Keyword(s):  
Markov Chain ◽  
State Vector ◽  
Probability Distributions ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
The State ◽  
Linear Dynamics ◽  
Deterministic Function ◽  
Finite State ◽  
Definition Of

Let (φ(X n )) n be a function of a finite-state Markov chain (X n ) n . In this article, we investigate the conditions under which the random variables φ( n ) have the same distribution as Y n (for every n), where (Y n ) n is a Markov chain with fixed transition probability matrix. In other words, for a deterministic function φ, we investigate the conditions under which (X n ) n is weakly lumpable for the state vector. We show that the set of all probability distributions of X 0, such that (X n ) n is weakly lumpable for the state vector, can be finitely generated. The connections between our definition of lumpability and the usual one (i.e. as the proportional dynamics property) are discussed.

scholarly journals Complex Probability and Markov Stochastic Process

2019 ◽  
Vol 3 (1) ◽  
pp. 13-22
Author(s):  
Bijan Bidabad ◽  
Behrouz Bidabad
Keyword(s):  
Stochastic Process ◽  
Markov Chain ◽  
Real World ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
The Real ◽  
Definition Of ◽  
Discrete Markov Chain

This note discusses the existence of "complex probability" in the real world sensible problems. By defining a measure more general than the conventional definition of probability, the transition probability matrix of discrete Markov chain is broken to the periods shorter than a complete step of the transition. In this regard, the complex probability is implied.

Some explicit results for correlated random walks

1989 ◽  
Vol 26 (4) ◽  
pp. 757-766 ◽  
Author(s):  
Ram Lal ◽  
U. Narayan Bhat
Keyword(s):  
Markov Chain ◽  
State Space ◽  
Equilibrium Distribution ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Correlated Random Walks ◽  
Finite State ◽  
Direction Of Movement ◽  
Step Transition

In a correlated random walk (CRW) the probabilities of movement in the positive and negative direction are given by the transition probabilities of a Markov chain. The walk can be represented as a Markov chain if we use a bivariate state space, with the location of the particle and the direction of movement as the two variables. In this paper we derive explicit results for the following characteristics of the walk directly from its transition probability matrix: (i) n -step transition probabilities for the unrestricted CRW, (ii) equilibrium distribution for the CRW restricted on one side, and (iii) equilibrium distribution and first-passage characteristics for the CRW restricted on both sides (i.e., with finite state space).

Some explicit results for correlated random walks

1989 ◽  
Vol 26 (04) ◽  
pp. 757-766 ◽  
Author(s):  
Ram Lal ◽  
U. Narayan Bhat
Keyword(s):  
Markov Chain ◽  
State Space ◽  
Equilibrium Distribution ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Correlated Random Walks ◽  
Finite State ◽  
Direction Of Movement ◽  
Step Transition

In a correlated random walk (CRW) the probabilities of movement in the positive and negative direction are given by the transition probabilities of a Markov chain. The walk can be represented as a Markov chain if we use a bivariate state space, with the location of the particle and the direction of movement as the two variables. In this paper we derive explicit results for the following characteristics of the walk directly from its transition probability matrix: (i) n -step transition probabilities for the unrestricted CRW, (ii) equilibrium distribution for the CRW restricted on one side, and (iii) equilibrium distribution and first-passage characteristics for the CRW restricted on both sides (i.e., with finite state space).

scholarly journals Correlated random walks with stay

1988 ◽  
Vol 1 (3) ◽  
pp. 197-222
Author(s):  
Ram Lal ◽  
U. Narayan Bhat
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Random Walks ◽  
Equilibrium Solution ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Finite State ◽  
Direction Of Movement ◽  
One Step ◽  
Bivariate Markov Chain

A random walk describes the movement of a particle in discrete time, with the direction and the distance traversed in one step being governed by a probability distribution. In a correlated random walk (CRW) the movement follows a Markov chain and induces correlation in the state of the walk at various epochs. Then, the walk can be modelled as a bivariate Markov chain with the location of the particle and the direction of movement as the two variables. In such random walks, normally, the particle is not allowed to stay at one location from one step to the next. In this paper we derive explicit results for the following characteristics of the CRW when it is allowed to stay at the same location, directly from its transition probability matrix: (i) equilibrium solution and the fast passage probabilities for the CRW restricted on one side, and (ii) equilibrium solution and first passage characteristics for the CRW restricted on bath sides (i.e., with finite state space).

Some distribution and moment formulae for the Markov renewal process

1970 ◽  
Vol 68 (1) ◽  
pp. 159-166 ◽  
Author(s):  
A. M. Kshirsagar ◽  
R. Wysocki
Keyword(s):  
Distribution Function ◽  
Markov Chain ◽  
Renewal Process ◽  
Transition Probability ◽  
Random Variable ◽  
Transition Probability Matrix ◽  
The State ◽  
Holding Time ◽  
Markov Renewal Process ◽  
Time Required

1. Introduction. A Markov Renewal Process (MRP) with m(<∞) states is one which records at each time t, the number of times a system visits each of the m states up to time t, if the system moves from state to state according to a Markov chain with transition probability matrix P0 = [pij] and if the time required for each successive move is a random variable whose distribution function (d.f.) depends on the two states between which the move is made. Thus, if the system moves from state i to state j, the holding time in the state i has Fij(x) as its d.f. (i, j = 1,2, …, m).

The censored Markov chain and the best augmentation

1996 ◽  
Vol 33 (03) ◽  
pp. 623-629 ◽  
Author(s):  
Y. Quennel Zhao ◽  
Danielle Liu
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Stationary Probability ◽  
Stationary Probability Distribution ◽  
The Best Approximation ◽  
Finite Matrix ◽  
Countable State ◽  
Stationary Probabilities

Computationally, when we solve for the stationary probabilities for a countable-state Markov chain, the transition probability matrix of the Markov chain has to be truncated, in some way, into a finite matrix. Different augmentation methods might be valid such that the stationary probability distribution for the truncated Markov chain approaches that for the countable Markov chain as the truncation size gets large. In this paper, we prove that the censored (watched) Markov chain provides the best approximation in the sense that, for a given truncation size, the sum of errors is the minimum and show, by examples, that the method of augmenting the last column only is not always the best.

Decoding and modelling of time series count data using Poisson hidden Markov model and Markov ordinal logistic regression models

2018 ◽  
Vol 28 (5) ◽  
pp. 1552-1563 ◽  
Author(s):  
Tunny Sebastian ◽  
Visalakshi Jeyaseelan ◽  
Lakshmanan Jeyaseelan ◽  
Shalini Anandan ◽  
Sebastian George ◽  
...  
Keyword(s):  
Logistic Regression ◽  
Markov Chain ◽  
Hidden Markov Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Transition Probability ◽  
Passage Time ◽  
Transition Probability Matrix ◽  
Ordinal Logistic Regression ◽  
The Mean

Hidden Markov models are stochastic models in which the observations are assumed to follow a mixture distribution, but the parameters of the components are governed by a Markov chain which is unobservable. The issues related to the estimation of Poisson-hidden Markov models in which the observations are coming from mixture of Poisson distributions and the parameters of the component Poisson distributions are governed by an m-state Markov chain with an unknown transition probability matrix are explained here. These methods were applied to the data on Vibrio cholerae counts reported every month for 11-year span at Christian Medical College, Vellore, India. Using Viterbi algorithm, the best estimate of the state sequence was obtained and hence the transition probability matrix. The mean passage time between the states were estimated. The 95% confidence interval for the mean passage time was estimated via Monte Carlo simulation. The three hidden states of the estimated Markov chain are labelled as ‘Low’, ‘Moderate’ and ‘High’ with the mean counts of 1.4, 6.6 and 20.2 and the estimated average duration of stay of 3, 3 and 4 months, respectively. Environmental risk factors were studied using Markov ordinal logistic regression analysis. No significant association was found between disease severity levels and climate components.

scholarly journals Frekuensi Hari Hujan Menurut Bulan di Kota Balikpapan dengan Rantai Markov Waktu Diskrit

2019 ◽  
Vol 1 (2) ◽  
pp. 5-10
Author(s):  
Muhammad Azka
Keyword(s):  
Markov Chain ◽  
Discrete Time ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Transition Diagram ◽  
Discrete Time Markov Chain ◽  
Applied Method ◽  
Frequency Rate

The problem proposed in this research is about the amount rainy day per a month at Balikpapan city and discretetime markov chain. The purpose is finding the probability of rainy day with the frequency rate of rainy at the next month if given the frequency rate of rainy at the prior month. The applied method in this research is classifying the amount of rainy day be three frequency levels, those are, high, medium, and low. If a month, the amount of rainy day is less than 11 then the frequency rate for the month is classified low, if a month, the amount of rainy day between 10 and 20, then it is classified medium and if it is more than 20, then it is classified high. The result is discrete-time markov chain represented with the transition probability matrix, and the transition diagram.

Sign in / Sign up

Export Citation Format

Share Document