scholarly journals Markov chain properties in terms of column sums of the transition matrix

2012 ◽  
Vol 16 (1) ◽  
pp. 33-51
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Markov Chain ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Stochastic Matrix ◽  
Discrete Time Markov Chain ◽  
Mean First Passage Times ◽  
Generalized Matrix ◽  
The Mean ◽  
Generalized Matrix Inverse ◽  
Passage Times

Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the associated irreducible discrete time Markov chain. Some new relationships, including some inequalities, and partial answers to the questions, are given using a special generalized matrix inverse that has not previously been considered in the literature on Markov chains.

scholarly journals SIMPLE PROCEDURES FOR FINDING MEAN FIRST PASSAGE TIMES IN MARKOV CHAINS

2007 ◽  
Vol 24 (06) ◽  
pp. 813-829 ◽  
Author(s):  
JEFFREY J. HUNTER
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Linear Equations ◽  
First Passage Times ◽  
First Passage ◽  
Mean First Passage Times ◽  
The Mean ◽  
Computational Procedures ◽  
Passage Times

The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell's fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse of I - P + eπT where eT = (1, 1, …, 1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures.

scholarly journals ONE DIMENSIONAL ASYNCHRONOUS COOPERATIVE PARRONDO'S GAMES

2003 ◽  
Vol 03 (04) ◽  
pp. L389-L398 ◽  
Author(s):  
ZORAN MIHAILOVIĆ ◽  
MILAN RAJKOVIĆ
Keyword(s):  
Computer Simulation ◽  
Markov Chain ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Mean Field ◽  
Discrete Time Markov Chain ◽  
One Dimensional ◽  
Field Approach ◽  
The Mean ◽  
The One

A discrete-time Markov chain solution with exact rules for general computation of transition probabilities of the one-dimensional cooperative Parrondo's games is presented. We show that winning and the occurrence of the paradox depends on the number of players. Analytical results are compared to the results of the computer simulation and to the results based on the mean-field approach.

A Bayesian Approach in Estimating Transition Probabilities of a Discrete-time Markov Chain for Ignorable Intermittent Missing Data

2014 ◽  
Vol 45 (7) ◽  
pp. 2598-2616 ◽  
Author(s):  
Junsheng Ma ◽  
Xiaoying Yu ◽  
Elaine Symanski ◽  
Rachelle Doody ◽  
Wenyaw Chan
Keyword(s):  
Markov Chain ◽  
Missing Data ◽  
Discrete Time ◽  
Bayesian Approach ◽  
Transition Probabilities ◽  
Discrete Time Markov Chain

Estimating Transition Probabilities for Ignorable Intermittent Missing Data in a Discrete-Time Markov Chain

2010 ◽  
Vol 39 (2) ◽  
pp. 433-448 ◽  
Author(s):  
Hung-Wen Yeh ◽  
Wenyaw Chan ◽  
Elaine Symanski ◽  
Barry R. Davis
Keyword(s):  
Markov Chain ◽  
Missing Data ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Discrete Time Markov Chain

scholarly journals On the Embedding Problem for Discrete-Time Markov Chains

2013 ◽  
Vol 50 (04) ◽  
pp. 918-930 ◽  
Author(s):  
Marie-Anne Guerry
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Embedding Problem ◽  
Time Intervals ◽  
Square Roots ◽  
Probability Matrices

When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. This characterization is referred to in the literature as the embedding problem for discrete time Markov chains. The probability matrix which has probability root(s) is called embeddable. In this paper for two-state Markov chains, necessary and sufficient conditions for embeddability are formulated and the probability square roots of the transition matrix are presented in analytic form. In finding conditions for the existence of probability square roots for (k x k) transition matrices, properties of row-normalized matrices are examined. Besides the existence of probability square roots, the uniqueness of these solutions is discussed: In the case of nonuniqueness, a procedure is introduced to identify a transition matrix that takes into account the specificity of the concrete context. In the case of nonexistence of a probability root, the concept of an approximate probability root is introduced as a solution of an optimization problem related to approximate nonnegative matrix factorization.

A decomposition theorem for infinite stochastic matrices

1995 ◽  
Vol 32 (04) ◽  
pp. 893-901 ◽  
Author(s):  
Daniel P. Heyman
Keyword(s):  
Stochastic Matrix ◽  
Decomposition Theorem ◽  
Stochastic Matrices ◽  
First Passage ◽  
Mean First Passage Times ◽  
Expected Values ◽  
Lower Triangular ◽  
Infinite State ◽  
Steady State Probabilities ◽  
Passage Times

We prove that every infinite-state stochastic matrix P say, that is irreducible and consists of positive-recurrrent states can be represented in the form I – P=(A – I)(B – S), where A is strictly upper-triangular, B is strictly lower-triangular, and S is diagonal. Moreover, the elements of A are expected values of random variables that we will specify, and the elements of B and S are probabilities of events that we will specify. The decomposition can be used to obtain steady-state probabilities, mean first-passage-times and the fundamental matrix.

scholarly journals Markov Chain Modeling of HIV, Tuberculosis, and Hepatitis B Transmission in Ghana

2019 ◽  
Vol 2019 ◽  
pp. 1-8 ◽  
Author(s):  
Clement Twumasi ◽  
Louis Asiedu ◽  
Ezekiel N. N. Nortey
Keyword(s):  
Markov Chain ◽  
Hepatitis B ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Markov Chain Model ◽  
Chain Model ◽  
Disease Dynamics ◽  
Epidemiological Models ◽  
Discrete Time Markov Chain ◽  
Expected Time

Several mathematical and standard epidemiological models have been proposed in studying infectious disease dynamics. These models help to understand the spread of disease infections. However, most of these models are not able to estimate other relevant disease metrics such as probability of first infection and recovery as well as the expected time to infection and recovery for both susceptible and infected individuals. That is, most of the standard epidemiological models used in estimating transition probabilities (TPs) are not able to generalize the transition estimates of disease outcomes at discrete time steps for future predictions. This paper seeks to address the aforementioned problems through a discrete-time Markov chain model. Secondary datasets from cohort studies were collected on HIV, tuberculosis (TB), and hepatitis B (HB) cases from a regional hospital in Ghana. The Markov chain model revealed that hepatitis B was more infectious over time than tuberculosis and HIV even though the probability of first infection of these diseases was relatively low within the study population. However, individuals infected with HIV had comparatively lower life expectancies than those infected with tuberculosis and hepatitis B. Discrete-time Markov chain technique is recommended as viable for modeling disease dynamics in Ghana.

General transition probabilities for finite Markov chains

1964 ◽  
Vol 60 (1) ◽  
pp. 83-91 ◽  
Author(s):  
Marcel F. Neuts
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Initial State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

We consider a stationary discrete-time Markov chain with a finite number m of possible states which we designate by 1,…,m. We assume that at time t = 0 the process is in an initial state i with probability (i = 1,…, m) and such that and .

Recurrence times of draft-patterns from reservoirs

1975 ◽  
Vol 12 (03) ◽  
pp. 647-652 ◽  
Author(s):  
G. G. S. Pegram
Keyword(s):  
Markov Chain ◽  
Discrete Time ◽  
Recurrence Time ◽  
Reservoir Model ◽  
Discrete State ◽  
Discrete Time Markov Chain ◽  
Recurrence Times ◽  
Mean And Variance ◽  
The Mean ◽  
Run Theory

Expressions for the mean and variance of the recurrence time of non-overlapping draft-patterns of draft from a Moran Reservoir Model (discrete-state and discrete-time Markov chain) are derived using Feller's Renewal argument. In addition an expression for the mean recurrence time for self-overlapping patterns of draft is derived using run-theory.

scholarly journals The Limiting Behaviour of Hanski's Incidence Function Metapopulation Model

2014 ◽  
Vol 51 (2) ◽  
pp. 297-316 ◽  
Author(s):  
R. McVinish ◽  
P. K. Pollett
Keyword(s):  
Markov Chain ◽  
Transition Probabilities ◽  
Metapopulation Model ◽  
Function Model ◽  
Discrete Time Markov Chain ◽  
Incidence Function Model ◽  
Incidence Function ◽  
Limiting Behaviour ◽  
Metapopulation Models ◽  
Near Extinction

Hanski's incidence function model is one of the most widely used metapopulation models in ecology. It models the presence/absence of a species at spatially distinct habitat patches as a discrete-time Markov chain whose transition probabilities are determined by the physical landscape. In this analysis, the limiting behaviour of the model is studied as the number of patches increases and the size of the patches decreases. Two different limiting cases are identified depending on whether or not the metapopulation is initially near extinction. Basic properties of the limiting models are derived.

Sign in / Sign up

Export Citation Format

Share Document