Occupation times for two state Markov chains

1971 ◽  
Vol 8 (2) ◽  
pp. 381-390 ◽  
Author(s):  
P. J. Pedler
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Random Variables ◽  
Time Parameter ◽  
Conditional Probabilities ◽  
Occupation Times ◽  
The Matrix ◽  
Image Position

Consider first a Markov chain with two ergodic states E1 and E2, and discrete time parameter set {0, 1, 2, ···, n}. Define the random variables Z0, Z1, Z2, ···, Znby then the conditional probabilities for k = 1,2,···, n, are independent of k. Thus the matrix of transition probabilities is

Occupation times for two state Markov chains

1971 ◽  
Vol 8 (02) ◽  
pp. 381-390 ◽  
Author(s):  
P. J. Pedler
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Random Variables ◽  
Time Parameter ◽  
Conditional Probabilities ◽  
Occupation Times ◽  
The Matrix ◽  
Image Position

Consider first a Markov chain with two ergodic states E 1 and E 2, and discrete time parameter set {0, 1, 2, ···, n}. Define the random variables Z 0, Z 1, Z 2, ···, Zn by then the conditional probabilities for k = 1,2,···, n, are independent of k. Thus the matrix of transition probabilities is

A Note on Approximating Mean Occupation Times of Continuous-Time Markov Chains

1988 ◽  
Vol 2 (2) ◽  
pp. 267-268
Author(s):  
Sheldon M. Ross
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Random Variables ◽  
Continuous Time Markov Chain ◽  
Occupation Times ◽  
Continuous Time Markov Chains ◽  
Exponential Random Variables

In [1] an approach to approximate the transition probabilities and mean occupation times of a continuous-time Markov chain is presented. For the chain under consideration, let Pij(t) and Tij(t) denote respectively the probability that it is in state j at time t, and the total time spent in j by time t, in both cases conditional on the chain starting in state i. Also, let Y1,…, Yn be independent exponential random variables each with rate λ = n/t, which are also independent of the Markov chain.

scholarly journals On the Embedding Problem for Discrete-Time Markov Chains

2013 ◽  
Vol 50 (04) ◽  
pp. 918-930 ◽  
Author(s):  
Marie-Anne Guerry
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Embedding Problem ◽  
Time Intervals ◽  
Square Roots ◽  
Probability Matrices

When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. This characterization is referred to in the literature as the embedding problem for discrete time Markov chains. The probability matrix which has probability root(s) is called embeddable. In this paper for two-state Markov chains, necessary and sufficient conditions for embeddability are formulated and the probability square roots of the transition matrix are presented in analytic form. In finding conditions for the existence of probability square roots for (k x k) transition matrices, properties of row-normalized matrices are examined. Besides the existence of probability square roots, the uniqueness of these solutions is discussed: In the case of nonuniqueness, a procedure is introduced to identify a transition matrix that takes into account the specificity of the concrete context. In the case of nonexistence of a probability root, the concept of an approximate probability root is introduced as a solution of an optimization problem related to approximate nonnegative matrix factorization.

On a class of non-homogeneous Markov chains

1982 ◽  
Vol 92 (3) ◽  
pp. 527-534 ◽  
Author(s):  
Harry Cohn
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Homogeneous Markov Chain ◽  
Image Position

AbstractSuppose that {Xn} is a countable non-homogeneous Markov chain andIf converges for any i, l, m, j with , thenwhenever lim , whereas if converges, thenwhere and . The behaviour of transition probabilities between various groups of states is studied and criteria for recurrence and transience are given.

General transition probabilities for finite Markov chains

1964 ◽  
Vol 60 (1) ◽  
pp. 83-91 ◽  
Author(s):  
Marcel F. Neuts
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Initial State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

We consider a stationary discrete-time Markov chain with a finite number m of possible states which we designate by 1,…,m. We assume that at time t = 0 the process is in an initial state i with probability (i = 1,…, m) and such that and .

scholarly journals Sequential Tests for Normal Markov Sequence

1971 ◽  
Vol 12 (4) ◽  
pp. 433-440 ◽  
Author(s):  
R. M. Phatarfod
Keyword(s):  
Markov Chain ◽  
Finite Number ◽  
Transition Probabilities ◽  
Random Variables ◽  
Important Case ◽  
Fundamental Identity ◽  
Markov Sequence ◽  
Sequential Tests ◽  
Image Position

This is a sequel to the author's (Phatarfod [9]) paper in which an analogue of Wald's Fundamental Identity (F.I.) for random variables defined on a Markov chain with a finite number of states was derived. From it the sampling properties of sequential tests of simple hypotheses about the parameters occurring in the transition probabilities were obtained. In this paper we consider the case of continuous Markovian variables. We restrict our attention to the practically important case of a Normal Markov sequence X0,X1,X2,… such that the Yr being independent normal variables with mean zero and variance σ2.

On sums of random variables defined on a two-state Markov chain

1970 ◽  
Vol 7 (3) ◽  
pp. 761-765 ◽  
Author(s):  
H. J. Helgert
Keyword(s):  
Markov Chain ◽  
Transition Probabilities ◽  
Random Variables ◽  
Homogeneous Markov Chain ◽  
Sums Of Random Variables ◽  
Image Position

Assume the sequence of random variables x0, x1, x2, ··· forms a two-state, homogeneous Markov chain with transition probabilities and initial probabilities

scholarly journals On the Embedding Problem for Discrete-Time Markov Chains

2013 ◽  
Vol 50 (4) ◽  
pp. 918-930 ◽  
Author(s):  
Marie-Anne Guerry
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Embedding Problem ◽  
Time Intervals ◽  
Square Roots ◽  
Probability Matrices

When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. This characterization is referred to in the literature as the embedding problem for discrete time Markov chains. The probability matrix which has probability root(s) is called embeddable.In this paper for two-state Markov chains, necessary and sufficient conditions for embeddability are formulated and the probability square roots of the transition matrix are presented in analytic form. In finding conditions for the existence of probability square roots for (kxk) transition matrices, properties of row-normalized matrices are examined. Besides the existence of probability square roots, the uniqueness of these solutions is discussed: In the case of nonuniqueness, a procedure is introduced to identify a transition matrix that takes into account the specificity of the concrete context. In the case of nonexistence of a probability root, the concept of an approximate probability root is introduced as a solution of an optimization problem related to approximate nonnegative matrix factorization.

A Matrix Factorization Problem in the Theory of Random Variables Defined on a Finite Markov Chain

1962 ◽  
Vol 58 (2) ◽  
pp. 268-285 ◽  
Author(s):  
H. D. Miller
Keyword(s):  
Markov Chain ◽  
Matrix Factorization ◽  
Transition Matrix ◽  
Random Variables ◽  
Finite Markov Chain ◽  
Stieltjes Transform ◽  
Factorization Problem ◽  
Regular Elements ◽  
The Matrix ◽  
Image Position

SummaryLet {kr} (r = 0, 1, 2, …; 1 ≤ kr ≤ h) be a positively regular, finite Markov chain with transition matrix P = (pjk). For each possible transition j → k let gjk(x)(− ∞ ≤ x ≤ ∞) be a given distribution function. The sequence of random variables {ξr} is defined where ξr has the distribution gjk(x) if the rth transition takes the chain from state j to state k. It is supposed that each distribution gjk(x) admits a two-sided Laplace-Stieltjes transform in a real t-interval surrounding t = 0. Let P(t) denote the matrix {Pjkmjk(t)}. It is shown, using probability arguments, that I − sP(t) admits a Wiener-Hopf type of factorization in two ways for suitable values of s where the plus-factors are non-singular, bounded and have regular elements in a right half of the complex t-plane and the minus-factors have similar properties in an overlapping left half-plane (Theorem 1).

On sums of random variables defined on a two-state Markov chain

1970 ◽  
Vol 7 (03) ◽  
pp. 761-765 ◽  
Author(s):  
H. J. Helgert
Keyword(s):  
Markov Chain ◽  
Transition Probabilities ◽  
Random Variables ◽  
Homogeneous Markov Chain ◽  
Sums Of Random Variables ◽  
Image Position

Assume the sequence of random variables x 0, x 1, x 2, ··· forms a two-state, homogeneous Markov chain with transition probabilities and initial probabilities

Sign in / Sign up

Export Citation Format

Share Document