Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons

2000 ◽  
Vol 37 (3) ◽  
pp. 795-806 ◽  
Author(s):  
Laurent Truffet
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Stochastic Comparison ◽  
Homogeneous Markov Chain ◽  
Monotone Functions ◽  
Stochastic Comparisons ◽  
Reduction Techniques ◽  
Ordered State

We propose in this paper two methods to compute Markovian bounds for monotone functions of a discrete time homogeneous Markov chain evolving in a totally ordered state space. The main interest of such methods is to propose algorithms to simplify analysis of transient characteristics such as the output process of a queue, or sojourn time in a subset of states. Construction of bounds are based on two kinds of results: well-known results on stochastic comparison between Markov chains with the same state space; and the fact that in some cases a function of Markov chain is again a homogeneous Markov chain but with smaller state space. Indeed, computation of bounds uses knowledge on the whole initial model. However, only part of this data is necessary at each step of the algorithms.

Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons

2000 ◽  
Vol 37 (03) ◽  
pp. 795-806 ◽  
Author(s):  
Laurent Truffet
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Stochastic Comparison ◽  
Homogeneous Markov Chain ◽  
Monotone Functions ◽  
Stochastic Comparisons ◽  
Reduction Techniques ◽  
Ordered State

We propose in this paper two methods to compute Markovian bounds for monotone functions of a discrete time homogeneous Markov chain evolving in a totally ordered state space. The main interest of such methods is to propose algorithms to simplify analysis of transient characteristics such as the output process of a queue, or sojourn time in a subset of states. Construction of bounds are based on two kinds of results: well-known results on stochastic comparison between Markov chains with the same state space; and the fact that in some cases a function of Markov chain is again a homogeneous Markov chain but with smaller state space. Indeed, computation of bounds uses knowledge on the whole initial model. However, only part of this data is necessary at each step of the algorithms.

Lumpability for non-irreducible finite markov chains

1984 ◽  
Vol 21 (03) ◽  
pp. 567-574 ◽  
Author(s):  
Atef M. Abdel-Moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
The State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

Conditions under which a function of a finite, discrete-time Markov chain, X(t), is again Markov are given, when X(t) is not irreducible. These conditions are given in terms of an interrelationship between two partitions of the state space of X(t), the partition induced by the minimal essential classes of X(t) and the partition with respect to which lumping is to be considered.

scholarly journals First Hitting Problems for Markov Chains That Converge to a Geometric Brownian Motion

2011 ◽  
Vol 2011 ◽  
pp. 1-15 ◽  
Author(s):  
Mario Lefebvre ◽  
Moussa Kounta
Keyword(s):  
Brownian Motion ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Geometric Brownian Motion ◽  
Expected Number ◽  
Discrete Time Markov Chain

We consider a discrete-time Markov chain with state space {1,1+Δx,…,1+kΔx=N}. We compute explicitly the probability pj that the chain, starting from 1+jΔx, will hit N before 1, as well as the expected number dj of transitions needed to end the game. In the limit when Δx and the time Δt between the transitions decrease to zero appropriately, the Markov chain tends to a geometric Brownian motion. We show that pj and djΔt tend to the corresponding quantities for the geometric Brownian motion.

scholarly journals On weak lumpability in Markov chains

1989 ◽  
Vol 26 (03) ◽  
pp. 446-457 ◽  
Author(s):  
Gerardo Rubino
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
Homogeneous Markov Chain ◽  
Past Work ◽  
The Past ◽  
The Subject

We analyse the conditions under which the aggregated process constructed from an homogeneous Markov chain over a given partition of the state space is also Markov homogeneous. The past work on the subject is revised and new properties are obtained.

scholarly journals On weak lumpability in Markov chains

1989 ◽  
Vol 26 (3) ◽  
pp. 446-457 ◽  
Author(s):  
Gerardo Rubino ◽  
Gerardo Rubino
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
Homogeneous Markov Chain ◽  
Past Work ◽  
The Past ◽  
The Subject

We analyse the conditions under which the aggregated process constructed from an homogeneous Markov chain over a given partition of the state space is also Markov homogeneous. The past work on the subject is revised and new properties are obtained.

Lumpability for non-irreducible finite markov chains

1984 ◽  
Vol 21 (3) ◽  
pp. 567-574 ◽  
Author(s):  
Atef M. Abdel-Moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
The State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

Conditions under which a function of a finite, discrete-time Markov chain, X(t), is again Markov are given, when X(t) is not irreducible. These conditions are given in terms of an interrelationship between two partitions of the state space of X(t), the partition induced by the minimal essential classes of X(t) and the partition with respect to which lumping is to be considered.

On the absorption probabilities and mean time for absorption for discrete Markov chains

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Nikolaos Halidias
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Generating Function ◽  
Discrete Time ◽  
Probability Generating Function ◽  
The Mean ◽  
Mean Time ◽  
Absorption Probabilities

Abstract In this note we study the probability and the mean time for absorption for discrete time Markov chains. In particular, we are interested in estimating the mean time for absorption when absorption is not certain and connect it with some other known results. Computing a suitable probability generating function, we are able to estimate the mean time for absorption when absorption is not certain giving some applications concerning the random walk. Furthermore, we investigate the probability for a Markov chain to reach a set A before reach B generalizing this result for a sequence of sets A 1 , A 2 , … , A k {A_{1},A_{2},\dots,A_{k}} .

Strong ergodicity for continuous-time, non-homogeneous Markov chains

1982 ◽  
Vol 19 (3) ◽  
pp. 692-694 ◽  
Author(s):  
Mark Scott ◽  
Barry C. Arnold ◽  
Dean L. Isaacson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Homogeneous Markov Chain ◽  
Strong Ergodicity

Characterizations of strong ergodicity for Markov chains using mean visit times have been found by several authors (Huang and Isaacson (1977), Isaacson and Arnold (1978)). In this paper a characterization of uniform strong ergodicity for a continuous-time non-homogeneous Markov chain is given. This extends the characterization, using mean visit times, that was given by Isaacson and Arnold.

Criteria for the non-ergodicity of stochastic processes: application to the exponential back-off protocol

1987 ◽  
Vol 24 (02) ◽  
pp. 347-354 ◽  
Author(s):  
Guy Fayolle ◽  
Rudolph Iasnogorodski
Keyword(s):  
Stochastic Process ◽  
Markov Chain ◽  
Stochastic Processes ◽  
State Space ◽  
Discrete Time ◽  
Random Variables ◽  
Countable State Space ◽  
Countable State ◽  
New Criteria

In this paper, we present some simple new criteria for the non-ergodicity of a stochastic process (Yn ), n ≧ 0 in discrete time, when either the upward or downward jumps are majorized by i.i.d. random variables. This situation is encountered in many practical situations, where the (Yn ) are functionals of some Markov chain with countable state space. An application to the exponential back-off protocol is described.

Criteria for classifying general Markov chains

1976 ◽  
Vol 8 (04) ◽  
pp. 737-771 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
General State ◽  
Classical Work ◽  
The Status ◽  
General State Space

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

Sign in / Sign up

Export Citation Format

Share Document