scholarly journals Multi-dimensional quasitoeplitz Markov chains

1999 ◽  
Vol 12 (4) ◽  
pp. 393-415 ◽  
Author(s):  
Alexander N. Dudin ◽  
Valentina I. Klimenok
Keyword(s):  
Markov Chains ◽  
Generating Function ◽  
Random Environment ◽  
Matrix Equation ◽  
Equilibrium Condition ◽  
Functional Matrix

This paper deals with multi-dimensional quasitoeplitz Markov chains. We establish a sufficient equilibrium condition and derive a functional matrix equation for the corresponding vector-generating function, whose solution is given algorithmically. The results are demonstrated in the form of examples and applications in queues with BMAP-input, which operate in synchronous random environment.

On the absorption probabilities and mean time for absorption for discrete Markov chains

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Nikolaos Halidias
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Generating Function ◽  
Discrete Time ◽  
Probability Generating Function ◽  
The Mean ◽  
Mean Time ◽  
Absorption Probabilities

Abstract In this note we study the probability and the mean time for absorption for discrete time Markov chains. In particular, we are interested in estimating the mean time for absorption when absorption is not certain and connect it with some other known results. Computing a suitable probability generating function, we are able to estimate the mean time for absorption when absorption is not certain giving some applications concerning the random walk. Furthermore, we investigate the probability for a Markov chain to reach a set A before reach B generalizing this result for a sequence of sets A 1 , A 2 , … , A k {A_{1},A_{2},\dots,A_{k}} .

On determining absorption probabilities for Markov chains in random environments

1981 ◽  
Vol 13 (2) ◽  
pp. 369-387 ◽  
Author(s):  
Richard D. Bourgin ◽  
Robert Cogburn
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Random Environment ◽  
Efficient Method ◽  
General Framework ◽  
Random Environments ◽  
Extinction Probabilities ◽  
Environmental Process ◽  
Branching Chains ◽  
Absorption Probabilities

The general framework of a Markov chain in a random environment is presented and the problem of determining extinction probabilities is discussed. An efficient method for determining absorption probabilities and criteria for certain absorption are presented in the case that the environmental process is a two-state Markov chain. These results are then applied to birth and death, queueing and branching chains in random environments.

THE SHANNON–MCMILLAN THEOREM FOR MARKOV CHAINS INDEXED BY A CAYLEY TREE IN RANDOM ENVIRONMENT

2017 ◽  
Vol 32 (4) ◽  
pp. 626-639 ◽  
Author(s):  
Zhiyan Shi ◽  
Pingping Zhong ◽  
Yan Fan
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Random Environment ◽  
Cayley Tree ◽  
Law Of Large Numbers ◽  
Strong Law ◽  
Countable State Space ◽  
Large Numbers ◽  
Countable State ◽  
Definition Of

In this paper, we give the definition of tree-indexed Markov chains in random environment with countable state space, and then study the realization of Markov chain indexed by a tree in random environment. Finally, we prove the strong law of large numbers and Shannon–McMillan theorem for Markov chains indexed by a Cayley tree in a Markovian environment with countable state space.

Random environment branching processes with equal environmental extinction probabilities

1973 ◽  
Vol 10 (03) ◽  
pp. 659-665
Author(s):  
Donald C. Raffety
Keyword(s):  
Markov Chains ◽  
Population Size ◽  
Random Environment ◽  
Transition Matrix ◽  
Branching Processes ◽  
Random Variables ◽  
Conditional Probabilities ◽  
Left Eigenvector ◽  
Extinction Probabilities ◽  
Limiting Values

R-positivity theory for Markov chains is used to obtain results for random environment branching processes whose environment random variables are independent and identically distributed and whose environmental extinction probabilities are equal. For certain processes whose eventual extinction is almost sure, it is shown that the distribution of population size conditioned by non-extinction at time n tends to a left eigenvector of the transition matrix. Limiting values of other conditional probabilities are given in terms of this left eigenvector and it is shown that the probability of non-extinction at time n approaches zero geometrically as n approaches ∞. Analogous results are obtained for processes whose extinction is not almost sure.

Newton-Shamanskii Method for a Quadratic Matrix Equation Arising in Quasi-Birth-Death Problems

2014 ◽  
Vol 4 (4) ◽  
pp. 386-395
Author(s):  
Pei-Chang Guo
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Matrix Equation ◽  
Nonnegative Solution ◽  
Minimal Nonnegative Solution ◽  
The Minimal Nonnegative Solution ◽  
Quadratic Matrix Equation ◽  
Quadratic Matrix ◽  
Birth Death

AbstractIn order to determine the stationary distribution for discrete time quasi-birth-death Markov chains, it is necessary to find the minimal nonnegative solution of a quadratic matrix equation. The Newton-Shamanskii method is applied to solve this equation, and the sequence of matrices produced is monotonically increasing and converges to its minimal nonnegative solution. Numerical results illustrate the effectiveness of this procedure.

Random environment branching processes with equal environmental extinction probabilities

1973 ◽  
Vol 10 (3) ◽  
pp. 659-665 ◽  
Author(s):  
Donald C. Raffety
Keyword(s):  
Markov Chains ◽  
Population Size ◽  
Random Environment ◽  
Transition Matrix ◽  
Branching Processes ◽  
Random Variables ◽  
Conditional Probabilities ◽  
Left Eigenvector ◽  
Extinction Probabilities ◽  
Limiting Values

R-positivity theory for Markov chains is used to obtain results for random environment branching processes whose environment random variables are independent and identically distributed and whose environmental extinction probabilities are equal. For certain processes whose eventual extinction is almost sure, it is shown that the distribution of population size conditioned by non-extinction at time n tends to a left eigenvector of the transition matrix. Limiting values of other conditional probabilities are given in terms of this left eigenvector and it is shown that the probability of non-extinction at time n approaches zero geometrically as n approaches ∞. Analogous results are obtained for processes whose extinction is not almost sure.

scholarly journals Light-tailed asymptotics of stationary probability vectors of Markov chains of GI/G/1 type

2005 ◽  
Vol 37 (04) ◽  
pp. 1075-1093 ◽  
Author(s):  
Quan-Lin Li ◽  
Yiqiang Q. Zhao
Keyword(s):  
Asymptotic Behavior ◽  
Markov Chain ◽  
Markov Chains ◽  
Asymptotic Analysis ◽  
Positive Solution ◽  
Generating Function ◽  
Spectral Properties ◽  
Stationary Probability ◽  
Probability Vector ◽  
Stationary Probability Vector

In this paper, we consider the asymptotic behavior of stationary probability vectors of Markov chains of GI/G/1 type. The generating function of the stationary probability vector is explicitly expressed by theR-measure. This expression of the generating function is more convenient for the asymptotic analysis than those in the literature. TheRG-factorization of both the repeating row and the Wiener-Hopf equations for the boundary row are used to provide necessary spectral properties. The stationary probability vector of a Markov chain of GI/G/1 type is shown to be light tailed if the blocks of the repeating row and the blocks of the boundary row are light tailed. We derive two classes of explicit expression for the asymptotic behavior, the geometric tail, and the semigeometric tail, based on the repeating row, the boundary row, or the minimal positive solution of a crucial equation involved in the generating function, and discuss the singularity classes of the stationary probability vector.

Sign in / Sign up

Export Citation Format

Share Document