scholarly journals Markov chains in random environment with applications in queuing theory and machine learning

2021 ◽  
Vol 137 ◽  
pp. 294-326
Author(s):  
Attila Lovas ◽  
Miklós Rásonyi
Keyword(s):  
Machine Learning ◽  
Markov Chains ◽  
Random Environment ◽  
Queuing Theory

On determining absorption probabilities for Markov chains in random environments

1981 ◽  
Vol 13 (2) ◽  
pp. 369-387 ◽  
Author(s):  
Richard D. Bourgin ◽  
Robert Cogburn
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Random Environment ◽  
Efficient Method ◽  
General Framework ◽  
Random Environments ◽  
Extinction Probabilities ◽  
Environmental Process ◽  
Branching Chains ◽  
Absorption Probabilities

The general framework of a Markov chain in a random environment is presented and the problem of determining extinction probabilities is discussed. An efficient method for determining absorption probabilities and criteria for certain absorption are presented in the case that the environmental process is a two-state Markov chain. These results are then applied to birth and death, queueing and branching chains in random environments.

THE SHANNON–MCMILLAN THEOREM FOR MARKOV CHAINS INDEXED BY A CAYLEY TREE IN RANDOM ENVIRONMENT

2017 ◽  
Vol 32 (4) ◽  
pp. 626-639 ◽  
Author(s):  
Zhiyan Shi ◽  
Pingping Zhong ◽  
Yan Fan
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Random Environment ◽  
Cayley Tree ◽  
Law Of Large Numbers ◽  
Strong Law ◽  
Countable State Space ◽  
Large Numbers ◽  
Countable State ◽  
Definition Of

In this paper, we give the definition of tree-indexed Markov chains in random environment with countable state space, and then study the realization of Markov chain indexed by a tree in random environment. Finally, we prove the strong law of large numbers and Shannon–McMillan theorem for Markov chains indexed by a Cayley tree in a Markovian environment with countable state space.

Markov Chains and Queuing Theory

2021 ◽  
pp. 235-282
Author(s):  
Giovanni Giambene
Keyword(s):  
Markov Chains ◽  
Queuing Theory

Random environment branching processes with equal environmental extinction probabilities

1973 ◽  
Vol 10 (03) ◽  
pp. 659-665
Author(s):  
Donald C. Raffety
Keyword(s):  
Markov Chains ◽  
Population Size ◽  
Random Environment ◽  
Transition Matrix ◽  
Branching Processes ◽  
Random Variables ◽  
Conditional Probabilities ◽  
Left Eigenvector ◽  
Extinction Probabilities ◽  
Limiting Values

R-positivity theory for Markov chains is used to obtain results for random environment branching processes whose environment random variables are independent and identically distributed and whose environmental extinction probabilities are equal. For certain processes whose eventual extinction is almost sure, it is shown that the distribution of population size conditioned by non-extinction at time n tends to a left eigenvector of the transition matrix. Limiting values of other conditional probabilities are given in terms of this left eigenvector and it is shown that the probability of non-extinction at time n approaches zero geometrically as n approaches ∞. Analogous results are obtained for processes whose extinction is not almost sure.

Random environment branching processes with equal environmental extinction probabilities

1973 ◽  
Vol 10 (3) ◽  
pp. 659-665 ◽  
Author(s):  
Donald C. Raffety
Keyword(s):  
Markov Chains ◽  
Population Size ◽  
Random Environment ◽  
Transition Matrix ◽  
Branching Processes ◽  
Random Variables ◽  
Conditional Probabilities ◽  
Left Eigenvector ◽  
Extinction Probabilities ◽  
Limiting Values

R-positivity theory for Markov chains is used to obtain results for random environment branching processes whose environment random variables are independent and identically distributed and whose environmental extinction probabilities are equal. For certain processes whose eventual extinction is almost sure, it is shown that the distribution of population size conditioned by non-extinction at time n tends to a left eigenvector of the transition matrix. Limiting values of other conditional probabilities are given in terms of this left eigenvector and it is shown that the probability of non-extinction at time n approaches zero geometrically as n approaches ∞. Analogous results are obtained for processes whose extinction is not almost sure.

MONOTONICITY AND CONVEXITY OF SOME FUNCTIONS ASSOCIATED WITH DENUMERABLE MARKOV CHAINS AND THEIR APPLICATIONS TO QUEUING SYSTEMS

2005 ◽  
Vol 20 (1) ◽  
pp. 67-86 ◽  
Author(s):  
Hai-Bo Yu ◽  
Qi-Ming He ◽  
Hanqin Zhang
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Stochastic Dominance ◽  
Continuous Time ◽  
Queue Length ◽  
Queuing Theory ◽  
Queuing Systems ◽  
Discrete Time Case ◽  
Monotone Matrix

Motivated by various applications in queuing theory, this article is devoted to the monotonicity and convexity of some functions associated with discrete-time or continuous-time denumerable Markov chains. For the discrete-time case, conditions for the monotonicity and convexity of the functions are obtained by using the properties of stochastic dominance and monotone matrix. For the continuous-time case, by using the uniformization technique, similar results are obtained. As an application, the results are applied to analyze the monotonicity and convexity of functions associated with the queue length of some queuing systems.

scholarly journals Multi-dimensional quasitoeplitz Markov chains

1999 ◽  
Vol 12 (4) ◽  
pp. 393-415 ◽  
Author(s):  
Alexander N. Dudin ◽  
Valentina I. Klimenok
Keyword(s):  
Markov Chains ◽  
Generating Function ◽  
Random Environment ◽  
Matrix Equation ◽  
Equilibrium Condition ◽  
Functional Matrix

This paper deals with multi-dimensional quasitoeplitz Markov chains. We establish a sufficient equilibrium condition and derive a functional matrix equation for the corresponding vector-generating function, whose solution is given algorithmically. The results are demonstrated in the form of examples and applications in queues with BMAP-input, which operate in synchronous random environment.

Sign in / Sign up

Export Citation Format

Share Document