scholarly journals Imprecise Discrete-Time Markov Chains

2021 ◽  
pp. 51-65
Author(s):  
Gert de Cooman
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Finite State ◽  
Finite State Space

AbstractI present a short and easy introduction to a number of basic definitions and important results from the theory of imprecise Markov chains in discrete time, with a finite state space. The approach is intuitive and graphical.

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

Extreme values, range and weak convergence of integrals of Markov chains

1982 ◽  
Vol 19 (02) ◽  
pp. 272-288 ◽  
Author(s):  
P. J. Brockwell ◽  
S. I. Resnick ◽  
N. Pacheco-Santiago
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
State Space ◽  
Extreme Values ◽  
Finite State ◽  
Integral Process ◽  
Maximum Minimum ◽  
Finite State Space

A study is made of the maximum, minimum and range on [0,t] of the integral processwhereSis a finite state-space Markov chain. Approximate results are derived by establishing weak convergence of a sequence of such processes to a Wiener process. For a particular family of two-state stationary Markov chains we show that the corresponding centered integral processes exhibit the Hurst phenomenon to a remarkable degree in their pre-asymptotic behaviour.

A discrete-time storage process with a general release rule

1989 ◽  
Vol 26 (03) ◽  
pp. 566-583
Author(s):  
John E. Glynn
Keyword(s):  
State Space ◽  
Discrete Time ◽  
Storage System ◽  
Simple Condition ◽  
Storage Process ◽  
Finite State ◽  
Stationary Behaviour ◽  
Finite State Space

A discrete-time storage system with a general release rule and stationary nonnegative inflows is examined. A simple condition is found for the existence of a stationary storage and outflow for a general possibly non-monotone release function. It is also shown that in the Markov case (i.e. independent inflows) these distributions are unique under certain conditions. It is demonstrated that under these conditions the stationary behaviour in the Markov case varies continuously with parametric changes in the release rule. This result is used to prove convergence of a finite state space approximation for the Markov storage system.

A discrete-time storage process with a general release rule

1989 ◽  
Vol 26 (3) ◽  
pp. 566-583 ◽  
Author(s):  
John E. Glynn
Keyword(s):  
State Space ◽  
Discrete Time ◽  
Storage System ◽  
Simple Condition ◽  
Storage Process ◽  
Finite State ◽  
Stationary Behaviour ◽  
Finite State Space

A discrete-time storage system with a general release rule and stationary nonnegative inflows is examined. A simple condition is found for the existence of a stationary storage and outflow for a general possibly non-monotone release function. It is also shown that in the Markov case (i.e. independent inflows) these distributions are unique under certain conditions. It is demonstrated that under these conditions the stationary behaviour in the Markov case varies continuously with parametric changes in the release rule. This result is used to prove convergence of a finite state space approximation for the Markov storage system.

scholarly journals Sensitivity and convergence of uniformly ergodic Markov chains

2005 ◽  
Vol 42 (4) ◽  
pp. 1003-1014 ◽  
Author(s):  
A. Yu. Mitrophanov
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Rate Of Convergence ◽  
Transition Kernel ◽  
Numerical Examples ◽  
Perturbation Bounds ◽  
Finite State ◽  
New Perturbation ◽  
Finite State Space

For uniformly ergodic Markov chains, we obtain new perturbation bounds which relate the sensitivity of the chain under perturbation to its rate of convergence to stationarity. In particular, we derive sensitivity bounds in terms of the ergodicity coefficient of the iterated transition kernel, which improve upon the bounds obtained by other authors. We discuss convergence bounds that hold in the case of finite state space, and consider numerical examples to compare the accuracy of different perturbation bounds.

scholarly journals Non-homogeneous Markov chains with a finite state space and a Doeblin type theorem

1995 ◽  
Vol 18 (2) ◽  
pp. 365-370
Author(s):  
Rita Chattopadhyay
Keyword(s):  
Asymptotic Behavior ◽  
Markov Chains ◽  
State Space ◽  
Type Theorem ◽  
Nonnegative Matrices ◽  
Nonhomogeneous Markov Chains ◽  
Finite State ◽  
Finite State Space

Doeblin [1] considered some classes of finite state nonhomogeneous Markov chains and studied their asymptotic behavior. Later Cohn [2] considered another class of such Markov chains (not covered earlier) and obtained Doeblin type results. Though this paper does not present the “best possible” results, the method of proof will be of interest to the reader. It is elementary and based on Hajnal's results on products of nonnegative matrices.

scholarly journals Sensitivity and convergence of uniformly ergodic Markov chains

2005 ◽  
Vol 42 (04) ◽  
pp. 1003-1014 ◽  
Author(s):  
A. Yu. Mitrophanov
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Rate Of Convergence ◽  
Transition Kernel ◽  
Numerical Examples ◽  
Perturbation Bounds ◽  
Finite State ◽  
New Perturbation ◽  
Finite State Space

For uniformly ergodic Markov chains, we obtain new perturbation bounds which relate the sensitivity of the chain under perturbation to its rate of convergence to stationarity. In particular, we derive sensitivity bounds in terms of the ergodicity coefficient of the iterated transition kernel, which improve upon the bounds obtained by other authors. We discuss convergence bounds that hold in the case of finite state space, and consider numerical examples to compare the accuracy of different perturbation bounds.

scholarly journals Why is Kemeny’s constant a constant?

2018 ◽  
Vol 55 (4) ◽  
pp. 1025-1036 ◽  
Author(s):  
Dario Bini ◽  
Jeffrey J. Hunter ◽  
Guy Latouche ◽  
Beatrice Meini ◽  
Peter Taylor
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Death Process ◽  
Birth And Death Process ◽  
Finite State ◽  
Finite Markov Chains ◽  
Infinite State ◽  
Special Case ◽  
Finite State Space

Abstract In their 1960 book on finite Markov chains, Kemeny and Snell established that a certain sum is invariant. The value of this sum has become known as Kemeny’s constant. Various proofs have been given over time, some more technical than others. We give here a very simple physical justification, which extends without a hitch to continuous-time Markov chains on a finite state space. For Markov chains with denumerably infinite state space, the constant may be infinite and even if it is finite, there is no guarantee that the physical argument will hold. We show that the physical interpretation does go through for the special case of a birth-and-death process with a finite value of Kemeny’s constant.

Extreme values, range and weak convergence of integrals of Markov chains

1982 ◽  
Vol 19 (2) ◽  
pp. 272-288 ◽  
Author(s):  
P. J. Brockwell ◽  
S. I. Resnick ◽  
N. Pacheco-Santiago
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
State Space ◽  
Extreme Values ◽  
Finite State ◽  
Integral Process ◽  
Maximum Minimum ◽  
Finite State Space

A study is made of the maximum, minimum and range on [0, t] of the integral process where S is a finite state-space Markov chain. Approximate results are derived by establishing weak convergence of a sequence of such processes to a Wiener process. For a particular family of two-state stationary Markov chains we show that the corresponding centered integral processes exhibit the Hurst phenomenon to a remarkable degree in their pre-asymptotic behaviour.

Sign in / Sign up

Export Citation Format

Share Document