Formalization of Finite-State Discrete-Time Markov Chains in HOL

2011 ◽  
pp. 90-104 ◽  
Author(s):  
Liya Liu ◽  
Osman Hasan ◽  
Sofiène Tahar
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Finite State

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

Approximations of general discrete time queues by discrete time queues with arrivals modulated by finite chains

1997 ◽  
Vol 29 (04) ◽  
pp. 1039-1059
Author(s):  
Vinod Sharma
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Point Processes ◽  
Marked Point ◽  
Marked Point Processes ◽  
Marked Point Process ◽  
Discrete Time Queues ◽  
Time Queues ◽  
Finite State

Recently, Asmussen and Koole (Journal of Applied Probability 30, pp. 365–372) showed that any discrete or continuous time marked point process can be approximated by a sequence of arrival streams modulated by finite state continuous time Markov chains. If the original process is customer (time) stationary then so are the approximating processes. Also, the moments in the stationary case converge. For discrete marked point processes we construct a sequence of discrete processes modulated by discrete time finite state Markov chains. All the above features of approximating sequences of Asmussen and Koole continue to hold. For discrete arrival sequences (to a queue) which are modulated by a countable state Markov chain we form a different sequence of approximating arrival streams by which, unlike in the Asmussen and Koole case, even the stationary moments of waiting times can be approximated. Explicit constructions for the output process of a queue and the total input process of a discrete time Jackson network with these characteristics are obtained.

scholarly journals Perturbed Markov chains

2003 ◽  
Vol 40 (1) ◽  
pp. 107-122 ◽  
Author(s):  
Eilon Solan ◽  
Nicolas Vieille
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Matrix Analysis ◽  
Transition Matrices ◽  
Analysis Techniques ◽  
Graph Theoretic ◽  
The Matrix ◽  
Finite State ◽  
Finite State Space

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.

Approximations of general discrete time queues by discrete time queues with arrivals modulated by finite chains

1997 ◽  
Vol 29 (4) ◽  
pp. 1039-1059
Author(s):  
Vinod Sharma
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Point Processes ◽  
Marked Point ◽  
Marked Point Processes ◽  
Marked Point Process ◽  
Discrete Time Queues ◽  
Time Queues ◽  
Finite State

Recently, Asmussen and Koole (Journal of Applied Probability30, pp. 365–372) showed that any discrete or continuous time marked point process can be approximated by a sequence of arrival streams modulated by finite state continuous time Markov chains. If the original process is customer (time) stationary then so are the approximating processes. Also, the moments in the stationary case converge. For discrete marked point processes we construct a sequence of discrete processes modulated by discrete time finite state Markov chains. All the above features of approximating sequences of Asmussen and Koole continue to hold. For discrete arrival sequences (to a queue) which are modulated by a countable state Markov chain we form a different sequence of approximating arrival streams by which, unlike in the Asmussen and Koole case, even the stationary moments of waiting times can be approximated. Explicit constructions for the output process of a queue and the total input process of a discrete time Jackson network with these characteristics are obtained.

Ergodic theorems for sequences of infinite stochastic matrices

1967 ◽  
Vol 63 (3) ◽  
pp. 777-784 ◽  
Author(s):  
A. Paz ◽  
M. Reichaw
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Ergodic Theorems ◽  
Stochastic Matrices ◽  
Interested Reader ◽  
Stability Properties ◽  
Long Period ◽  
Finite Case ◽  
Long Run ◽  
Finite State

In the theory of finite state, discrete time, non-homogeneous Markov chains, different notions of ergodicity have been introduced in the literature. These notions are concerned with the long-run behaviour of chains and with their tendency to get some stability properties after a sufficiently long period of time. The aim of this paper is the study of non-homogeneous Markov chains with a denumerable number of states. It will be shown that some theorems which are valid in the finite case are also valid for chains with a denumerable number of states as well. Moreover, a new notion of stability is introduced and it is shown to be satisfied for some chains. Although the paper is self-contained some familiarity with the theory of finite state non-homogeneous Markov chains is desired. Without any attempt of completeness we list for the interested reader the papers: (1–3, 8, 9).

Sign in / Sign up

Export Citation Format

Share Document