scholarly journals Finite-State-Space Truncations for Infinite Quasi-Birth-Death Processes

2020 ◽  
Vol 2020 ◽  
pp. 1-23
Author(s):  
Hendrik Baumann
Keyword(s):  
Markov Chain ◽  
Lower Bound ◽  
State Space ◽  
Upper Bound ◽  
Speed Of Convergence ◽  
Drift Condition ◽  
Finite State ◽  
Infinite State ◽  
Finite State Space ◽  
Birth Death

For dealing numerically with the infinite-state-space Markov chains, a truncation of the state space is inevitable, that is, an approximation by a finite-state-space Markov chain has to be performed. In this paper, we consider level-dependent quasi-birth-death processes, and we focus on the computation of stationary expectations. In previous literature, efficient methods for computing approximations to these characteristics have been suggested and established. These methods rely on truncating the process at some level N, and for N⟶∞, convergence of the approximation to the desired characteristic is guaranteed. This paper’s main goal is to quantify the speed of convergence. Under the assumption of an f-modulated drift condition, we derive terms for a lower bound and an upper bound on stationary expectations which converge quickly to the same value and which can be efficiently computed.

scholarly journals A Time-Inhomogeneous Prendiville Model with Failures and Repairs

Mathematics ◽  
2022 ◽  
Vol 10 (2) ◽  
pp. 251
Author(s):  
Virginia Giorno ◽  
Amelia G. Nobile
Keyword(s):  
Markov Chain ◽  
State Space ◽  
The State ◽  
Random Time ◽  
Death Process ◽  
Inhomogeneous Markov Chain ◽  
Finite State ◽  
Intensity Functions ◽  
Finite State Space ◽  
Birth Death

We consider a time-inhomogeneous Markov chain with a finite state-space which models a system in which failures and repairs can occur at random time instants. The system starts from any state j (operating, F, R). Due to a failure, a transition from an operating state to F occurs after which a repair is required, so that a transition leads to the state R. Subsequently, there is a restore phase, after which the system restarts from one of the operating states. In particular, we assume that the intensity functions of failures, repairs and restores are proportional and that the birth-death process that models the system is a time-inhomogeneous Prendiville process.

scholarly journals Conditioning an additive functional of a Markov chain to stay nonnegative. I. Survival for a long time

2005 ◽  
Vol 37 (4) ◽  
pp. 1015-1034 ◽  
Author(s):  
Saul D. Jacka ◽  
Zorana Lazic ◽  
Jon Warren
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
State Space ◽  
Continuous Time ◽  
Additive Functional ◽  
Finite State ◽  
Long Time ◽  
Negative Drift ◽  
Irreducible Markov Chain ◽  
Finite State Space

Let (Xt)t≥0 be a continuous-time irreducible Markov chain on a finite state space E, let v be a map v: E→ℝ\{0}, and let (φt)t≥0 be an additive functional defined by φt=∫0tv(Xs)d s. We consider the case in which the process (φt)t≥0 is oscillating and that in which (φt)t≥0 has a negative drift. In each of these cases, we condition the process (Xt,φt)t≥0 on the event that (φt)t≥0 is nonnegative until time T and prove weak convergence of the conditioned process as T→∞.

Extreme values, range and weak convergence of integrals of Markov chains

1982 ◽  
Vol 19 (02) ◽  
pp. 272-288 ◽  
Author(s):  
P. J. Brockwell ◽  
S. I. Resnick ◽  
N. Pacheco-Santiago
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
State Space ◽  
Extreme Values ◽  
Finite State ◽  
Integral Process ◽  
Maximum Minimum ◽  
Finite State Space

A study is made of the maximum, minimum and range on [0,t] of the integral processwhereSis a finite state-space Markov chain. Approximate results are derived by establishing weak convergence of a sequence of such processes to a Wiener process. For a particular family of two-state stationary Markov chains we show that the corresponding centered integral processes exhibit the Hurst phenomenon to a remarkable degree in their pre-asymptotic behaviour.

scholarly journals Antiduality and Möbius monotonicity: generalized coupon collector problem

2019 ◽  
Vol 23 ◽  
pp. 739-769
Author(s):  
Paweł Lorek
Keyword(s):  
Markov Chain ◽  
State Space ◽  
Partial Ordering ◽  
Absorption Time ◽  
Absorbing Markov Chain ◽  
Coupon Collector ◽  
Strong Stationary Duality ◽  
Finite State ◽  
A Chain ◽  
Finite State Space

For a given absorbing Markov chain X* on a finite state space, a chain X is a sharp antidual of X* if the fastest strong stationary time (FSST) of X is equal, in distribution, to the absorption time of X*. In this paper, we show a systematic way of finding such an antidual based on some partial ordering of the state space. We use a theory of strong stationary duality developed recently for Möbius monotone Markov chains. We give several sharp antidual chains for Markov chain corresponding to a generalized coupon collector problem. As a consequence – utilizing known results on the limiting distribution of the absorption time – we indicate separation cutoffs (with their window sizes) in several chains. We also present a chain which (under some conditions) has a prescribed stationary distribution and its FSST is distributed as a prescribed mixture of sums of geometric random variables.

Doubly stochastic Poisson processes and process control

1972 ◽  
Vol 4 (02) ◽  
pp. 318-338 ◽  
Author(s):  
Mats Rudemo
Keyword(s):  
Markov Chain ◽  
Differential Equations ◽  
State Space ◽  
Point Process ◽  
Doubly Stochastic ◽  
Long Run ◽  
Finite State ◽  
Control State ◽  
Special Case ◽  
Finite State Space

Consider a Poisson point process with an intensity parameter forming a Markov chain with continuous time and finite state space. A system of ordinary differential equations is derived for the conditional distribution of the Markov chain given observations of the point process. An estimate of the current intensity, optimal in the least-squares sense, is computed from this distribution. Applications to reliability and replacement theory are given. A special case with two states, corresponding to a process in control and out of control, is discussed at length. Adjustment rules, based on the conditional probability of the out of control state, are studied. Regarded as a function of time, this probability forms a Markov process with the unit interval as state space. For the distribution of this process, integro-differential equations are derived. They are used to compute the average long run cost of adjustment rules.

scholarly journals Why is Kemeny’s constant a constant?

2018 ◽  
Vol 55 (4) ◽  
pp. 1025-1036 ◽  
Author(s):  
Dario Bini ◽  
Jeffrey J. Hunter ◽  
Guy Latouche ◽  
Beatrice Meini ◽  
Peter Taylor
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Death Process ◽  
Birth And Death Process ◽  
Finite State ◽  
Finite Markov Chains ◽  
Infinite State ◽  
Special Case ◽  
Finite State Space

Abstract In their 1960 book on finite Markov chains, Kemeny and Snell established that a certain sum is invariant. The value of this sum has become known as Kemeny’s constant. Various proofs have been given over time, some more technical than others. We give here a very simple physical justification, which extends without a hitch to continuous-time Markov chains on a finite state space. For Markov chains with denumerably infinite state space, the constant may be infinite and even if it is finite, there is no guarantee that the physical argument will hold. We show that the physical interpretation does go through for the special case of a birth-and-death process with a finite value of Kemeny’s constant.

Extreme values, range and weak convergence of integrals of Markov chains

1982 ◽  
Vol 19 (2) ◽  
pp. 272-288 ◽  
Author(s):  
P. J. Brockwell ◽  
S. I. Resnick ◽  
N. Pacheco-Santiago
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
State Space ◽  
Extreme Values ◽  
Finite State ◽  
Integral Process ◽  
Maximum Minimum ◽  
Finite State Space

A study is made of the maximum, minimum and range on [0, t] of the integral process where S is a finite state-space Markov chain. Approximate results are derived by establishing weak convergence of a sequence of such processes to a Wiener process. For a particular family of two-state stationary Markov chains we show that the corresponding centered integral processes exhibit the Hurst phenomenon to a remarkable degree in their pre-asymptotic behaviour.

scholarly journals Simple SIR models with Markovian control

2021 ◽  
Author(s):  
Krzysztof Bartoszek ◽  
Wojciech Bartoszek ◽  
Michał Krzemiński
Keyword(s):  
Dynamical System ◽  
Markov Chain ◽  
State Space ◽  
High Frequency ◽  
Random Dynamical System ◽  
Special Focus ◽  
Mathematical Apparatus ◽  
Frequency Limit ◽  
Finite State ◽  
Finite State Space

AbstractWe consider a random dynamical system, where the deterministic dynamics are driven by a finite-state space Markov chain. We provide a comprehensive introduction to the required mathematical apparatus and then turn to a special focus on the susceptible-infected-recovered epidemiological model with random steering. Through simulations we visualize the behaviour of the system and the effect of the high-frequency limit of the driving Markov chain. We formulate some questions and conjectures of a purely theoretical nature.

Un principe d'invariance pour une classe de marches p-corrélées sur ℤd

1998 ◽  
Vol 35 (3) ◽  
pp. 557-565
Author(s):  
Alexis Bienvenüe
Keyword(s):  
Markov Chain ◽  
Random Walks ◽  
State Space ◽  
Explicit Expression ◽  
Covariance Matrix ◽  
Reinforced Random Walks ◽  
Finite State ◽  
Invariance Theorem ◽  
Finite State Space

Let ζ be a Markov chain on a finite state space D, f a function from D to ℝd, and Sn = ∑k=1nf(ζk). We prove an invariance theorem for S and derive an explicit expression of the limit covariance matrix. We give its exact value for p-reinforced random walks on ℤ2 with p = 1, 2, 3.

scholarly journals Conditioning an additive functional of a Markov chain to stay nonnegative. I. Survival for a long time

2005 ◽  
Vol 37 (04) ◽  
pp. 1015-1034 ◽  
Author(s):  
Saul D. Jacka ◽  
Zorana Lazic ◽  
Jon Warren
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
State Space ◽  
Continuous Time ◽  
Additive Functional ◽  
Finite State ◽  
Long Time ◽  
Negative Drift ◽  
Irreducible Markov Chain ◽  
Finite State Space

Let (X t ) t≥0 be a continuous-time irreducible Markov chain on a finite state space E, let v be a map v: E→ℝ\{0}, and let (φ t ) t≥0 be an additive functional defined by φ t =∫0 t v(X s )d s. We consider the case in which the process (φ t ) t≥0 is oscillating and that in which (φ t ) t≥0 has a negative drift. In each of these cases, we condition the process (X t ,φ t ) t≥0 on the event that (φ t ) t≥0 is nonnegative until time T and prove weak convergence of the conditioned process as T→∞.

Sign in / Sign up

Export Citation Format

Share Document