On First-Passage Times and Sojourn Times in Finite QBD Processes and Their Applications in Epidemics

In this paper, we revisit level-dependent quasi-birth-death processes with finitely many possible values of the level and phase variables by complementing the work of Gaver, Jacobs, and Latouche (Adv. Appl. Probab. 1984), where the emphasis is upon obtaining numerical methods for evaluating stationary probabilities and moments of first-passage times to higher and lower levels. We provide a matrix-analytic scheme for numerically computing hitting probabilities, the number of upcrossings, sojourn time analysis, and the random area under the level trajectory. Our algorithmic solution is inspired from Gaussian elimination, which is applicable in all our descriptors since the underlying rate matrices have a block-structured form. Using the results obtained, numerical examples are given in the context of varicella-zoster virus infections.

EXPLICIT SOLUTIONS FOR CONTINUOUS-TIME QBD PROCESSES BY USING RELATIONS BETWEEN MATRIX GEOMETRIC ANALYSIS AND THE PROBABILITY GENERATING FUNCTIONS METHOD

Two main methods are used to solve continuous-time quasi birth-and-death processes: matrix geometric (MG) and probability generating functions (PGFs). MG requires a numerical solution (via successive substitutions) of a matrix quadratic equation A0 + RA1 + R2A2 = 0. PGFs involve a row vector $\vec{G}(z)$ of unknown generating functions satisfying $H(z)\vec{G}{(z)^\textrm{T}} = \vec{b}{(z)^\textrm{T}},$ where the row vector $\vec{b}(z)$ contains unknown “boundary” probabilities calculated as functions of roots of the matrix H(z). We show that: (a) H(z) and $\vec{b}(z)$ can be explicitly expressed in terms of the triple A0, A1, and A2; (b) when each matrix of the triple is lower (or upper) triangular, then (i) R can be explicitly expressed in terms of roots of $\det [H(z)]$ ; and (ii) the stability condition is readily extracted.

On the ergodicity of a class of level-dependent quasi-birth-and-death processes

We examine necessary and sufficient conditions for recurrence and positive recurrence of a class of irreducible, level-dependent quasi-birth-and-death (LDQBD) processes with a block tridiagonal structure that exhibits asymptotic convergence in the rows as the level tends to infinity. These conditions are obtained by exploiting a multi-dimensional Lyapunov drift approach, along with the theory of generalized Markov group inverses. Additionally, we highlight analogies to well-known average drift results for level-independent quasi-birth-and-death (QBD) processes.

Computing Stationary Expectations in Level-Dependent QBD Processes

Stationary expectations corresponding to long-run averages of additive functionals on level-dependent quasi-birth-and-death processes are considered. Special cases include long-run average costs or rewards, moments and cumulants of steady-state queueing network performance measures, and many others. We provide a matrix-analytic scheme for numerically computing such stationary expectations without explicitly computing the stationary distribution of the process, which yields an algorithm that is as quick as its counterparts for stationary distributions but requires far less computer storage. Specific problems arising in the case of infinite state spaces are discussed and the application of the algorithm is demonstrated by a queueing network example.

Computing Stationary Expectations in Level-Dependent QBD Processes

Stationary expectations corresponding to long-run averages of additive functionals on level-dependent quasi-birth-and-death processes are considered. Special cases include long-run average costs or rewards, moments and cumulants of steady-state queueing network performance measures, and many others. We provide a matrix-analytic scheme for numerically computing such stationary expectations without explicitly computing the stationary distribution of the process, which yields an algorithm that is as quick as its counterparts for stationary distributions but requires far less computer storage. Specific problems arising in the case of infinite state spaces are discussed and the application of the algorithm is demonstrated by a queueing network example.

Kronecker-Based Infinite Level-Dependent QBD Processes

Markovian systems with multiple interacting subsystems under the influence of a control unit are considered. The state spaces of the subsystems are countably infinite, whereas that of the control unit is finite. A recent infinite level-dependent quasi-birth-and-death model for such systems is extended by facilitating the automatic representation and generation of the nonzero blocks in its underlying infinitesimal generator matrix with sums of Kronecker products. Experiments are performed on systems of stochastic chemical kinetics having two or more countably infinite state space subsystems. Results indicate that, even though more memory is consumed, there are many cases where a matrix-analytic solution coupled with Lyapunov theory yields a faster and more accurate steady-state measure compared to that obtained with simulation.

Kronecker-Based Infinite Level-Dependent QBD Processes

Markovian systems with multiple interacting subsystems under the influence of a control unit are considered. The state spaces of the subsystems are countably infinite, whereas that of the control unit is finite. A recent infinite level-dependent quasi-birth-and-death model for such systems is extended by facilitating the automatic representation and generation of the nonzero blocks in its underlying infinitesimal generator matrix with sums of Kronecker products. Experiments are performed on systems of stochastic chemical kinetics having two or more countably infinite state space subsystems. Results indicate that, even though more memory is consumed, there are many cases where a matrix-analytic solution coupled with Lyapunov theory yields a faster and more accurate steady-state measure compared to that obtained with simulation.