Error bounds for augmented truncation approximations of Markov chains via the perturbation method

LetPbe the transition matrix of a positive recurrent Markov chain on the integers with invariant probability vectorπT, and let(n)P̃ be a stochastic matrix, formed by augmenting the entries of the (n+ 1) x (n+ 1) northwest corner truncation ofParbitrarily, with invariant probability vector(n)πT. We derive computableV-norm bounds on the error betweenπTand(n)πTin terms of the perturbation method from three different aspects: the Poisson equation, the residual matrix, and the norm ergodicity coefficient, which we prove to be effective by showing that they converge to 0 asntends to ∞ under suitable conditions. We illustrate our results through several examples. Comparing our error bounds with the ones of Tweedie (1998), we see that our bounds are more applicable and accurate. Moreover, we also consider possible extensions of our results to continuous-time Markov chains.

NULL RECURRENT UNIT ROOT PROCESSES

The classical nonstationary autoregressive models are both linear and Markov. They include unit root and cointegration models. A possible nonlinear extension is to relax the linearity and at the same time keep general properties such as nonstationarity and the Markov property. A null recurrent Markov chain is nonstationary, and β-null recurrence is of vital importance for statistical inference in nonstationary Markov models, such as, e.g., in nonparametric estimation in nonlinear cointegration within the Markov models. The standard random walk is an example of a null recurrent Markov chain.In this paper we suggest that the concept of null recurrence is an appropriate nonlinear generalization of the linear unit root concept and as such it may be a starting point for a nonlinear cointegration concept within the Markov framework. In fact, we establish the link between null recurrent processes and autoregressive unit root models. It turns out that null recurrence is closely related to the location of the roots of the characteristic polynomial of the state space matrix and the associated eigenvectors. Roughly speaking the process is β-null recurrent if one root is on the unit circle, null recurrent if two distinct roots are on the unit circle, whereas the others are inside the unit circle. It is transient if there are more than two roots on the unit circle. These results are closely connected to the random walk being null recurrent in one and two dimensions but transient in three dimensions. We also give an example of a process that by appropriate adjustments can be made β-null recurrent for any β ∈ (0, 1) and can also be made null recurrent without being β-null recurrent.

Pair formation in a Markovian arrival process with two event labels

The stochastic process resulting when pairs of events are formed from two point processes is a rich source of questions. When the two point processes have different rates, the resulting stochastic process has a mean drift towards either -∞ or +∞. However, when the two processes have equal rates, we end up with a null-recurrent Markov chain and this has interesting behavior. We study this process for both discrete and continuous times and consider special cases with applications in communications networks. One interesting result for applications is the waiting time of a packet waiting for a token, a special case of this pair-formation process. Pair formation by two independent Poisson processes of equal rates results in a point process that is asymptotically a Poisson process of the same rate.

Pair formation in a Markovian arrival process with two event labels

The stochastic process resulting when pairs of events are formed from two point processes is a rich source of questions. When the two point processes have different rates, the resulting stochastic process has a mean drift towards either -∞ or +∞. However, when the two processes have equal rates, we end up with a null-recurrent Markov chain and this has interesting behavior. We study this process for both discrete and continuous times and consider special cases with applications in communications networks. One interesting result for applications is the waiting time of a packet waiting for a token, a special case of this pair-formation process. Pair formation by two independent Poisson processes of equal rates results in a point process that is asymptotically a Poisson process of the same rate.

One-dimensional branching random walks in a Markovian random environment

We study a one-dimensional supercritical branching random walk in a non-i.i.d. random environment, which considers both the branching mechanism and the step transition. This random environment is constructed using a recurrent Markov chain on a finite or countable state space. Criteria of (strong) recurrence and transience are presented for this model.

One-dimensional branching random walks in a Markovian random environment

We study a one-dimensional supercritical branching random walk in a non-i.i.d. random environment, which considers both the branching mechanism and the step transition. This random environment is constructed using a recurrent Markov chain on a finite or countable state space. Criteria of (strong) recurrence and transience are presented for this model.

Asymptotic behaviour of population-size-dependent branching processes in Markovian random environments

A population-size-dependent branching process {Zn} is considered where the population's evolution is controlled by a Markovian environment process {ξn}. For this model, let mk,θ and be the mean and the variance respectively of the offspring distribution when the population size is k and a environment θ is given. Let B = {ω : Zn(ω) = 0 for some n} and q = P(B). The asymptotic behaviour of limnZn and is studied in the case where supθ|mk,θ − mθ| → 0 for some real numbers {mθ} such that infθmθ > 1. When the environmental sequence {ξn} is a irreducible positive recurrent Markov chain (particularly, when its state space is finite), certain extinction (q = 1) and non-certain extinction (q < 1) are studied.