Stationary Distributions of a Markov Chain

1999 ◽  
pp. 43-71
Author(s):  
Rinaldo B. Schinazi
Keyword(s):  
Markov Chain ◽  
Stationary Distributions

Quasi-stationary distributions for Markov chains on a general state space

1974 ◽  
Vol 11 (4) ◽  
pp. 726-741 ◽  
Author(s):  
Richard. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Invariant Measure ◽  
Markov Chains ◽  
State Space ◽  
Finiteness Condition ◽  
General State ◽  
Stationary Distributions ◽  
General State Space ◽  
Countably Infinite ◽  
Stationary Behaviour

The quasi-stationary behaviour of a Markov chain which is φ-irreducible when restricted to a subspace of a general state space is investigated. It is shown that previous work on the case where the subspace is finite or countably infinite can be extended to general chains, and the existence of certain quasi-stationary limits as honest distributions is equivalent to the restricted chain being R-positive with the unique R-invariant measure satisfying a certain finiteness condition.

scholarly journals A Cost-Effective Smoothed Multigrid with Modified Neighborhood-Based Aggregation for Markov Chains

2015 ◽  
Vol 2015 ◽  
pp. 1-15 ◽  
Author(s):  
Zhao-Li Shen ◽  
Ting-Zhu Huang ◽  
Bruno Carpentieri ◽  
Chun Wen
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Numerical Experiments ◽  
Multigrid Method ◽  
Cost Effective ◽  
Multigrid Methods ◽  
Stationary Distributions ◽  
Convergence Behavior ◽  
Aggregation Procedure ◽  
Smoothed Aggregation

Smoothed aggregation multigrid method is considered for computing stationary distributions of Markov chains. A judgement which determines whether to implement the whole aggregation procedure is proposed. Through this strategy, a large amount of time in the aggregation procedure is saved without affecting the convergence behavior. Besides this, we explain the shortage and irrationality of the Neighborhood-Based aggregation which is commonly used in multigrid methods. Then a modified version is presented to remedy and improve it. Numerical experiments on some typical Markov chain problems are reported to illustrate the performance of these methods.

GEOMETRIC-FORM BOUNDS FOR THE GIX/M/1 QUEUEING SYSTEM

1999 ◽  
Vol 13 (4) ◽  
pp. 509-520 ◽  
Author(s):  
Antonis Economou
Keyword(s):  
Markov Chain ◽  
Continuous Time ◽  
Queueing System ◽  
Embedded Markov Chain ◽  
Geometric Form ◽  
Stationary Distributions ◽  
Discrete Time Markov Chain ◽  
Numerical Studies ◽  
Simple Product

The GI/M/1 queueing system was long ago studied by considering the embedded discrete-time Markov chain at arrival epochs and was proved to have remarkably simple product-form stationary distributions both at arrival epochs and in continuous time. Although this method works well also in several variants of this system, it breaks down when customers arrive in batches. The resulting GIX/M/1 system has no tractable stationary distribution. In this paper we use a recent result of Miyazawa and Taylor (1997) to obtain a stochastic upper bound for the GIX/M/1 system. We also introduce a class of continuous-time Markov chains which are related to the original GIX/M/1 embedded Markov chain that are shown to have modified geometric stationary distributions. We use them to obtain easily computed stochastic lower bounds for the GIX/M/1 system. Numerical studies demonstrate the quality of these bounds.

scholarly journals Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Geometric Drift Conditions

2015 ◽  
Vol 47 (1) ◽  
pp. 83-105 ◽  
Author(s):  
Hiroyuki Masuyama
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Total Variation ◽  
Discrete Time ◽  
Error Bounds ◽  
Total Variation Distance ◽  
Time Block ◽  
Stationary Distributions ◽  
Variation Distance ◽  
Drift Conditions

In this paper we study the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases, where an original Markov chain itself is not necessarily block monotone but is blockwise dominated by a block-monotone Markov chain. Finally, we discuss the application of our results to GI/G/1-type Markov chains.

scholarly journals Transient limits

2009 ◽  
Vol 3 (1) ◽  
pp. 52-63
Author(s):  
Fred Richman ◽  
Katarzyna Winkowska-Nowak
Keyword(s):  
Markov Chain ◽  
Small Parameter ◽  
Time Scales ◽  
Stationary Distributions ◽  
Markov Matrix ◽  
Different Time Scales

Let A be a Markov matrix depending on a small parameter ?, and Cn the average of the first n powers of A. The stationary distributions of A are the rows of S = lim n?+? Cn. The limiting stationary distributions are the rows of lim ??0 S. We investigate transient limits of the sequence Cn. These idempotent Markov matrices come up implicitly in an algorithm to compute limiting stationary distributions. They represent the intermediate-term behavior of the Markov chain at different time scales.

scholarly journals Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Geometric Drift Conditions

2015 ◽  
Vol 47 (01) ◽  
pp. 83-105 ◽  
Author(s):  
Hiroyuki Masuyama
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Total Variation ◽  
Discrete Time ◽  
Error Bounds ◽  
Total Variation Distance ◽  
Time Block ◽  
Stationary Distributions ◽  
Variation Distance ◽  
Drift Conditions

In this paper we study the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases, where an original Markov chain itself is not necessarily block monotone but is blockwise dominated by a block-monotone Markov chain. Finally, we discuss the application of our results to GI/G/1-type Markov chains.

An Efficient Procedure for Computing Quasi-Stationary Distributions of Markov Chains by Sparse Transition Structure

1994 ◽  
Vol 26 (01) ◽  
pp. 68-79
Author(s):  
P. K. Pollett ◽  
D. E. Stewart
Keyword(s):  
Markov Chain ◽  
Continuous Time ◽  
Computational Procedure ◽  
Transition Rates ◽  
Continuous Time Markov Chain ◽  
Transition Structure ◽  
Stationary Distributions ◽  
Efficient Procedure ◽  
The Matrix ◽  
Quasi Stationary Distribution

We describe a computational procedure for evaluating the quasi-stationary distributions of a continuous-time Markov chain. Our method, which is an ‘iterative version' of Arnoldi's algorithm, is appropriate for dealing with cases where the matrix of transition rates is large and sparse, but does not exhibit a banded structure which might otherwise be usefully exploited. We illustrate the method with reference to an epidemic model and we compare the computed quasi-stationary distribution with an appropriate diffusion approximation.

Stationary distributions for piecewise-deterministic Markov processes

1990 ◽  
Vol 27 (1) ◽  
pp. 60-73 ◽  
Author(s):  
O. L. V. Costa
Keyword(s):  
Markov Chain ◽  
Markov Processes ◽  
Existence And Uniqueness ◽  
Sufficient Conditions ◽  
Capacity Expansion ◽  
Stationary Distributions ◽  
Mild Conditions ◽  
Piecewise Deterministic Markov Processes

In this paper we show that the problem of existence and uniqueness of stationary distributions for piecewise-deterministic Markov processes (PDPs) is equivalent to the same problem for the associated Markov chain, so long as some mild conditions on the parameters of the PDP are satisfied. Our main result is the construction of an invertible mapping from the set of stationary distributions for the PDP to the set of stationary distributions for the Markov chain. Some sufficient conditions for existence are presented and an application to capacity expansion is given.

An Efficient Procedure for Computing Quasi-Stationary Distributions of Markov Chains by Sparse Transition Structure

1994 ◽  
Vol 26 (1) ◽  
pp. 68-79 ◽  
Author(s):  
P. K. Pollett ◽  
D. E. Stewart
Keyword(s):  
Markov Chain ◽  
Continuous Time ◽  
Computational Procedure ◽  
Transition Rates ◽  
Continuous Time Markov Chain ◽  
Transition Structure ◽  
Stationary Distributions ◽  
Efficient Procedure ◽  
The Matrix ◽  
Quasi Stationary Distribution

We describe a computational procedure for evaluating the quasi-stationary distributions of a continuous-time Markov chain. Our method, which is an ‘iterative version' of Arnoldi's algorithm, is appropriate for dealing with cases where the matrix of transition rates is large and sparse, but does not exhibit a banded structure which might otherwise be usefully exploited. We illustrate the method with reference to an epidemic model and we compare the computed quasi-stationary distribution with an appropriate diffusion approximation.

scholarly journals Application of generating functions to a problem in finite dam theory

1959 ◽  
Vol 1 (1) ◽  
pp. 116-120 ◽  
Author(s):  
N. U. Prabhu
Keyword(s):  
Markov Chain ◽  
Generating Functions ◽  
Negative Binomial ◽  
Stationary Distributions ◽  
Finite Dam

SummaryWe consider the finite dam model due to Moran, in which the storage {Zt} is known to be a Markov chain. The method of generating functions is used to derive stationary distributions of Zt in the two particular cases where the input is of geometric and negative binomial types.

Sign in / Sign up

Export Citation Format

Share Document