scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (01) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (1) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

CLOSED-FORM STOCHASTIC BOUNDS ON THE STATIONARY DISTRIBUTION OF MARKOV CHAINS

2002 ◽  
Vol 16 (4) ◽  
pp. 403-426 ◽  
Author(s):  
Mouad Ben Mamoun ◽  
Nihal Pekergin
Keyword(s):  
Markov Chains ◽  
Closed Form ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Matrix Structure ◽  
Stochastic Monotonicity ◽  
Monotonicity Properties ◽  
Transition Probability Matrices ◽  
Probability Matrices

We propose a particular class of transition probability matrices for discrete-time Markov chains with a closed form to compute the stationary distribution. The stochastic monotonicity properties of this class are established. We give algorithms to construct monotone, bounding matrices belonging to the proposed class for the variability orders. The accuracy of bounds with respect to the underlying matrix structure is discussed through an example.

Reversibility of Markov chains with applications to storage models

1985 ◽  
Vol 22 (01) ◽  
pp. 123-137 ◽  
Author(s):  
Hideo Ōsawa
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Risk Model ◽  
Sufficient Conditions ◽  
Single Server ◽  
General State ◽  
Insurance Risk ◽  
General State Space ◽  
Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

scholarly journals Stochastic Stability for Time-Delay Markovian Jump Systems with Sector-Bounded Nonlinearities and More General Transition Probabilities

2013 ◽  
Vol 2013 ◽  
pp. 1-9
Author(s):  
Dan Ye ◽  
Quan-Yong Fan ◽  
Xin-Gang Zhao ◽  
Guang-Hong Yang
Keyword(s):  
Time Delay ◽  
Stochastic Stability ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Markovian Jump Systems ◽  
Markovian Jump ◽  
Delay Dependent ◽  
Jump Systems

This paper is concerned with delay-dependent stochastic stability for time-delay Markovian jump systems (MJSs) with sector-bounded nonlinearities and more general transition probabilities. Different from the previous results where the transition probability matrix is completely known, a more general transition probability matrix is considered which includes completely known elements, boundary known elements, and completely unknown ones. In order to get less conservative criterion, the state and transition probability information is used as much as possible to construct the Lyapunov-Krasovskii functional and deal with stability analysis. The delay-dependent sufficient conditions are derived in terms of linear matrix inequalities to guarantee the stability of systems. Finally, numerical examples are exploited to demonstrate the effectiveness of the proposed method.

Reversibility of Markov chains with applications to storage models

1985 ◽  
Vol 22 (1) ◽  
pp. 123-137 ◽  
Author(s):  
Hideo Ōsawa
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Risk Model ◽  
Sufficient Conditions ◽  
Single Server ◽  
General State ◽  
Insurance Risk ◽  
General State Space ◽  
Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

Conditions for strong ergodicity using intensity matrices

1988 ◽  
Vol 25 (1) ◽  
pp. 34-42 ◽  
Author(s):  
Jean Johnson ◽  
Dean Isaacson
Keyword(s):  
Markov Chains ◽  
Rate Of Convergence ◽  
Discrete Time ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Limiting Behavior ◽  
Strong Ergodicity ◽  
Continuous Time Markov Chains

Sufficient conditions for strong ergodicity of discrete-time non-homogeneous Markov chains have been given in several papers. Conditions have been given using the left eigenvectors ψn of Pn(ψ nPn = ψ n) and also using the limiting behavior of Pn. In this paper we consider the analogous results in the case of continuous-time Markov chains where one uses the intensity matrices Q(t) instead of P(s, t). A bound on the rate of convergence of certain strongly ergodic chains is also given.

Newton-Shamanskii Method for a Quadratic Matrix Equation Arising in Quasi-Birth-Death Problems

2014 ◽  
Vol 4 (4) ◽  
pp. 386-395
Author(s):  
Pei-Chang Guo
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Matrix Equation ◽  
Nonnegative Solution ◽  
Minimal Nonnegative Solution ◽  
The Minimal Nonnegative Solution ◽  
Quadratic Matrix Equation ◽  
Quadratic Matrix ◽  
Birth Death

AbstractIn order to determine the stationary distribution for discrete time quasi-birth-death Markov chains, it is necessary to find the minimal nonnegative solution of a quadratic matrix equation. The Newton-Shamanskii method is applied to solve this equation, and the sequence of matrices produced is monotonically increasing and converges to its minimal nonnegative solution. Numerical results illustrate the effectiveness of this procedure.

scholarly journals On the Embedding Problem for Discrete-Time Markov Chains

2013 ◽  
Vol 50 (04) ◽  
pp. 918-930 ◽  
Author(s):  
Marie-Anne Guerry
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Embedding Problem ◽  
Time Intervals ◽  
Square Roots ◽  
Probability Matrices

When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. This characterization is referred to in the literature as the embedding problem for discrete time Markov chains. The probability matrix which has probability root(s) is called embeddable. In this paper for two-state Markov chains, necessary and sufficient conditions for embeddability are formulated and the probability square roots of the transition matrix are presented in analytic form. In finding conditions for the existence of probability square roots for (k x k) transition matrices, properties of row-normalized matrices are examined. Besides the existence of probability square roots, the uniqueness of these solutions is discussed: In the case of nonuniqueness, a procedure is introduced to identify a transition matrix that takes into account the specificity of the concrete context. In the case of nonexistence of a probability root, the concept of an approximate probability root is introduced as a solution of an optimization problem related to approximate nonnegative matrix factorization.

Sign in / Sign up

Export Citation Format

Share Document