On the Distribution and covariance structure of the present value of a random income stream

1982 ◽  
Vol 19 (01) ◽  
pp. 240-244
Author(s):  
J. Keilson
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Covariance Structure ◽  
Present Value ◽  
Income Stream ◽  
Finite Markov Chains

The present value is studied when I(t) is a stationary random income stream. The stationary distribution of V(t) for a family of simple streams modeled by stationary finite Markov chains is given explicitly. The process V(t) is shown to be observable in a special sense when I(t) is time-reversible.

On the Distribution and covariance structure of the present value of a random income stream

1982 ◽  
Vol 19 (1) ◽  
pp. 240-244 ◽  
Author(s):  
J. Keilson
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Covariance Structure ◽  
Present Value ◽  
Income Stream ◽  
Finite Markov Chains

The present value is studied when I(t) is a stationary random income stream. The stationary distribution of V(t) for a family of simple streams modeled by stationary finite Markov chains is given explicitly. The process V(t) is shown to be observable in a special sense when I(t) is time-reversible.

Perturbation theory and finite Markov chains

1968 ◽  
Vol 5 (2) ◽  
pp. 401-413 ◽  
Author(s):  
Paul J. Schweitzer
Keyword(s):  
Perturbation Theory ◽  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Fundamental Matrix ◽  
Transition Probabilities ◽  
Semi Group ◽  
Partial Derivatives ◽  
Finite Markov Chains ◽  
Derivatives Of

A perturbation formalism is presented which shows how the stationary distribution and fundamental matrix of a Markov chain containing a single irreducible set of states change as the transition probabilities vary. Expressions are given for the partial derivatives of the stationary distribution and fundamental matrix with respect to the transition probabilities. Semi-group properties of the generators of transformations from one Markov chain to another are investigated. It is shown that a perturbation formalism exists in the multiple subchain case if and only if the change in the transition probabilities does not alter the number of, or intermix the various subchains. The formalism is presented when this condition is satisfied.

Some theorems on the transient covariance of Markov chains

1972 ◽  
Vol 9 (01) ◽  
pp. 214-218 ◽  
Author(s):  
John F. Reynolds
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Covariance Structure ◽  
Continuous Parameter ◽  
Transient Period

Several authors have considered the covariance structure of continuous parameter Markov chains. Most of this work has dealt with particular process ses, notably Morse (1955) who analysed the simple M/M/1 queue and Bene-(1961) who considered a telephone trunking model. Furthermore, the results obtained apply only when the process has attained its limiting (stationary) distribution. A recent paper by Reynolds (1968) gave some general results for finite chains, still assuming stationarity. This note generalises the results obtained therein, and considers the covariance structure during the transient period prior to attaining the stationary distribution where this exists. In the case where no such distribution exists, the results are valid throughout the whole lifetime of the process.

scholarly journals Normal distributions of finite Markov chains

2019 ◽  
Vol 29 (08) ◽  
pp. 1431-1449
Author(s):  
John Rhodes ◽  
Anne Schilling
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Planar Graphs ◽  
Finite Semigroup ◽  
Finite Markov Chain ◽  
Normal Distributions ◽  
Straight Line ◽  
Finite Markov Chains ◽  
The Right

We show that the stationary distribution of a finite Markov chain can be expressed as the sum of certain normal distributions. These normal distributions are associated to planar graphs consisting of a straight line with attached loops. The loops touch only at one vertex either of the straight line or of another attached loop. Our analysis is based on our previous work, which derives the stationary distribution of a finite Markov chain using semaphore codes on the Karnofsky–Rhodes and McCammond expansion of the right Cayley graph of the finite semigroup underlying the Markov chain.

Some theorems on the transient covariance of Markov chains

1972 ◽  
Vol 9 (1) ◽  
pp. 214-218 ◽  
Author(s):  
John F. Reynolds
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Covariance Structure ◽  
Continuous Parameter ◽  
Transient Period

Several authors have considered the covariance structure of continuous parameter Markov chains. Most of this work has dealt with particular process ses, notably Morse (1955) who analysed the simple M/M/1 queue and Bene-(1961) who considered a telephone trunking model. Furthermore, the results obtained apply only when the process has attained its limiting (stationary) distribution. A recent paper by Reynolds (1968) gave some general results for finite chains, still assuming stationarity. This note generalises the results obtained therein, and considers the covariance structure during the transient period prior to attaining the stationary distribution where this exists. In the case where no such distribution exists, the results are valid throughout the whole lifetime of the process.

Perturbation theory and finite Markov chains

1968 ◽  
Vol 5 (02) ◽  
pp. 401-413 ◽  
Author(s):  
Paul J. Schweitzer
Keyword(s):  
Perturbation Theory ◽  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Fundamental Matrix ◽  
Transition Probabilities ◽  
Semi Group ◽  
Partial Derivatives ◽  
Finite Markov Chains ◽  
Derivatives Of

A perturbation formalism is presented which shows how the stationary distribution and fundamental matrix of a Markov chain containing a single irreducible set of states change as the transition probabilities vary. Expressions are given for the partial derivatives of the stationary distribution and fundamental matrix with respect to the transition probabilities. Semi-group properties of the generators of transformations from one Markov chain to another are investigated. It is shown that a perturbation formalism exists in the multiple subchain case if and only if the change in the transition probabilities does not alter the number of, or intermix the various subchains. The formalism is presented when this condition is satisfied.

On Quasi-Stationary distributions in absorbing discrete-time finite Markov chains

1965 ◽  
Vol 2 (1) ◽  
pp. 88-100 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Stationary Distributions ◽  
Transient States ◽  
Reverse Process ◽  
Finite Markov Chains ◽  
Quasi Stationary Distribution

The time to absorption from the set T of transient states of a Markov chain may be sufficiently long for the probability distribution over T to settle down in some sense to a “quasi-stationary” distribution. Various analogues of the stationary distribution of an irreducible chain are suggested and compared. The reverse process of an absorbing chain is found to be relevant.

On Quasi-Stationary distributions in absorbing discrete-time finite Markov chains

1965 ◽  
Vol 2 (01) ◽  
pp. 88-100 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Stationary Distributions ◽  
Transient States ◽  
Reverse Process ◽  
Finite Markov Chains ◽  
Quasi Stationary Distribution

The time to absorption from the set T of transient states of a Markov chain may be sufficiently long for the probability distribution over T to settle down in some sense to a “quasi-stationary” distribution. Various analogues of the stationary distribution of an irreducible chain are suggested and compared. The reverse process of an absorbing chain is found to be relevant.

Characterization of the conditional stationary distribution in Markov chains via systems of linear inequalities

2020 ◽  
Vol 52 (4) ◽  
pp. 1249-1283
Author(s):  
Masatoshi Kimura ◽  
Tetsuya Takine
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Continuous Time ◽  
State Transition ◽  
Convex Polytope ◽  
Linear Inequalities ◽  
Systems Of Linear Inequalities ◽  
Free Sets ◽  
Practical Implications

AbstractThis paper considers ergodic, continuous-time Markov chains $\{X(t)\}_{t \in (\!-\infty,\infty)}$ on $\mathbb{Z}^+=\{0,1,\ldots\}$ . For an arbitrarily fixed $N \in \mathbb{Z}^+$ , we study the conditional stationary distribution $\boldsymbol{\pi}(N)$ given the Markov chain being in $\{0,1,\ldots,N\}$ . We first characterize $\boldsymbol{\pi}(N)$ via systems of linear inequalities and identify simplices that contain $\boldsymbol{\pi}(N)$ , by examining the $(N+1) \times (N+1)$ northwest corner block of the infinitesimal generator $\textbf{\textit{Q}}$ and the subset of the first $N+1$ states whose members are directly reachable from at least one state in $\{N+1,N+2,\ldots\}$ . These results are closely related to the augmented truncation approximation (ATA), and we provide some practical implications for the ATA. Next we consider an extension of the above results, using the $(K+1) \times (K+1)$ ( $K > N$ ) northwest corner block of $\textbf{\textit{Q}}$ and the subset of the first $K+1$ states whose members are directly reachable from at least one state in $\{K+1,K+2,\ldots\}$ . Furthermore, we introduce new state transition structures called (K, N)-skip-free sets, using which we obtain the minimum convex polytope that contains $\boldsymbol{\pi}(N)$ .

Sign in / Sign up

Export Citation Format

Share Document