Entropy of killed-resurrected stationary Markov chains

2021 ◽  
Vol 58 (1) ◽  
pp. 177-196
Author(s):  
Servet Martínez
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Probability Measure ◽  
Markov Chains ◽  
Stochastic Matrix ◽  
Hitting Times ◽  
Stationary Distributions ◽  
Natural Factors ◽  
Absorbing States ◽  
Stationary Representation

AbstractWe consider a strictly substochastic matrix or a stochastic matrix with absorbing states. By using quasi-stationary distributions we show that there is an associated canonical Markov chain that is built from the resurrected chain, the absorbing states, and the hitting times, together with a random walk on the absorbing states, which is necessary for achieving time stationarity. Based upon the 2-stringing representation of the resurrected chain, we supply a stationary representation of the killed and the absorbed chains. The entropies of these representations have a clear meaning when one identifies the probability measure of natural factors. The balance between the entropies of these representations and the entropy of the canonical chain serves to check the correctness of the whole construction.

A note on the inverse problem for reducible Markov chains

1977 ◽  
Vol 14 (03) ◽  
pp. 621-625
Author(s):  
A. O. Pittenger
Keyword(s):  
Inverse Problem ◽  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Physical Process ◽  
Raw Materials ◽  
Transition Probability ◽  
Linear Inequalities ◽  
System Of Linear Inequalities ◽  
Absorbing States

Suppose a physical process is modelled by a Markov chain with transition probability on S 1 ∪ S 2, S 1 denoting the transient states and S 2 a set of absorbing states. If v denotes the output distribution on S 2, the question arises as to what input distributions (of raw materials) on S 1 produce v. In this note we give an alternative to the formulation of Ray and Margo [2] and reduce the problem to one system of linear inequalities. An application to random walk is given and the equiprobability case examined in detail.

A note on the inverse problem for reducible Markov chains

1977 ◽  
Vol 14 (3) ◽  
pp. 621-625 ◽  
Author(s):  
A. O. Pittenger
Keyword(s):  
Inverse Problem ◽  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Physical Process ◽  
Raw Materials ◽  
Transition Probability ◽  
Linear Inequalities ◽  
System Of Linear Inequalities ◽  
Absorbing States

Suppose a physical process is modelled by a Markov chain with transition probability on S1 ∪ S2, S1 denoting the transient states and S2 a set of absorbing states.If v denotes the output distribution on S2, the question arises as to what input distributions (of raw materials) on S1 produce v. In this note we give an alternative to the formulation of Ray and Margo [2] and reduce the problem to one system of linear inequalities. An application to random walk is given and the equiprobability case examined in detail.

On the absorption probabilities and mean time for absorption for discrete Markov chains

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Nikolaos Halidias
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Generating Function ◽  
Discrete Time ◽  
Probability Generating Function ◽  
The Mean ◽  
Mean Time ◽  
Absorption Probabilities

Abstract In this note we study the probability and the mean time for absorption for discrete time Markov chains. In particular, we are interested in estimating the mean time for absorption when absorption is not certain and connect it with some other known results. Computing a suitable probability generating function, we are able to estimate the mean time for absorption when absorption is not certain giving some applications concerning the random walk. Furthermore, we investigate the probability for a Markov chain to reach a set A before reach B generalizing this result for a sequence of sets A 1 , A 2 , … , A k {A_{1},A_{2},\dots,A_{k}} .

On identifiability and order of continuous-time aggregated Markov chains, Markov-modulated Poisson processes, and phase-type distributions

1996 ◽  
Vol 33 (3) ◽  
pp. 640-653 ◽  
Author(s):  
Tobias Rydén
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Poisson Processes ◽  
Hitting Times ◽  
Phase Type ◽  
Phase Type Distributions ◽  
Minimum Number ◽  
Markov Modulated ◽  
Two Parameters

An aggregated Markov chain is a Markov chain for which some states cannot be distinguished from each other by the observer. In this paper we consider the identifiability problem for such processes in continuous time, i.e. the problem of determining whether two parameters induce identical laws for the observable process or not. We also study the order of a continuous-time aggregated Markov chain, which is the minimum number of states needed to represent it. In particular, we give a lower bound on the order. As a by-product, we obtain results of this kind also for Markov-modulated Poisson processes, i.e. doubly stochastic Poisson processes whose intensities are directed by continuous-time Markov chains, and phase-type distributions, which are hitting times in finite-state Markov chains.

On quasi-stationary distributions in discrete-time Markov chains with a denumerable infinity of states

1966 ◽  
Vol 3 (02) ◽  
pp. 403-434 ◽  
Author(s):  
E. Seneta ◽  
D. Vere-Jones
Keyword(s):  
Random Walk ◽  
Markov Chains ◽  
Recent Work ◽  
Discrete Time ◽  
Branching Process ◽  
Stationary Distributions

Distributions appropriate to the description of long-term behaviour within an irreducible class of discrete-time denumerably infinite Markov chains are considered. The first four sections are concerned with general reslts, extending recent work on this subject. In Section 5 these are applied to the branching process, and give refinements of several well-known results. The last section deals with the semi-infinite random walk with an absorbing barrier at the origin.

The existence of moments for stationary Markov chains

1983 ◽  
Vol 20 (01) ◽  
pp. 191-196 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Time Dependent ◽  
General Function ◽  
Convergence Of Moments ◽  
Special Cases

We give conditions under which the stationary distribution π of a Markov chain admits moments of the general form ∫ f(x)π(dx), where f is a general function; specific examples include f(x) = xr and f(x) = esx . In general the time-dependent moments of the chain then converge to the stationary moments. We show that in special cases this convergence of moments occurs at a geometric rate. The results are applied to random walk on [0, ∞).

Quasi-stationary distributions for Markov chains on a general state space

1974 ◽  
Vol 11 (4) ◽  
pp. 726-741 ◽  
Author(s):  
Richard. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Invariant Measure ◽  
Markov Chains ◽  
State Space ◽  
Finiteness Condition ◽  
General State ◽  
Stationary Distributions ◽  
General State Space ◽  
Countably Infinite ◽  
Stationary Behaviour

The quasi-stationary behaviour of a Markov chain which is φ-irreducible when restricted to a subspace of a general state space is investigated. It is shown that previous work on the case where the subspace is finite or countably infinite can be extended to general chains, and the existence of certain quasi-stationary limits as honest distributions is equivalent to the restricted chain being R-positive with the unique R-invariant measure satisfying a certain finiteness condition.

scholarly journals A Cost-Effective Smoothed Multigrid with Modified Neighborhood-Based Aggregation for Markov Chains

2015 ◽  
Vol 2015 ◽  
pp. 1-15 ◽  
Author(s):  
Zhao-Li Shen ◽  
Ting-Zhu Huang ◽  
Bruno Carpentieri ◽  
Chun Wen
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Numerical Experiments ◽  
Multigrid Method ◽  
Cost Effective ◽  
Multigrid Methods ◽  
Stationary Distributions ◽  
Convergence Behavior ◽  
Aggregation Procedure ◽  
Smoothed Aggregation

Smoothed aggregation multigrid method is considered for computing stationary distributions of Markov chains. A judgement which determines whether to implement the whole aggregation procedure is proposed. Through this strategy, a large amount of time in the aggregation procedure is saved without affecting the convergence behavior. Besides this, we explain the shortage and irrationality of the Neighborhood-Based aggregation which is commonly used in multigrid methods. Then a modified version is presented to remedy and improve it. Numerical experiments on some typical Markov chain problems are reported to illustrate the performance of these methods.

scholarly journals Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Geometric Drift Conditions

2015 ◽  
Vol 47 (1) ◽  
pp. 83-105 ◽  
Author(s):  
Hiroyuki Masuyama
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Total Variation ◽  
Discrete Time ◽  
Error Bounds ◽  
Total Variation Distance ◽  
Time Block ◽  
Stationary Distributions ◽  
Variation Distance ◽  
Drift Conditions

In this paper we study the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases, where an original Markov chain itself is not necessarily block monotone but is blockwise dominated by a block-monotone Markov chain. Finally, we discuss the application of our results to GI/G/1-type Markov chains.

Geometric bounds on iterative approximations for nearly completely decomposable Markov chains

1990 ◽  
Vol 27 (03) ◽  
pp. 521-529 ◽  
Author(s):  
Guy Louchard ◽  
Guy Latouche
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Perturbation Analysis ◽  
Stochastic Matrix ◽  
Stationary Probability ◽  
Probability Vector ◽  
Finite Markov Chain ◽  
Stationary Probability Vector

We consider a finite Markov chain with nearly-completely decomposable stochastic matrix. We determine bounds for the error, when the stationary probability vector is approximated via a perturbation analysis.

Sign in / Sign up

Export Citation Format

Share Document