Discrete Time Markov Chains

Abstract In this note we study the probability and the mean time for absorption for discrete time Markov chains. In particular, we are interested in estimating the mean time for absorption when absorption is not certain and connect it with some other known results. Computing a suitable probability generating function, we are able to estimate the mean time for absorption when absorption is not certain giving some applications concerning the random walk. Furthermore, we investigate the probability for a Markov chain to reach a set A before reach B generalizing this result for a sequence of sets A 1 , A 2 , … , A k {A_{1},A_{2},\dots,A_{k}} .

Download Full-text

On the use of Cauchy integral formula for the embedding problem of discrete-time Markov chains

Communication in Statistics- Theory and Methods ◽

10.1080/03610926.2021.1921806 ◽

2021 ◽

pp. 1-15

Author(s):

Virtue U. Ekhosuehi

Keyword(s):

Markov Chains ◽

Discrete Time ◽

Integral Formula ◽

Cauchy Integral Formula ◽

Embedding Problem ◽

Cauchy Integral

Download Full-text

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

Journal of Applied Probability ◽

10.2307/3212311 ◽

1967 ◽

Vol 4 (1) ◽

pp. 192-196 ◽

Cited By ~ 126

Author(s):

J. N. Darroch ◽

E. Seneta

Keyword(s):

Markov Chains ◽

Discrete Time ◽

Continuous Time ◽

Present Note ◽

Time Parameter ◽

Stationary Distributions ◽

Finite State ◽

Absorbing Markov Chains ◽

Finite Markov Chains ◽

Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

Download Full-text

Discrete-Time Markov Chains [Book Review]

IEEE Control Systems Magazine ◽

10.1109/mcs.2005.1573616 ◽

2005 ◽

Vol 25 (6) ◽

pp. 106-108

Keyword(s):

Markov Chains ◽

Discrete Time

Download Full-text

Ruin probability with certain stationary stable claims generated by conservative flows

Advances in Applied Probability ◽

10.1017/s0001867800001804 ◽

2007 ◽

Vol 39 (02) ◽

pp. 360-384 ◽

Cited By ~ 1

Author(s):

Uğur Tuncay Alparslan ◽

Gennady Samorodnitsky

Keyword(s):

Markov Chains ◽

Discrete Time ◽

Continuous Time ◽

Ruin Probability ◽

Stable Process ◽

Independent Interest ◽

Stable Processes ◽

Auxiliary Result ◽

Initial Capital ◽

Order Of Magnitude

We study the ruin probability where the claim sizes are modeled by a stationary ergodic symmetric α-stable process. We exploit the flow representation of such processes, and we consider the processes generated by conservative flows. We focus on two classes of conservative α-stable processes (one discrete-time and one continuous-time), and give results for the order of magnitude of the ruin probability as the initial capital goes to infinity. We also prove a solidarity property for null-recurrent Markov chains as an auxiliary result, which might be of independent interest.

Download Full-text