Representation of denumerable Markov chains with multiple states by weighted circuits

The constructive solution to the problem of representing a strictly stationary Markov chain ζ with a countable infinity of r-sequences (i 1, i 2, · ··, ir ), r > 1, as states by a class of directed weighted circuits is given. Associating the chain ζ with its dual chain η having reversed states and the same transition law, a connection with physical laws that govern diffusion of electrical current through a directed planar network with r-series-connected nodes is shown.

Download Full-text

Asymptotic behaviour of sample weighted circuits representing recurrent Markov chains

Journal of Applied Probability ◽

10.1017/s0021900200039103 ◽

1990 ◽

Vol 27 (03) ◽

pp. 545-556 ◽

Cited By ~ 2

Author(s):

S. Kalpazidou

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Asymptotic Behaviour ◽

Probabilistic Interpretation ◽

Stationary Markov Chain

The asymptotic behaviour of the sequence (𝒞 n (ω), wc,n (ω)/n), is studied where 𝒞 n (ω) is the class of all cycles c occurring along the trajectory ωof a recurrent strictly stationary Markov chain (ξ n ) until time n and wc,n (ω) is the number of occurrences of the cycle c until time n. The previous sequence of sample weighted classes converges almost surely to a class of directed weighted cycles (𝒞∞, ω c ) which represents uniquely the chain (ξ n ) as a circuit chain, and ω c is given a probabilistic interpretation.

Download Full-text

Exponential ergodicity in derived Markov chains

Journal of Applied Probability ◽

10.1017/s0021900200114482 ◽

1968 ◽

Vol 5 (03) ◽

pp. 669-678 ◽

Cited By ~ 1

Author(s):

Jozef L. Teugels

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

Exponential Ergodicity ◽

Stationary Markov Chain ◽

Continuous Time Markov Chains ◽

General Proposition

A general proposition is proved stating that the exponential ergodicity of a stationary Markov chain is preserved for derived Markov chains as defined by Cohen [2], [3]. An application to a certain type of continuous time Markov chains is included.

Download Full-text

Markov Chains in Many Dimensions

Advances in Applied Probability ◽

10.2307/1427819 ◽

1994 ◽

Vol 26 (3) ◽

pp. 756-774 ◽

Cited By ~ 9

Author(s):

Dimitris N. Politis

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Maximum Entropy ◽

Stationary Distribution ◽

Random Fields ◽

Special Class ◽

Markov Random Fields ◽

One Dimension ◽

Stationary Markov Chain ◽

Markov Random

A generalization of the notion of a stationary Markov chain in more than one dimension is proposed, and is found to be a special class of homogeneous Markov random fields. Stationary Markov chains in many dimensions are shown to possess a maximum entropy property, analogous to the corresponding property for Markov chains in one dimension. In addition, a representation of Markov chains in many dimensions is provided, together with a method for their generation that converges to their stationary distribution.

Download Full-text

Poisson limit law for Markov chains

Ergodic Theory and Dynamical Systems ◽

10.1017/s0143385700006301 ◽

1991 ◽

Vol 11 (3) ◽

pp. 501-513 ◽

Cited By ~ 61

Author(s):

B. Pitskel

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Stationary Markov Chain ◽

Poisson Limit

AbstractFor a mixing stationary Markov chain we prove a Poisson limit law for the recurrence to small cylindrical sets. Since hyperbolic torus automorphisms are Markov chains, the result carries over to these transformations.

Download Full-text

Exponential ergodicity in derived Markov chains

Journal of Applied Probability ◽

10.2307/3211929 ◽

1968 ◽

Vol 5 (3) ◽

pp. 669-678 ◽

Cited By ~ 4

Author(s):

Jozef L. Teugels

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Continuous Time ◽

Exponential Ergodicity ◽

Stationary Markov Chain ◽

Continuous Time Markov Chains ◽

General Proposition

A general proposition is proved stating that the exponential ergodicity of a stationary Markov chain is preserved for derived Markov chains as defined by Cohen [2], [3]. An application to a certain type of continuous time Markov chains is included.

Download Full-text

Asymptotic behaviour of sample weighted circuits representing recurrent Markov chains

Journal of Applied Probability ◽

10.2307/3214540 ◽

1990 ◽

Vol 27 (3) ◽

pp. 545-556 ◽

Cited By ~ 12

Author(s):

S. Kalpazidou

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Asymptotic Behaviour ◽

Probabilistic Interpretation ◽

Stationary Markov Chain

The asymptotic behaviour of the sequence (𝒞n(ω), wc,n(ω)/n), is studied where 𝒞n(ω) is the class of all cycles c occurring along the trajectory ωof a recurrent strictly stationary Markov chain (ξ n) until time n and wc,n(ω) is the number of occurrences of the cycle c until time n. The previous sequence of sample weighted classes converges almost surely to a class of directed weighted cycles (𝒞∞, ωc) which represents uniquely the chain (ξ n) as a circuit chain, and ω c is given a probabilistic interpretation.

Download Full-text

Markov Chains in Many Dimensions

Advances in Applied Probability ◽

10.1017/s0001867800026537 ◽

1994 ◽

Vol 26 (03) ◽

pp. 756-774 ◽

Cited By ~ 1

Author(s):

Dimitris N. Politis

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Maximum Entropy ◽

Stationary Distribution ◽

Random Fields ◽

Special Class ◽

Markov Random Fields ◽

One Dimension ◽

Stationary Markov Chain ◽

Markov Random

A generalization of the notion of a stationary Markov chain in more than one dimension is proposed, and is found to be a special class of homogeneous Markov random fields. Stationary Markov chains in many dimensions are shown to possess a maximum entropy property, analogous to the corresponding property for Markov chains in one dimension. In addition, a representation of Markov chains in many dimensions is provided, together with a method for their generation that converges to their stationary distribution.

Download Full-text

Poisson Approximation for the Non-Overlapping Appearances of Several Words in Markov Chains

Combinatorics Probability Computing ◽

10.1017/s096354830100476x ◽

2001 ◽

Vol 10 (4) ◽

pp. 293-308 ◽

Cited By ~ 8

Author(s):

OURANIA CHRYSSAPHINOU ◽

STAVROS PAPASTAVRIDIS ◽

EUTICHIA VAGGELATOU

Keyword(s):

Markov Chain ◽

Markov Chains ◽

State Space ◽

Total Variation ◽

Analogous Result ◽

Poisson Approximation ◽

Total Variation Distance ◽

Numerical Example ◽

Stationary Markov Chain ◽

Variation Distance

Let X1, …, Xn be a sequence of r.v.s produced by a stationary Markov chain with state space an alphabet Ω = {ω1, …, ωq}, q [ges ] 2. We consider a set of words {A1, …, Ar}, r [ges ] 2, with letters from the alphabet Ω. We allow the words to have self-overlaps as well as overlaps between them. Let [Escr ] denote the event of the appearance of a word from the set {A1, …, Ar} at a given position. Moreover, define by N the number of non-overlapping (competing renewal) appearances of [Escr ] in the sequence X1, …, Xn. We derive a bound on the total variation distance between the distribution of N and a Poisson distribution with parameter [ ]N. The Stein–Chen method and combinatorial arguments concerning the structure of words are employed. As a corollary, we obtain an analogous result for the i.i.d. case. Furthermore, we prove that, under quite general conditions, the r.v. N converges in distribution to a Poisson r.v. A numerical example is presented to illustrate the performance of the bound in the Markov case.

Download Full-text

Application of Markov processes for analysis and control of aircraft maintainability

Civil Aviation High TECHNOLOGIES ◽

10.26467/2079-0619-2020-23-1-71-83 ◽

2020 ◽

Vol 23 (1) ◽

pp. 71-83

Author(s):

Yu. M. Chinyuchin ◽

A. S. Solov'ev

Keyword(s):

Markov Chain ◽

Markov Chains ◽

Markov Processes ◽

Operating Time ◽

Operation Time ◽

Spare Parts ◽

Time Interval ◽

Maintenance And Repair ◽

Stationary Markov Chain ◽

And Control

The process of aircraft operation involves constant effects of various factors on its components leading to accidental or systematic changes in their technical condition. Markov processes are a particular case of stochastic processes, which take place during aeronautical equipment operation. The relationship of the reliability characteristics with the cost recovery of the objects allows us to apply the analytic apparatus of Markov processes for the analysis and optimization of maintainability factors. The article describes two methods of the analysis and control of object maintainability based on stationary and non-stationary Markov chains. The model of a stationary Markov chain is used for the equipment with constant in time intensity of the events. For the objects with time-varying events intensity, a non-stationary Markov chain is used. In order to reduce the number of the mathematical operations for the analysis of aeronautical engineering maintainability by using non-stationary Markov processes an algorithm for their optimization is presented. The suggested methods of the analysis by means of Markov chains allow to execute comparative assessments of expected maintenance and repair costs for one or several one-type objects taking into account their original conditions and operation time. The process of maintainability control using Markov chains includes search of the optimal strategy of maintenance and repair considering each state of an object under which maintenance costs will be minimal. The given approbation of the analysis methods and maintainability control using Markov processes for an object under control allowed to build a predictive-controlled model in which the expected costs for its maintenance and repair are calculated as well as the required number of spare parts for each specified operating time interval. The possibility of using the mathematical apparatus of Markov processes for a large number of objects with different reliability factors distribution is shown. The software implementation of the described methods as well as the usage of tabular adapted software will contribute to reducing the complexity of the calculations and improving data visualization.

Download Full-text