scholarly journals Computing Hitting Probabilities of Markov Chains: Structural Results with regard to the Solution Space of the Corresponding System of Equations

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Hendrik Baumann ◽  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Solution Space ◽  
Transition Probability Matrix ◽  
Discrete State ◽  
Side Condition ◽  
Hitting Probabilities ◽  
All Solutions ◽  
The Difference ◽  
State Difference

In a previous paper, we have shown that forward use of the steady-state difference equations arising from homogeneous discrete-state space Markov chains may be subject to inherent numerical instability. More precisely, we have proven that, under some appropriate assumptions on the transition probability matrix P, the solution space S of the difference equation may be partitioned into two subspaces S=S1⊕S2, where the stationary measure of P is an element of S1, and all solutions in S1 are asymptotically dominated by the solutions corresponding to S2. In this paper, we discuss the analogous problem of computing hitting probabilities of Markov chains, which is affected by the same numerical phenomenon. In addition, we have to fulfill a somewhat complicated side condition which essentially differs from those conditions one is usually confronted with when solving initial and boundary value problems. To extract the desired solution, an efficient and numerically stable generalized-continued-fraction-based algorithm is developed.

scholarly journals Computable bounds of an 𝓁²-spectral gap for discrete Markov chains with band transition matrices

2016 ◽  
Vol 53 (3) ◽  
pp. 946-952
Author(s):  
Loï Hervé ◽  
James Ledoux
Keyword(s):  
Convergence Rate ◽  
Markov Chains ◽  
Spectral Radius ◽  
Spectral Gap ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Geometric Ergodicity ◽  
Transition Matrices ◽  
Essential Spectral Radius ◽  
Band Transition

AbstractWe analyse the 𝓁²(𝜋)-convergence rate of irreducible and aperiodic Markov chains with N-band transition probability matrix P and with invariant distribution 𝜋. This analysis is heavily based on two steps. First, the study of the essential spectral radius ress(P|𝓁²(𝜋)) of P|𝓁²(𝜋) derived from Hennion’s quasi-compactness criteria. Second, the connection between the spectral gap property (SG2) of P on 𝓁²(𝜋) and the V-geometric ergodicity of P. Specifically, the (SG2) is shown to hold under the condition α0≔∑m=−NNlim supi→+∞(P(i,i+m)P*(i+m,i)1∕2<1. Moreover, ress(P|𝓁²(𝜋)≤α0. Effective bounds on the convergence rate can be provided from a truncation procedure.

Cheeger inequalities for absorbing Markov chains

2016 ◽  
Vol 48 (3) ◽  
pp. 631-647
Author(s):  
Gary Froyland ◽  
Robyn M. Stuart
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Transition Matrices ◽  
Stochastic Transition ◽  
Absorbing Markov Chains ◽  
Second Eigenvalue

Abstract We construct Cheeger-type bounds for the second eigenvalue of a substochastic transition probability matrix in terms of the Markov chain's conductance and metastability (and vice versa) with respect to its quasistationary distribution, extending classical results for stochastic transition matrices.

The Reliability Analysis of Mechanical Leg (2UPS+UP) Based on a Discrete Time Repairable Markov Model

2015 ◽  
Vol 713-715 ◽  
pp. 760-763
Author(s):  
Jia Lei Zhang ◽  
Zhen Lin Jin ◽  
Dong Mei Zhao
Keyword(s):  
Steady State ◽  
Discrete Time ◽  
Transition Probability ◽  
Continuous Model ◽  
Transition Probability Matrix ◽  
State Equations ◽  
One Step ◽  
The Difference ◽  
Step Transition ◽  
The One

We have analyzed some reliability problems of the 2UPS+UP mechanism using continuous Markov repairable model in our previous work. According to the check and repair of the robot is periodic, the discrete time Markov repairable model should be more appropriate. Firstly we built up the discrete time repairable model and got the one step transition probability matrix. Secondly solved the steady state equations and got the steady state availability of the mechanical leg, by the solution of the difference equations the reliability and the mean time to first failure were obtained. In the end we compared the reliability indexes with the continuous model.

scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (01) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

The square root law in the embedding detection problem for Markov chains with unknown matrix of transition probabilities

2019 ◽  
Vol 29 (1) ◽  
pp. 59-68
Author(s):  
Artem V. Volgin
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Classical Model ◽  
Transition Probability Matrix ◽  
Square Root ◽  
Detection Problem ◽  
Asymptotic Growth

Abstract We consider the classical model of embeddings in a simple binary Markov chain with unknown transition probability matrix. We obtain conditions on the asymptotic growth of lengths of the original and embedded sequences sufficient for the consistency of the proposed statistical embedding detection test.

scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (1) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

Computer Simulation Forecast of the Markov Chain for Payback Period of Construction Projects Investment

2013 ◽  
Vol 671-674 ◽  
pp. 3096-3099
Author(s):  
Meng Fang Zhang ◽  
Liang Huang ◽  
Lu Yang Shan
Keyword(s):  
Computer Simulation ◽  
Markov Chains ◽  
Cash Flow ◽  
Construction Projects ◽  
Transition Probability ◽  
Economic Effect ◽  
Cash Flows ◽  
Scientific Basis ◽  
Transition Probability Matrix ◽  
Payback Period

The investment payback period of construction projects is an important index that evaluate and measure economic effect of project investment. It is difficult that the investment payback period of construction projects is calculated generally using analytic method.We established the mathematical model of the payback period, the annual net cash flows are Markov chains. According to the similar projects, collected net yearly cash flow and each quarter net cash flow, A one-step transition probability matrix was described by using the computer simulation of Markov chains, forecasted the dynamic and static payback period of construction projects investment. so as to provide the scientific basis data for decision makers.

scholarly journals OPTIMIZING THE MUTUAL ARRANGEMENT OF PILOT INDICATORS ON AN AIRCRAFT DASHBOARD AND ANALYSIS OF THIS PROCEDURE FROM THE VIEWPOINT OF QUANTUM REPRESENTATIONS

2021 ◽  
pp. 1-10
Author(s):  
L.S. Kuravsky ◽  
I.I. Greshnikov
Keyword(s):  
Transition Probability ◽  
Practical Experience ◽  
Transition Probability Matrix ◽  
Mutual Arrangement ◽  
Similarity Matrix ◽  
Design Decisions ◽  
The Gaze ◽  
Iterative Correction ◽  
Expert Assessments ◽  
The Difference

The purpose of this work is to present the first attempt to provide quantitative analysis and objective justification for designers’ decisions that relate to the arrangement of pilot indicators on an aircraft dashboard with the use of video oculography measurements. To date, such decisions have been made only based on the practical experience accumulated by designers and subjective expert assessments. A new method for optimizing the mutual arrangement of the dashboard indicators is under consideration. This is based on iterative correction of the gaze transition probability matrix between the selected zones of attention, to minimize the difference between the stationary distribution of relative frequencies of gaze that are staying in these zones and the corresponding desirable target eye movements that are given for distribution for qualified pilots. When solving the subsequent multidimensional scaling problem, the gaze transition probability matrix that is obtained is considered to be the similarity matrix, the elements of which quantitatively characterize the proximity between the zones of attention. The main findings of this novel work are as follows: the use of oculography data to justify dashboard design decisions, the optimizing method itself, and its mathematical components, as well as analysis of the optimization in question from the viewpoint of quantum representations, all revealed design mistakes. The results that were obtained can be applied for prototyping variants of aircraft dashboards by rearranging the display areas associated with the corresponding zones of attention.

scholarly journals A geometric invariant in weak lumpability of finite Markov chains

1997 ◽  
Vol 34 (4) ◽  
pp. 847-858 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Spectral Properties ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Homogeneous Markov Chain ◽  
Geometric Invariant ◽  
Initial State ◽  
Direct Sum ◽  
Finite Markov Chains

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

Some Results on an Optimal Maintenance Policy for a Markovian Deteriorating System Subject to Random Shocks

2017 ◽  
Vol 24 (02) ◽  
pp. 1750007 ◽  
Author(s):  
Nobuyuki Tamura
Keyword(s):  
Markov Chain ◽  
Transition Probability ◽  
Maintenance Policy ◽  
Discrete State ◽  
Discounted Cost ◽  
Special Cases ◽  
Optimal Maintenance ◽  
The Difference ◽  
Random Shocks ◽  
Optimal Maintenance Policy

This paper considers a system whose deterioration is modeled as a discrete-time and discrete-state Markov chain and is subject to randomly occurring shocks. The system is more likely to deteriorate after the occurrence of shocks. At each time epoch, we can select from one of three possible actions: operate, repair, or replace. The difference between repair and replace is whether the influence of shocks is removed or not. While repair does not change the number of shocks that the system has suffered, replacement can return it to zero. We derive an expected discounted cost for an unbounded horizon and show that a generalized control limit policy holds under certain assumptions on the costs and the transition probability. Several structural properties of the optimal maintenance policy are also investigated under different assumptions for the occurrence of shocks. These results are useful for numerically determining the optimal maintenance policy. Finally, we consider some special cases by imposing constraints on the costs and the probabilities.

Sign in / Sign up

Export Citation Format

Share Document