Estimation of transition probabilities in a multiparticle semi-Markov system

1976 ◽  
Vol 13 (4) ◽  
pp. 696-706 ◽  
Author(s):  
David Burman
Keyword(s):  
Markov Process ◽  
Time Distribution ◽  
Limit Theorems ◽  
Transition Probabilities ◽  
Strong Limit ◽  
Stieltjes Transform ◽  
Markov System ◽  
Sample Paths ◽  
Finite State ◽  
Semi Markov Process

Particles enter a finite-state system and move according to independent sample paths from a semi-Markov process. Strong limit theorems are developed for the ratio of the flow of particles from states i to j and the flow out of When the cumulative arrival of particles into the system up to time t, A (t) ∼ λtα, then a.s. When A (t)∼ λekt, then the flow between states must be normalized by the Laplace–Stieltjes transform of the conditional holding time distribution, in order to make the ratio an unbiased estimator of ρij.

Estimation of transition probabilities in a multiparticle semi-Markov system

1976 ◽  
Vol 13 (04) ◽  
pp. 696-706
Author(s):  
David Burman
Keyword(s):  
Markov Process ◽  
Time Distribution ◽  
Limit Theorems ◽  
Transition Probabilities ◽  
Strong Limit ◽  
Stieltjes Transform ◽  
Markov System ◽  
Sample Paths ◽  
Finite State ◽  
Semi Markov Process

Particles enter a finite-state system and move according to independent sample paths from a semi-Markov process. Strong limit theorems are developed for the ratio of the flow of particles from states i to j and the flow out of When the cumulative arrival of particles into the system up to time t, A (t) ∼ λt α, then a.s. When A (t)∼ λekt, then the flow between states must be normalized by the Laplace–Stieltjes transform of the conditional holding time distribution, in order to make the ratio an unbiased estimator of ρij.

scholarly journals A study on association in time of a finite semi markov process

10.26524/cm67 ◽  
2020 ◽  
Vol 4 (1) ◽  
Author(s):  
Syed Tahir Hussainy ◽  
Mohamed Ali A ◽  
Ravi kumar S
Keyword(s):  
Markov Process ◽  
Transition Probabilities ◽  
Sufficient Conditions ◽  
Hazard Rates ◽  
Finite State ◽  
Semi Markov Process

We derive three equivalent sufficient conditions for association in time of a finite state semi Markov process in terms of transition probabilities and crude hazard rates. This result generalizes the earlier results of Esary and Proschan (1970)for a binary Markov process and Hjort, Natvig and Funnemark (1985) for a multistate Markov process.

General solidarity theorems for semi-Markov processes

1972 ◽  
Vol 9 (04) ◽  
pp. 789-802
Author(s):  
Choong K. Cheong ◽  
Jozef L. Teugels
Keyword(s):  
Markov Process ◽  
Markov Processes ◽  
Transition Probabilities ◽  
Large Set ◽  
Semi Markov Process ◽  
Regularly Varying ◽  
Specific Pair ◽  
Semi Markov Processes

Let {Zt, t ≧ 0} be an irreducible regular semi-Markov process with transition probabilities Pij (t). Let f(t) be non-negative and non-decreasing to infinity, and let λ ≧ 0. This paper identifies a large set of functions f(t) with the solidarity property that convergence of the integral ≧ eλtf(t)Pij (t) dt for a specific pair of states i and j implies convergence of the integral for all pairs of states. Similar results are derived for the Markov renewal functions Mij (t). Among others it is shown that f(t) can be taken regularly varying.

scholarly journals Developing A Semi-Markov Process Model for Bridge Deterioration Prediction in Shanghai

2019 ◽  
Vol 11 (19) ◽  
pp. 5524 ◽  
Author(s):  
Yu Fang ◽  
Lijun Sun
Keyword(s):  
Markov Process ◽  
Service Life ◽  
Life Expectancy ◽  
Weibull Distribution ◽  
Process Model ◽  
Transition Probabilities ◽  
Bridge Management ◽  
Markov Process Model ◽  
The Future ◽  
Semi Markov Process

The performance of urban bridges will deteriorate gradually throughout service life. Bridge deterioration prediction is essential for bridge management, especially for maintenance planning and decision-making. By considering the time-dependent reliability in the bridge deterioration process, a Weibull distribution based semi-Markov process model for urban bridge deterioration prediction was proposed in this paper. Historical inspection records stored in the Bridge Manage System (BMS) database in Shanghai since 2004 were investigated. The Weibull distribution was used to characterize the bridge deterioration behavior within each condition rating (CR), and the semi-Markov process was used to calculate the bridge transition probabilities between adjacent CRs. After that, the service life expectancy of urban bridges, the transition probabilities of the deck system and the substructure, and the future CR proportion change caused by deterioration was predicted. The prediction results indicate that the life expectancy of concrete beam bridges is about 77 years. The decay rate of the deck system is the fastest among three major parts, and the substructure has a much longer life expectancy. It suggests that the overall prediction accuracy of the semi-Markov model in network-level is better than the regression analysis method. Furthermore, the proportion of bridges in intact condition will gradually decrease in the next few decades, while the percentage of bridges in the qualified and bad state will increase rapidly. The prediction results show a good agreement with the actual deterioration trend of the urban bridges in Shanghai. In order to alleviate the pressure of bridge maintenance in the future, it is necessary to adopt a more targeted preventive maintenance strategy.

Reward processes with nonlinear reward functions

1996 ◽  
Vol 33 (4) ◽  
pp. 1011-1017 ◽  
Author(s):  
A. Reza Soltani
Keyword(s):  
Markov Process ◽  
Explicit Formula ◽  
Sojourn Time ◽  
Time Distribution ◽  
Reward Function ◽  
Sojourn Time Distribution ◽  
Semi Markov Process ◽  
Reward Process ◽  
Reward Functions

Based on a semi-Markov process J(t), t ≧ 0, a reward process Z(t), t ≧ 0, is introduced where it is assumed that the reward function, p(k, x) is nonlinear; if the reward function is linear, i.e. ρ (k, x) = kx, the reward process Z(t), t ≧ 0, becomes the classical one, which has been considered by many authors. An explicit formula for E(Z(t)) is given in terms of the moments of the sojourn time distribution at t, when the reward function is a polynomial.

Optimal decision procedures for finite Markov chains. Part II: Communicating systems

1973 ◽  
Vol 5 (3) ◽  
pp. 521-540 ◽  
Author(s):  
John Bather
Keyword(s):  
Markov Process ◽  
General Solution ◽  
Transition Probabilities ◽  
Optimal Decision ◽  
Positive Probability ◽  
Finite State ◽  
Finite Markov Chains ◽  
Special Case ◽  
Family Of Distributions ◽  
Finite State Space

A Markov process in discrete time with a finite state space is controlled by choosing the transition probabilities from a given convex family of distributions depending on the present state. The immediate cost is prescribed for each choice and it is required to minimise the average expected cost over an infinite future. The paper considers a special case of this general problem and provides the foundation for a general solution. The main result is that an optimal policy exists if each state of the system can be reached with positive probability from any other state by choosing a suitable policy.

The departure process from the GI/G/1 Queue

1969 ◽  
Vol 6 (3) ◽  
pp. 704-707 ◽  
Author(s):  
Thomas L. Vlach ◽  
Ralph L. Disney
Keyword(s):  
Markov Chain ◽  
Markov Process ◽  
Transition Probabilities ◽  
Two Dimensional ◽  
Departure Process ◽  
Dimensional State Space ◽  
Semi Markov Process ◽  
One Step ◽  
Step Transition ◽  
The One

The departure process from the GI/G/1 queue is shown to be a semi-Markov process imbedded at departure points with a two-dimensional state space. Transition probabilities for this process are defined and derived from the distributions of the arrival and service processes. The one step transition probabilities and a stationary distribution are obtained for the imbedded two-dimensional Markov chain.

Limit theorems for α-recurrent semi-Markov processes

1976 ◽  
Vol 8 (03) ◽  
pp. 531-547 ◽  
Author(s):  
Esa Nummelin
Keyword(s):  
Markov Processes ◽  
Continuous Time ◽  
Limit Theorems ◽  
Transition Probabilities ◽  
Renewal Processes ◽  
Markov Renewal Processes ◽  
Semi Markov Process ◽  
Limit Behaviour ◽  
Semi Markov Processes ◽  
Special Case

In this paper the limit behaviour of α-recurrent Markov renewal processes and semi-Markov processes is studied by using the recent results on the concept of α-recurrence for Markov renewal processes. Section 1 contains the preliminary results, which are needed later in the paper. In Section 2 we consider the limit behaviour of the transition probabilities Pij (t) of an α-recurrent semi-Markov process. Section 4 deals with quasi-stationarity. Our results extend the results of Cheong (1968), (1970) and of Flaspohler and Holmes (1972) to the case in which the functions to be considered are directly Riemann integrable. We also try to correct the errors we have found in these papers. As a special case from our results we consider continuous-time Markov processes in Sections 3 and 5.

General solidarity theorems for semi-Markov processes

1972 ◽  
Vol 9 (4) ◽  
pp. 789-802
Author(s):  
Choong K. Cheong ◽  
Jozef L. Teugels
Keyword(s):  
Markov Process ◽  
Markov Processes ◽  
Transition Probabilities ◽  
Large Set ◽  
Semi Markov Process ◽  
Regularly Varying ◽  
Specific Pair ◽  
Semi Markov Processes

Let {Zt, t ≧ 0} be an irreducible regular semi-Markov process with transition probabilities Pij (t). Let f(t) be non-negative and non-decreasing to infinity, and let λ ≧ 0. This paper identifies a large set of functions f(t) with the solidarity property that convergence of the integral ≧ eλtf(t)Pij(t) dt for a specific pair of states i and j implies convergence of the integral for all pairs of states. Similar results are derived for the Markov renewal functions Mij (t). Among others it is shown that f(t) can be taken regularly varying.

Limit theorems for α-recurrent semi-Markov processes

1976 ◽  
Vol 8 (3) ◽  
pp. 531-547 ◽  
Author(s):  
Esa Nummelin
Keyword(s):  
Markov Processes ◽  
Continuous Time ◽  
Limit Theorems ◽  
Transition Probabilities ◽  
Renewal Processes ◽  
Markov Renewal Processes ◽  
Semi Markov Process ◽  
Limit Behaviour ◽  
Semi Markov Processes ◽  
Special Case

In this paper the limit behaviour of α-recurrent Markov renewal processes and semi-Markov processes is studied by using the recent results on the concept of α-recurrence for Markov renewal processes. Section 1 contains the preliminary results, which are needed later in the paper. In Section 2 we consider the limit behaviour of the transition probabilities Pij(t) of an α-recurrent semi-Markov process. Section 4 deals with quasi-stationarity. Our results extend the results of Cheong (1968), (1970) and of Flaspohler and Holmes (1972) to the case in which the functions to be considered are directly Riemann integrable. We also try to correct the errors we have found in these papers. As a special case from our results we consider continuous-time Markov processes in Sections 3 and 5.

Sign in / Sign up

Export Citation Format

Share Document