A single sample path-based performance sensitivity formula for Markov chains

1996 ◽  
Vol 41 (12) ◽  
pp. 1814-1817 ◽  
Author(s):  
Xi-Ren Cao ◽  
Xue-Ming Yuan ◽  
Li Qiu
1996 ◽  
Vol 29 (1) ◽  
pp. 4795-4800
Author(s):  
Xi-Ren Cao ◽  
Xue-Ming Yuan ◽  
Li Qiu

1998 ◽  
Vol 30 (3) ◽  
pp. 676-692 ◽  
Author(s):  
Xi-Ren Cao

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.


2019 ◽  
Vol 29 (4) ◽  
pp. 2439-2480
Author(s):  
Daniel Hsu ◽  
Aryeh Kontorovich ◽  
David A. Levin ◽  
Yuval Peres ◽  
Csaba Szepesvári ◽  
...  

1998 ◽  
Vol 30 (03) ◽  
pp. 676-692 ◽  
Author(s):  
Xi-Ren Cao

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.


2008 ◽  
Vol 3 (1) ◽  
pp. 21-26 ◽  
Author(s):  
Hai-Tao Fang ◽  
Han-Fu Chen ◽  
Xi-Ren Cao

1995 ◽  
Vol 27 (03) ◽  
pp. 741-769
Author(s):  
Xi-Ren Cao

We study a fundamental feature of the generalized semi-Markov processes (GSMPs), called event coupling. The event coupling reflects the logical behavior of a GSMP that specifies which events can be affected by any given event. Based on the event-coupling property, GSMPs can be classified into three classes: the strongly coupled, the hierarchically coupled, and the decomposable GSMPs. The event-coupling property on a sample path of a GSMP can be represented by the event-coupling trees. With the event-coupling tree, we can quantify the effect of a single perturbation on a performance measure by using realization factors. A set of equations that specifies the realization factors is derived. We show that the sensitivity of steady-state performance with respect to a parameter of an event lifetime distribution can be obtained by a simple formula based on realization factors and that the sample-path performance sensitivity converges to the sensitivity of the steady-state performance with probability one as the length of the sample path goes to infinity. This generalizes the existing results of perturbation analysis of queueing networks to GSMPs.


Sign in / Sign up

Export Citation Format

Share Document