The Maclaurin series for performance functions of Markov chains

1998 ◽  
Vol 30 (03) ◽  
pp. 676-692 ◽  
Author(s):  
Xi-Ren Cao
Keyword(s):  
Markov Chains ◽  
Performance Measures ◽  
Sample Path ◽  
Single Sample ◽  
Maclaurin Series ◽  
Transition Matrices ◽  
Engineering Problems ◽  
Realization Factors ◽  
Derivatives Of ◽  
Steady State Performance

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.

The Maclaurin series for performance functions of Markov chains

1998 ◽  
Vol 30 (3) ◽  
pp. 676-692 ◽  
Author(s):  
Xi-Ren Cao
Keyword(s):  
Markov Chains ◽  
Performance Measures ◽  
Sample Path ◽  
Single Sample ◽  
Maclaurin Series ◽  
Transition Matrices ◽  
Engineering Problems ◽  
Realization Factors ◽  
Derivatives Of ◽  
Steady State Performance

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.

A single sample path-based performance sensitivity formula for Markov chains

1996 ◽  
Vol 41 (12) ◽  
pp. 1814-1817 ◽  
Author(s):  
Xi-Ren Cao ◽  
Xue-Ming Yuan ◽  
Li Qiu
Keyword(s):  
Markov Chains ◽  
Sample Path ◽  
Single Sample ◽  
Performance Sensitivity

Event coupling and performance sensitivity analysis of generalized semi-Markov processes

1995 ◽  
Vol 27 (03) ◽  
pp. 741-769
Author(s):  
Xi-Ren Cao
Keyword(s):  
Steady State ◽  
Markov Processes ◽  
Queueing Networks ◽  
Sample Path ◽  
Performance Measure ◽  
Coupling Property ◽  
Performance Sensitivity ◽  
Realization Factors ◽  
Semi Markov Processes ◽  
Steady State Performance

We study a fundamental feature of the generalized semi-Markov processes (GSMPs), called event coupling. The event coupling reflects the logical behavior of a GSMP that specifies which events can be affected by any given event. Based on the event-coupling property, GSMPs can be classified into three classes: the strongly coupled, the hierarchically coupled, and the decomposable GSMPs. The event-coupling property on a sample path of a GSMP can be represented by the event-coupling trees. With the event-coupling tree, we can quantify the effect of a single perturbation on a performance measure by using realization factors. A set of equations that specifies the realization factors is derived. We show that the sensitivity of steady-state performance with respect to a parameter of an event lifetime distribution can be obtained by a simple formula based on realization factors and that the sample-path performance sensitivity converges to the sensitivity of the steady-state performance with probability one as the length of the sample path goes to infinity. This generalizes the existing results of perturbation analysis of queueing networks to GSMPs.

Convergence of perturbation analysis estimates for discontinuous sample functions: a general approach

1988 ◽  
Vol 20 (1) ◽  
pp. 59-78 ◽  
Author(s):  
Reuven Y. Rubinstein ◽  
Ferenc Szidarovszky
Keyword(s):  
Monte Carlo ◽  
Rate Of Convergence ◽  
Performance Measures ◽  
Dynamic Systems ◽  
Perturbation Analysis ◽  
Sample Path ◽  
Single Sample ◽  
Parameter Vector ◽  
Discrete Events ◽  
Convergence Conditions

Generalized perturbation analysis (PA) estimates to study sensitivity of performance measures of discrete events dynamic systems for discontinuous sample functions are introduced. Their convergence conditions and rate of convergence are given. It is shown that the PA estimates based on a single sample path always converge faster to the unknown sensitivity parameter (vector of parameters) than their counterpart—crude Monte Carlo ones.

Potentially unlimited variance reduction in importance sampling of Markov chains

1996 ◽  
Vol 28 (1) ◽  
pp. 166-188 ◽  
Author(s):  
Sigrún Andradóttir ◽  
Daniel P. Heyman ◽  
Teunis J. Ott
Keyword(s):  
Steady State ◽  
Markov Chains ◽  
Importance Sampling ◽  
Path Length ◽  
Transition Matrix ◽  
Variance Reduction ◽  
Sample Path ◽  
Computer Time ◽  
Performance Measure ◽  
Transition Matrices

We consider the application of importance sampling in steady-state simulations of finite Markov chains. We show that, for a large class of performance measures, there is a choice of the alternative transition matrix for which the ratio of the variance of the importance sampling estimator to the variance of the naive simulation estimator converges to zero as the sample path length goes to infinity. Obtaining this ‘optimal’ transition matrix involves computing the performance measure of interest, so the optimal matrix cannot be computed in precisely those situations where simulation is required to estimate steady-state performance. However, our results show that alternative transition matrices of the form Q = P + E/T, where P is the original transition matrix and T is the sample path length, can be expected to provide good results. Moreover, we provide an iterative algorithm for obtaining alternative transition matrices of this form that converge to the optimal matrix as the number of iterations increases, and present an example that shows that spending some computer time iterating this algorithm and then conducting the simulation with the resulting alternative transition matrix may provide considerable variance reduction when compared to naive simulation.

A Single Sample Path-Based Performance Sensitivity Formula for Markov Chains

1996 ◽  
Vol 29 (1) ◽  
pp. 4795-4800
Author(s):  
Xi-Ren Cao ◽  
Xue-Ming Yuan ◽  
Li Qiu
Keyword(s):  
Markov Chains ◽  
Sample Path ◽  
Single Sample ◽  
Performance Sensitivity

scholarly journals Mixing time estimation in reversible Markov chains from a single sample path

2019 ◽  
Vol 29 (4) ◽  
pp. 2439-2480
Author(s):  
Daniel Hsu ◽  
Aryeh Kontorovich ◽  
David A. Levin ◽  
Yuval Peres ◽  
Csaba Szepesvári ◽  
...  
Keyword(s):  
Markov Chains ◽  
Mixing Time ◽  
Sample Path ◽  
Time Estimation ◽  
Single Sample ◽  
Reversible Markov Chains

Potentially unlimited variance reduction in importance sampling of Markov chains

1996 ◽  
Vol 28 (01) ◽  
pp. 166-188
Author(s):  
Sigrún Andradóttir ◽  
Daniel P. Heyman ◽  
Teunis J. Ott
Keyword(s):  
Steady State ◽  
Markov Chains ◽  
Importance Sampling ◽  
Path Length ◽  
Transition Matrix ◽  
Variance Reduction ◽  
Sample Path ◽  
Computer Time ◽  
Performance Measure ◽  
Transition Matrices

We consider the application of importance sampling in steady-state simulations of finite Markov chains. We show that, for a large class of performance measures, there is a choice of the alternative transition matrix for which the ratio of the variance of the importance sampling estimator to the variance of the naive simulation estimator converges to zero as the sample path length goes to infinity. Obtaining this ‘optimal’ transition matrix involves computing the performance measure of interest, so the optimal matrix cannot be computed in precisely those situations where simulation is required to estimate steady-state performance. However, our results show that alternative transition matrices of the form Q = P + E/T, where P is the original transition matrix and T is the sample path length, can be expected to provide good results. Moreover, we provide an iterative algorithm for obtaining alternative transition matrices of this form that converge to the optimal matrix as the number of iterations increases, and present an example that shows that spending some computer time iterating this algorithm and then conducting the simulation with the resulting alternative transition matrix may provide considerable variance reduction when compared to naive simulation.

Convergence of perturbation analysis estimates for discontinuous sample functions: a general approach

1988 ◽  
Vol 20 (01) ◽  
pp. 59-78
Author(s):  
Reuven Y. Rubinstein ◽  
Ferenc Szidarovszky
Keyword(s):  
Monte Carlo ◽  
Rate Of Convergence ◽  
Performance Measures ◽  
Dynamic Systems ◽  
Perturbation Analysis ◽  
Sample Path ◽  
Single Sample ◽  
Parameter Vector ◽  
Discrete Events ◽  
Convergence Conditions

Generalized perturbation analysis (PA) estimates to study sensitivity of performance measures of discrete events dynamic systems for discontinuous sample functions are introduced. Their convergence conditions and rate of convergence are given. It is shown that the PA estimates based on a single sample path always converge faster to the unknown sensitivity parameter (vector of parameters) than their counterpart—crude Monte Carlo ones.

Sign in / Sign up

Export Citation Format

Share Document