A single sample path-based performance sensitivity formula for Markov chains

1996 ◽  
Vol 41 (12) ◽  
pp. 1814-1817 ◽  
Author(s):  
Xi-Ren Cao ◽  
Xue-Ming Yuan ◽  
Li Qiu
Keyword(s):  
Markov Chains ◽  
Sample Path ◽  
Single Sample ◽  
Performance Sensitivity

A Single Sample Path-Based Performance Sensitivity Formula for Markov Chains

1996 ◽  
Vol 29 (1) ◽  
pp. 4795-4800
Author(s):  
Xi-Ren Cao ◽  
Xue-Ming Yuan ◽  
Li Qiu
Keyword(s):  
Markov Chains ◽  
Sample Path ◽  
Single Sample ◽  
Performance Sensitivity

The Maclaurin series for performance functions of Markov chains

1998 ◽  
Vol 30 (3) ◽  
pp. 676-692 ◽  
Author(s):  
Xi-Ren Cao
Keyword(s):  
Markov Chains ◽  
Performance Measures ◽  
Sample Path ◽  
Single Sample ◽  
Maclaurin Series ◽  
Transition Matrices ◽  
Engineering Problems ◽  
Realization Factors ◽  
Derivatives Of ◽  
Steady State Performance

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.

scholarly journals Mixing time estimation in reversible Markov chains from a single sample path

2019 ◽  
Vol 29 (4) ◽  
pp. 2439-2480
Author(s):  
Daniel Hsu ◽  
Aryeh Kontorovich ◽  
David A. Levin ◽  
Yuval Peres ◽  
Csaba Szepesvári ◽  
...  
Keyword(s):  
Markov Chains ◽  
Mixing Time ◽  
Sample Path ◽  
Time Estimation ◽  
Single Sample ◽  
Reversible Markov Chains

The Maclaurin series for performance functions of Markov chains

1998 ◽  
Vol 30 (03) ◽  
pp. 676-692 ◽  
Author(s):  
Xi-Ren Cao
Keyword(s):  
Markov Chains ◽  
Performance Measures ◽  
Sample Path ◽  
Single Sample ◽  
Maclaurin Series ◽  
Transition Matrices ◽  
Engineering Problems ◽  
Realization Factors ◽  
Derivatives Of ◽  
Steady State Performance

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.

Recursive Approaches for Single Sample Path Based Markov Reward Processes

2008 ◽  
Vol 3 (1) ◽  
pp. 21-26 ◽  
Author(s):  
Hai-Tao Fang ◽  
Han-Fu Chen ◽  
Xi-Ren Cao
Keyword(s):  
Sample Path ◽  
Single Sample ◽  
Markov Reward

Event coupling and performance sensitivity analysis of generalized semi-Markov processes

1995 ◽  
Vol 27 (03) ◽  
pp. 741-769
Author(s):  
Xi-Ren Cao
Keyword(s):  
Steady State ◽  
Markov Processes ◽  
Queueing Networks ◽  
Sample Path ◽  
Performance Measure ◽  
Coupling Property ◽  
Performance Sensitivity ◽  
Realization Factors ◽  
Semi Markov Processes ◽  
Steady State Performance

We study a fundamental feature of the generalized semi-Markov processes (GSMPs), called event coupling. The event coupling reflects the logical behavior of a GSMP that specifies which events can be affected by any given event. Based on the event-coupling property, GSMPs can be classified into three classes: the strongly coupled, the hierarchically coupled, and the decomposable GSMPs. The event-coupling property on a sample path of a GSMP can be represented by the event-coupling trees. With the event-coupling tree, we can quantify the effect of a single perturbation on a performance measure by using realization factors. A set of equations that specifies the realization factors is derived. We show that the sensitivity of steady-state performance with respect to a parameter of an event lifetime distribution can be obtained by a simple formula based on realization factors and that the sample-path performance sensitivity converges to the sensitivity of the steady-state performance with probability one as the length of the sample path goes to infinity. This generalizes the existing results of perturbation analysis of queueing networks to GSMPs.

Sign in / Sign up

Export Citation Format

Share Document