Single sample path-based sensitivity analysis of Markov processes using uniformization

1999 ◽  
Vol 44 (4) ◽  
pp. 872-875 ◽  
Author(s):  
Zikuan Liu ◽  
Fengsheng Tu
Keyword(s):  
Sensitivity Analysis ◽  
Markov Processes ◽  
Sample Path ◽  
Single Sample

scholarly journals A Sample Path Proof of the Duality for Stochastically Monotone Markov Processes

1985 ◽  
Vol 13 (2) ◽  
pp. 558-565 ◽  
Author(s):  
Peter Clifford ◽  
Aidan Sudbury
Keyword(s):  
Markov Processes ◽  
Sample Path

Recursive Approaches for Single Sample Path Based Markov Reward Processes

2008 ◽  
Vol 3 (1) ◽  
pp. 21-26 ◽  
Author(s):  
Hai-Tao Fang ◽  
Han-Fu Chen ◽  
Xi-Ren Cao
Keyword(s):  
Sample Path ◽  
Single Sample ◽  
Markov Reward

A single sample path-based performance sensitivity formula for Markov chains

1996 ◽  
Vol 41 (12) ◽  
pp. 1814-1817 ◽  
Author(s):  
Xi-Ren Cao ◽  
Xue-Ming Yuan ◽  
Li Qiu
Keyword(s):  
Markov Chains ◽  
Sample Path ◽  
Single Sample ◽  
Performance Sensitivity

The Maclaurin series for performance functions of Markov chains

1998 ◽  
Vol 30 (3) ◽  
pp. 676-692 ◽  
Author(s):  
Xi-Ren Cao
Keyword(s):  
Markov Chains ◽  
Performance Measures ◽  
Sample Path ◽  
Single Sample ◽  
Maclaurin Series ◽  
Transition Matrices ◽  
Engineering Problems ◽  
Realization Factors ◽  
Derivatives Of ◽  
Steady State Performance

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.

Sign in / Sign up

Export Citation Format

Share Document