On a control of a Markov chain under conditions with respect to the absolute stationary probabilities and cost

1980 ◽  
Vol 24 (3) ◽  
pp. 73-81
Author(s):  
A. Slobodová
Keyword(s):  
Markov Chain ◽  
The Absolute ◽  
Stationary Probabilities

Perturbation bounds for the stationary probabilities of a finite Markov chain

1984 ◽  
Vol 16 (04) ◽  
pp. 804-818 ◽  
Author(s):  
Moshe Haviv ◽  
Ludo Van Der Heyden
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Finite Markov Chain ◽  
Perturbation Bounds ◽  
Aggregation Technique ◽  
Stationary Probabilities

This paper discusses perturbation bounds for the stationary distribution of a finite indecomposable Markov chain. Existing bounds are reviewed. New bounds are presented which more completely exploit the stochastic features of the perturbation and which also are easily computable. Examples illustrate the tightness of the bounds and their application to bounding the error in the Simon–Ando aggregation technique for approximating the stationary distribution of a nearly completely decomposable Markov chain.

The censored Markov chain and the best augmentation

1996 ◽  
Vol 33 (03) ◽  
pp. 623-629 ◽  
Author(s):  
Y. Quennel Zhao ◽  
Danielle Liu
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Stationary Probability ◽  
Stationary Probability Distribution ◽  
The Best Approximation ◽  
Finite Matrix ◽  
Countable State ◽  
Stationary Probabilities

Computationally, when we solve for the stationary probabilities for a countable-state Markov chain, the transition probability matrix of the Markov chain has to be truncated, in some way, into a finite matrix. Different augmentation methods might be valid such that the stationary probability distribution for the truncated Markov chain approaches that for the countable Markov chain as the truncation size gets large. In this paper, we prove that the censored (watched) Markov chain provides the best approximation in the sense that, for a given truncation size, the sum of errors is the minimum and show, by examples, that the method of augmenting the last column only is not always the best.

An interesting random walk on the non-negative integers

1994 ◽  
Vol 31 (1) ◽  
pp. 48-58 ◽  
Author(s):  
Marcel F. Neuts
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Numerical Computation ◽  
Continuous Version ◽  
Positive Recurrence ◽  
Mathematical Arguments ◽  
Stationary Probabilities ◽  
Algorithmic Procedure

A particular random walk on the integers leads to a new, tractable Markov chain. Of the stationary probabilities, we discuss the existence, some analytic properties and a factorization which leads to an algorithmic procedure for their numerical computation. We also consider the positive recurrence of some variants which each call for different mathematical arguments. Analogous results are derived for a continuous version on the positive reals.

An interesting random walk on the non-negative integers

1994 ◽  
Vol 31 (01) ◽  
pp. 48-58 ◽  
Author(s):  
Marcel F. Neuts
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Numerical Computation ◽  
Continuous Version ◽  
Positive Recurrence ◽  
Mathematical Arguments ◽  
Stationary Probabilities ◽  
Algorithmic Procedure

A particular random walk on the integers leads to a new, tractable Markov chain. Of the stationary probabilities, we discuss the existence, some analytic properties and a factorization which leads to an algorithmic procedure for their numerical computation. We also consider the positive recurrence of some variants which each call for different mathematical arguments. Analogous results are derived for a continuous version on the positive reals.

Perturbation bounds for the stationary probabilities of a finite Markov chain

1984 ◽  
Vol 16 (4) ◽  
pp. 804-818 ◽  
Author(s):  
Moshe Haviv ◽  
Ludo Van Der Heyden
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Finite Markov Chain ◽  
Perturbation Bounds ◽  
Aggregation Technique ◽  
Stationary Probabilities

This paper discusses perturbation bounds for the stationary distribution of a finite indecomposable Markov chain. Existing bounds are reviewed. New bounds are presented which more completely exploit the stochastic features of the perturbation and which also are easily computable. Examples illustrate the tightness of the bounds and their application to bounding the error in the Simon–Ando aggregation technique for approximating the stationary distribution of a nearly completely decomposable Markov chain.

The censored Markov chain and the best augmentation

1996 ◽  
Vol 33 (3) ◽  
pp. 623-629 ◽  
Author(s):  
Y. Quennel Zhao ◽  
Danielle Liu
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Stationary Probability ◽  
Stationary Probability Distribution ◽  
The Best Approximation ◽  
Finite Matrix ◽  
Countable State ◽  
Stationary Probabilities

Computationally, when we solve for the stationary probabilities for a countable-state Markov chain, the transition probability matrix of the Markov chain has to be truncated, in some way, into a finite matrix. Different augmentation methods might be valid such that the stationary probability distribution for the truncated Markov chain approaches that for the countable Markov chain as the truncation size gets large. In this paper, we prove that the censored (watched) Markov chain provides the best approximation in the sense that, for a given truncation size, the sum of errors is the minimum and show, by examples, that the method of augmenting the last column only is not always the best.

scholarly journals Description of the Markov chain using the producing function

2021 ◽  
Vol 2131 (3) ◽  
pp. 032040
Author(s):  
T A Shornikova
Keyword(s):  
Markov Chain ◽  
Stochastic Model ◽  
Generating Function ◽  
Inverse Matrix ◽  
Characteristic Numbers ◽  
Secular Equations ◽  
Definition Of ◽  
Under Construction ◽  
Transitional Probabilities ◽  
Stationary Probabilities

Abstract In article the way of creation of an assumed function of transformation with the help of a generating function and also a method of use of characteristic numbers and vectors for creation of a matrix which elements well describe conditions of process at any moment is described. This approach differs from preceding that, having used a concept of characteristic numbers and characteristic vectors of a matrix of the transitional probabilities, it is possible to simplify considerably calculation of the elements characterizing process. In article methods of stochastic model operation, ways of the description of a generating function, the solution of matrixes of the equations by means of characteristic numbers and vectors are used. Using properties of a generating function, made “dictionary” of z-transformations which helped to define an assumed function of transformation. The generating function of a vector was applied to a research of behavior of a vector of absolute probabilities which elements represent stationary probabilities. For definition of degree of a matrix of transition of probabilities used a concept of characteristic numbers and characteristic vectors of the transitional probabilities. Determined by such way an unlimited set of latent vectors of which made matrixes which describe a condition of a system at any moment. Reception of definition of latent vectors in more difficult examples which is that along with required coefficients of secular equations the system of auxiliary matrixes and an inverse matrix is under construction is also described.

Sign in / Sign up

Export Citation Format

Share Document