Computing the Stationary Distribution of a Finite Markov Chain Through Stochastic Factorization

2011 ◽  
Vol 32 (4) ◽  
pp. 1513-1523
Author(s):  
André M. S. Barreto ◽  
Marcelo D. Fragoso
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Finite Markov Chain

Perturbation bounds for the stationary probabilities of a finite Markov chain

1984 ◽  
Vol 16 (04) ◽  
pp. 804-818 ◽  
Author(s):  
Moshe Haviv ◽  
Ludo Van Der Heyden
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Finite Markov Chain ◽  
Perturbation Bounds ◽  
Aggregation Technique ◽  
Stationary Probabilities

This paper discusses perturbation bounds for the stationary distribution of a finite indecomposable Markov chain. Existing bounds are reviewed. New bounds are presented which more completely exploit the stochastic features of the perturbation and which also are easily computable. Examples illustrate the tightness of the bounds and their application to bounding the error in the Simon–Ando aggregation technique for approximating the stationary distribution of a nearly completely decomposable Markov chain.

scholarly journals Normal distributions of finite Markov chains

2019 ◽  
Vol 29 (08) ◽  
pp. 1431-1449
Author(s):  
John Rhodes ◽  
Anne Schilling
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Planar Graphs ◽  
Finite Semigroup ◽  
Finite Markov Chain ◽  
Normal Distributions ◽  
Straight Line ◽  
Finite Markov Chains ◽  
The Right

We show that the stationary distribution of a finite Markov chain can be expressed as the sum of certain normal distributions. These normal distributions are associated to planar graphs consisting of a straight line with attached loops. The loops touch only at one vertex either of the straight line or of another attached loop. Our analysis is based on our previous work, which derives the stationary distribution of a finite Markov chain using semaphore codes on the Karnofsky–Rhodes and McCammond expansion of the right Cayley graph of the finite semigroup underlying the Markov chain.

Monotonicity of the Loss Probabilities of Single Server Finite Queues with Respect to Convex Order of Arrival or Service Processes

1991 ◽  
Vol 5 (1) ◽  
pp. 43-52 ◽  
Author(s):  
Masakiyo Miyazawa ◽  
J. George Shanthikumar
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Batch Size ◽  
Service Process ◽  
Single Server ◽  
Finite Markov Chain ◽  
Convex Order ◽  
Finite Queues ◽  
Loss Probabilities

The loss probabilities of customers in the Mx/GI/1/k, GI/Mx/l/k and their related queues such as server vacation models are compared with respect to the convex order of several characteristics, for example, batch size, of the arrival or service process. In the proof, we give a characterization of a truncation expression for a stationary distribution of a finite Markov chain, which is interesting in itself.

scholarly journals SERIES EXPANSIONS FOR FINITE-STATE MARKOV CHAINS

2007 ◽  
Vol 21 (3) ◽  
pp. 381-400 ◽  
Author(s):  
Bernd Heidergott ◽  
Arie Hordijk ◽  
Miranda van Uitert
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Numerical Algorithm ◽  
Series Expansions ◽  
Finite Markov Chain ◽  
Numerical Examples ◽  
Finite State

This article provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

Perturbation bounds for the stationary probabilities of a finite Markov chain

1984 ◽  
Vol 16 (4) ◽  
pp. 804-818 ◽  
Author(s):  
Moshe Haviv ◽  
Ludo Van Der Heyden
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Finite Markov Chain ◽  
Perturbation Bounds ◽  
Aggregation Technique ◽  
Stationary Probabilities

This paper discusses perturbation bounds for the stationary distribution of a finite indecomposable Markov chain. Existing bounds are reviewed. New bounds are presented which more completely exploit the stochastic features of the perturbation and which also are easily computable. Examples illustrate the tightness of the bounds and their application to bounding the error in the Simon–Ando aggregation technique for approximating the stationary distribution of a nearly completely decomposable Markov chain.

Approximating the stationary distribution of an infinite stochastic matrix

1991 ◽  
Vol 28 (1) ◽  
pp. 96-103 ◽  
Author(s):  
Daniel P. Heyman
Keyword(s):  
Markov Chain ◽  
Steady State ◽  
Stationary Distribution ◽  
Numerical Approximation ◽  
Stochastic Matrix ◽  
Sufficient Conditions ◽  
Finite System ◽  
Balance Equations ◽  
The Given ◽  
Steady State Balance

We are given a Markov chain with states 0, 1, 2, ···. We want to get a numerical approximation of the steady-state balance equations. To do this, we truncate the chain, keeping the first n states, make the resulting matrix stochastic in some convenient way, and solve the finite system. The purpose of this paper is to provide some sufficient conditions that imply that as n tends to infinity, the stationary distributions of the truncated chains converge to the stationary distribution of the given chain. Our approach is completely probabilistic, and our conditions are given in probabilistic terms. We illustrate how to verify these conditions with five examples.

scholarly journals Stationary distribution Markov chain for Covid-19 pandemic

2021 ◽  
Vol 1722 ◽  
pp. 012084
Author(s):  
A L H Achmad ◽  
Mahrudinda ◽  
B N Ruchjana
Keyword(s):  
Markov Chain ◽  
Stationary Distribution

An extension of a theorem concerning an interesting Markov chain

1973 ◽  
Vol 10 (4) ◽  
pp. 886-890 ◽  
Author(s):  
W. J. Hendricks
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Ergodic Markov Chain ◽  
The Right

In a single-shelf library of N books we suppose that books are selected one at a time and returned to the kth position on the shelf before another selection is made. Books are moved to the right or left as necessary to vacate position k. The probability of selecting each book is assumed to be known, and the N! arrangements of the books are considered as states of an ergodic Markov chain for which we find the stationary distribution.

Sign in / Sign up

Export Citation Format

Share Document