Applications of generalized inverses to Markov chains

1991 ◽  
Vol 23 (2) ◽  
pp. 293-302 ◽  
Author(s):  
William Rising
Keyword(s):  
Stationary Distribution ◽  
Infinitesimal Generator ◽  
First Passage Times ◽  
Generalized Inverses ◽  
Single Server ◽  
First Passage ◽  
Triangular Matrices ◽  
Special Case ◽  
Irreducible Markov Chain ◽  
Passage Times

First it is shown that any generalized inverse of the infinitesimal generator of an irreducible Markov chain can be used to compute the exact stationary distribution and all the expected first-passage times of the chain. In the special case of a single-server queue this allows all computations to be done with upper-triangular matrices.Next it is shown that the effect of a perturbation of the infinitesimal generator on the stationary distribution and expected first-passage times can also be computed using generalized inverses. These results extend and generalize Schweitzer's [9] original work using fundamental matrices. It is then shown that any perturbation can be broken up into a series of perturbations each involving a single state.

Applications of generalized inverses to Markov chains

1991 ◽  
Vol 23 (02) ◽  
pp. 293-302 ◽  
Author(s):  
William Rising
Keyword(s):  
Stationary Distribution ◽  
Infinitesimal Generator ◽  
First Passage Times ◽  
Generalized Inverses ◽  
Single Server ◽  
First Passage ◽  
Triangular Matrices ◽  
Special Case ◽  
Irreducible Markov Chain ◽  
Passage Times

First it is shown that any generalized inverse of the infinitesimal generator of an irreducible Markov chain can be used to compute the exact stationary distribution and all the expected first-passage times of the chain. In the special case of a single-server queue this allows all computations to be done with upper-triangular matrices. Next it is shown that the effect of a perturbation of the infinitesimal generator on the stationary distribution and expected first-passage times can also be computed using generalized inverses. These results extend and generalize Schweitzer's [9] original work using fundamental matrices. It is then shown that any perturbation can be broken up into a series of perturbations each involving a single state.

Geometric Markov chains

1995 ◽  
Vol 32 (02) ◽  
pp. 349-374
Author(s):  
William Rising
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Nearest Neighbor ◽  
First Passage Times ◽  
First Passage ◽  
Death Rates ◽  
Passage Times ◽  
Birth Death

A generalization of the familiar birth–death chain, called the geometric chain, is introduced and explored. By the introduction of two families of parameters in addition to the infinitesimal birth and death rates, the geometric chain allows transitions beyond the nearest neighbor, but is shown to retain the simple computational formulas of the birth–death chain for the stationary distribution and the expected first-passage times between states. It is also demonstrated that even when not reversible, a reversed geometric chain is again a geometric chain.

scholarly journals SIMPLE PROCEDURES FOR FINDING MEAN FIRST PASSAGE TIMES IN MARKOV CHAINS

2007 ◽  
Vol 24 (06) ◽  
pp. 813-829 ◽  
Author(s):  
JEFFREY J. HUNTER
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Linear Equations ◽  
First Passage Times ◽  
First Passage ◽  
Mean First Passage Times ◽  
The Mean ◽  
Computational Procedures ◽  
Passage Times

The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell's fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse of I - P + eπT where eT = (1, 1, …, 1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures.

Geometric Markov chains

1995 ◽  
Vol 32 (2) ◽  
pp. 349-374
Author(s):  
William Rising
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Nearest Neighbor ◽  
First Passage Times ◽  
First Passage ◽  
Death Rates ◽  
Passage Times ◽  
Birth Death

A generalization of the familiar birth–death chain, called the geometric chain, is introduced and explored. By the introduction of two families of parameters in addition to the infinitesimal birth and death rates, the geometric chain allows transitions beyond the nearest neighbor, but is shown to retain the simple computational formulas of the birth–death chain for the stationary distribution and the expected first-passage times between states. It is also demonstrated that even when not reversible, a reversed geometric chain is again a geometric chain.

ON THE DIFFERENCE BETWEEN TWO FIRST PA AGE TIME

2001 ◽  
Vol 37 (1-2) ◽  
pp. 53-67
Author(s):  
L. Larsson-Cohn
Keyword(s):  
Law Of Large Numbers ◽  
First Passage Times ◽  
First Passage ◽  
Large Numbers ◽  
The Difference ◽  
Special Case ◽  
Passage Times

First passage times for pe tu bed andom walks are compa ed to the cor esponding times for its unpertu bed version.A weak and a st ong law of large numbers for their di .erence is established and applied to a special case.

Sign in / Sign up

Export Citation Format

Share Document