Weak lumpability in finite Markov chains

1982 ◽  
Vol 19 (3) ◽  
pp. 685-691 ◽  
Author(s):  
Atef M. Abdel-moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Transition Probability ◽  
Linear Equations ◽  
Transition Probability Matrix ◽  
Finite Markov Chain ◽  
Systems Of Linear Equations ◽  
Finite Markov Chains ◽  
The Given

Criteria are given to determine whether a given finite Markov chain can be lumped weakly with respect to a given partition of its state space. These conditions are given in terms of solution classes of systems of linear equations associated with the transition probability matrix of the Markov chain and the given partition.

Weak lumpability in finite Markov chains

1982 ◽  
Vol 19 (03) ◽  
pp. 685-691 ◽  
Author(s):  
Atef M. Abdel-moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Transition Probability ◽  
Linear Equations ◽  
Transition Probability Matrix ◽  
Finite Markov Chain ◽  
Systems Of Linear Equations ◽  
Finite Markov Chains ◽  
The Given

Criteria are given to determine whether a given finite Markov chain can be lumped weakly with respect to a given partition of its state space. These conditions are given in terms of solution classes of systems of linear equations associated with the transition probability matrix of the Markov chain and the given partition.

scholarly journals A geometric invariant in weak lumpability of finite Markov chains

1997 ◽  
Vol 34 (4) ◽  
pp. 847-858 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Spectral Properties ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Homogeneous Markov Chain ◽  
Geometric Invariant ◽  
Initial State ◽  
Direct Sum ◽  
Finite Markov Chains

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

scholarly journals A geometric invariant in weak lumpability of finite Markov chains

1997 ◽  
Vol 34 (04) ◽  
pp. 847-858 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Spectral Properties ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Homogeneous Markov Chain ◽  
Geometric Invariant ◽  
Initial State ◽  
Direct Sum ◽  
Finite Markov Chains

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

Lumpability for non-irreducible finite markov chains

1984 ◽  
Vol 21 (03) ◽  
pp. 567-574 ◽  
Author(s):  
Atef M. Abdel-Moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
The State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

Conditions under which a function of a finite, discrete-time Markov chain, X(t), is again Markov are given, when X(t) is not irreducible. These conditions are given in terms of an interrelationship between two partitions of the state space of X(t), the partition induced by the minimal essential classes of X(t) and the partition with respect to which lumping is to be considered.

scholarly journals Normal distributions of finite Markov chains

2019 ◽  
Vol 29 (08) ◽  
pp. 1431-1449
Author(s):  
John Rhodes ◽  
Anne Schilling
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Planar Graphs ◽  
Finite Semigroup ◽  
Finite Markov Chain ◽  
Normal Distributions ◽  
Straight Line ◽  
Finite Markov Chains ◽  
The Right

We show that the stationary distribution of a finite Markov chain can be expressed as the sum of certain normal distributions. These normal distributions are associated to planar graphs consisting of a straight line with attached loops. The loops touch only at one vertex either of the straight line or of another attached loop. Our analysis is based on our previous work, which derives the stationary distribution of a finite Markov chain using semaphore codes on the Karnofsky–Rhodes and McCammond expansion of the right Cayley graph of the finite semigroup underlying the Markov chain.

Some explicit results for correlated random walks

1989 ◽  
Vol 26 (4) ◽  
pp. 757-766 ◽  
Author(s):  
Ram Lal ◽  
U. Narayan Bhat
Keyword(s):  
Markov Chain ◽  
State Space ◽  
Equilibrium Distribution ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Correlated Random Walks ◽  
Finite State ◽  
Direction Of Movement ◽  
Step Transition

In a correlated random walk (CRW) the probabilities of movement in the positive and negative direction are given by the transition probabilities of a Markov chain. The walk can be represented as a Markov chain if we use a bivariate state space, with the location of the particle and the direction of movement as the two variables. In this paper we derive explicit results for the following characteristics of the walk directly from its transition probability matrix: (i) n -step transition probabilities for the unrestricted CRW, (ii) equilibrium distribution for the CRW restricted on one side, and (iii) equilibrium distribution and first-passage characteristics for the CRW restricted on both sides (i.e., with finite state space).

The square root law in the embedding detection problem for Markov chains with unknown matrix of transition probabilities

2019 ◽  
Vol 29 (1) ◽  
pp. 59-68
Author(s):  
Artem V. Volgin
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Classical Model ◽  
Transition Probability Matrix ◽  
Square Root ◽  
Detection Problem ◽  
Asymptotic Growth

Abstract We consider the classical model of embeddings in a simple binary Markov chain with unknown transition probability matrix. We obtain conditions on the asymptotic growth of lengths of the original and embedded sequences sufficient for the consistency of the proposed statistical embedding detection test.

Some explicit results for correlated random walks

1989 ◽  
Vol 26 (04) ◽  
pp. 757-766 ◽  
Author(s):  
Ram Lal ◽  
U. Narayan Bhat
Keyword(s):  
Markov Chain ◽  
State Space ◽  
Equilibrium Distribution ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Correlated Random Walks ◽  
Finite State ◽  
Direction Of Movement ◽  
Step Transition

In a correlated random walk (CRW) the probabilities of movement in the positive and negative direction are given by the transition probabilities of a Markov chain. The walk can be represented as a Markov chain if we use a bivariate state space, with the location of the particle and the direction of movement as the two variables. In this paper we derive explicit results for the following characteristics of the walk directly from its transition probability matrix: (i) n -step transition probabilities for the unrestricted CRW, (ii) equilibrium distribution for the CRW restricted on one side, and (iii) equilibrium distribution and first-passage characteristics for the CRW restricted on both sides (i.e., with finite state space).

A modification of a result due to Moran

1968 ◽  
Vol 5 (01) ◽  
pp. 220-223 ◽  
Author(s):  
Barry C. Arnold
Keyword(s):  
Markov Chain ◽  
State Space ◽  
Transition Probability ◽  
Transition Probability Matrix

Let {X(n): n = 0, 1, 2, …} be a Markov chain with state space {0, 1, 2, …, N} and transition probability matrix P = (pij ).

Sign in / Sign up

Export Citation Format

Share Document