On some mixing times for nonreversible finite Markov chains

2017 ◽  
Vol 54 (2) ◽  
pp. 627-637 ◽  
Author(s):  
Lu-Jing Huang ◽  
Yong-Hua Mao
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Affirmative Answer ◽  
Hitting Time ◽  
Mixing Times ◽  
Reversible Transition ◽  
Commute Time ◽  
Finite Markov Chains

Abstract By adding a vorticity matrix to the reversible transition probability matrix, we show that the commute time and average hitting time are smaller than that of the original reversible one. In particular, we give an affirmative answer to a conjecture of Aldous and Fill (2002). Further quantitive properties are also studied for the nonreversible finite Markov chains.

scholarly journals A geometric invariant in weak lumpability of finite Markov chains

1997 ◽  
Vol 34 (4) ◽  
pp. 847-858 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Spectral Properties ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Homogeneous Markov Chain ◽  
Geometric Invariant ◽  
Initial State ◽  
Direct Sum ◽  
Finite Markov Chains

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

Weak lumpability in finite Markov chains

1982 ◽  
Vol 19 (03) ◽  
pp. 685-691 ◽  
Author(s):  
Atef M. Abdel-moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Transition Probability ◽  
Linear Equations ◽  
Transition Probability Matrix ◽  
Finite Markov Chain ◽  
Systems Of Linear Equations ◽  
Finite Markov Chains ◽  
The Given

Criteria are given to determine whether a given finite Markov chain can be lumped weakly with respect to a given partition of its state space. These conditions are given in terms of solution classes of systems of linear equations associated with the transition probability matrix of the Markov chain and the given partition.

Weak lumpability in finite Markov chains

1982 ◽  
Vol 19 (3) ◽  
pp. 685-691 ◽  
Author(s):  
Atef M. Abdel-moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Transition Probability ◽  
Linear Equations ◽  
Transition Probability Matrix ◽  
Finite Markov Chain ◽  
Systems Of Linear Equations ◽  
Finite Markov Chains ◽  
The Given

Criteria are given to determine whether a given finite Markov chain can be lumped weakly with respect to a given partition of its state space. These conditions are given in terms of solution classes of systems of linear equations associated with the transition probability matrix of the Markov chain and the given partition.

scholarly journals A geometric invariant in weak lumpability of finite Markov chains

1997 ◽  
Vol 34 (04) ◽  
pp. 847-858 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Spectral Properties ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Homogeneous Markov Chain ◽  
Geometric Invariant ◽  
Initial State ◽  
Direct Sum ◽  
Finite Markov Chains

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

scholarly journals Computable bounds of an 𝓁²-spectral gap for discrete Markov chains with band transition matrices

2016 ◽  
Vol 53 (3) ◽  
pp. 946-952
Author(s):  
Loï Hervé ◽  
James Ledoux
Keyword(s):  
Convergence Rate ◽  
Markov Chains ◽  
Spectral Radius ◽  
Spectral Gap ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Geometric Ergodicity ◽  
Transition Matrices ◽  
Essential Spectral Radius ◽  
Band Transition

AbstractWe analyse the 𝓁²(𝜋)-convergence rate of irreducible and aperiodic Markov chains with N-band transition probability matrix P and with invariant distribution 𝜋. This analysis is heavily based on two steps. First, the study of the essential spectral radius ress(P|𝓁²(𝜋)) of P|𝓁²(𝜋) derived from Hennion’s quasi-compactness criteria. Second, the connection between the spectral gap property (SG2) of P on 𝓁²(𝜋) and the V-geometric ergodicity of P. Specifically, the (SG2) is shown to hold under the condition α0≔∑m=−NNlim supi→+∞(P(i,i+m)P*(i+m,i)1∕2<1. Moreover, ress(P|𝓁²(𝜋)≤α0. Effective bounds on the convergence rate can be provided from a truncation procedure.

Spectral representations of the transition probability matrices for continuous time finite Markov chains

1996 ◽  
Vol 33 (1) ◽  
pp. 28-33 ◽  
Author(s):  
Nan Fu Peng
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Probability ◽  
Algebraic Method ◽  
Finite State ◽  
Transition Probability Matrices ◽  
Finite Markov Chains ◽  
Probability Matrices ◽  
Spectral Representations

Using an easy linear-algebraic method, we obtain spectral representations, without the need for eigenvector determination, of the transition probability matrices for completely general continuous time Markov chains with finite state space. Comparing the proof presented here with that of Brown (1991), who provided a similar result for a special class of finite Markov chains, we observe that ours is more concise.

Cheeger inequalities for absorbing Markov chains

2016 ◽  
Vol 48 (3) ◽  
pp. 631-647
Author(s):  
Gary Froyland ◽  
Robyn M. Stuart
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Transition Matrices ◽  
Stochastic Transition ◽  
Absorbing Markov Chains ◽  
Second Eigenvalue

Abstract We construct Cheeger-type bounds for the second eigenvalue of a substochastic transition probability matrix in terms of the Markov chain's conductance and metastability (and vice versa) with respect to its quasistationary distribution, extending classical results for stochastic transition matrices.

scholarly journals Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

2008 ◽  
Vol 45 (01) ◽  
pp. 211-225 ◽  
Author(s):  
Alexander Dudin ◽  
Chesoong Kim ◽  
Valentina Klimenok
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Transition Probability ◽  
Sufficient Conditions ◽  
Transition Probability Matrix ◽  
Toeplitz Structure

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

scholarly journals Computing Hitting Probabilities of Markov Chains: Structural Results with regard to the Solution Space of the Corresponding System of Equations

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Hendrik Baumann ◽  
Thomas Hanschke
Keyword(s):  
Markov Chains ◽  
Transition Probability ◽  
Solution Space ◽  
Transition Probability Matrix ◽  
Discrete State ◽  
Side Condition ◽  
Hitting Probabilities ◽  
All Solutions ◽  
The Difference ◽  
State Difference

In a previous paper, we have shown that forward use of the steady-state difference equations arising from homogeneous discrete-state space Markov chains may be subject to inherent numerical instability. More precisely, we have proven that, under some appropriate assumptions on the transition probability matrix P, the solution space S of the difference equation may be partitioned into two subspaces S=S1⊕S2, where the stationary measure of P is an element of S1, and all solutions in S1 are asymptotically dominated by the solutions corresponding to S2. In this paper, we discuss the analogous problem of computing hitting probabilities of Markov chains, which is affected by the same numerical phenomenon. In addition, we have to fulfill a somewhat complicated side condition which essentially differs from those conditions one is usually confronted with when solving initial and boundary value problems. To extract the desired solution, an efficient and numerically stable generalized-continued-fraction-based algorithm is developed.

The square root law in the embedding detection problem for Markov chains with unknown matrix of transition probabilities

2019 ◽  
Vol 29 (1) ◽  
pp. 59-68
Author(s):  
Artem V. Volgin
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Classical Model ◽  
Transition Probability Matrix ◽  
Square Root ◽  
Detection Problem ◽  
Asymptotic Growth

Abstract We consider the classical model of embeddings in a simple binary Markov chain with unknown transition probability matrix. We obtain conditions on the asymptotic growth of lengths of the original and embedded sequences sufficient for the consistency of the proposed statistical embedding detection test.

Sign in / Sign up

Export Citation Format

Share Document