Coefficients of ergodicity for stochastically monotone Markov chains

1992 ◽  
Vol 29 (4) ◽  
pp. 850-860 ◽  
Author(s):  
G. Ch. Pflug ◽  
W. Schachermayer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Probabilistic Interpretation ◽  
Speed Of Convergence ◽  
Transition Matrices ◽  
Wasserstein Metric ◽  
Ergodic Markov Chain ◽  
Finite State ◽  
Second Eigenvalue ◽  
Finite State Space

In this paper we show that to each distance d defined on the finite state space S of a strongly ergodic Markov chain there corresponds a coefficient ρd of ergodicity based on the Wasserstein metric. For a class of stochastically monotone transition matrices P, the infimum over all such coefficients is given by the spectral radius of P – R, where R = limkPk and is attained. This result has a probabilistic interpretation of a control of the speed of convergence of by the metric d and is linked to the second eigenvalue of P.

Coefficients of ergodicity for stochastically monotone Markov chains

1992 ◽  
Vol 29 (04) ◽  
pp. 850-860 ◽  
Author(s):  
G. Ch. Pflug ◽  
W. Schachermayer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Probabilistic Interpretation ◽  
Speed Of Convergence ◽  
Transition Matrices ◽  
Wasserstein Metric ◽  
Ergodic Markov Chain ◽  
Finite State ◽  
Second Eigenvalue ◽  
Finite State Space

In this paper we show that to each distance d defined on the finite state space S of a strongly ergodic Markov chain there corresponds a coefficient ρd of ergodicity based on the Wasserstein metric. For a class of stochastically monotone transition matrices P, the infimum over all such coefficients is given by the spectral radius of P – R, where R = lim k Pk and is attained. This result has a probabilistic interpretation of a control of the speed of convergence of by the metric d and is linked to the second eigenvalue of P.

Extreme values, range and weak convergence of integrals of Markov chains

1982 ◽  
Vol 19 (02) ◽  
pp. 272-288 ◽  
Author(s):  
P. J. Brockwell ◽  
S. I. Resnick ◽  
N. Pacheco-Santiago
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
State Space ◽  
Extreme Values ◽  
Finite State ◽  
Integral Process ◽  
Maximum Minimum ◽  
Finite State Space

A study is made of the maximum, minimum and range on [0,t] of the integral processwhereSis a finite state-space Markov chain. Approximate results are derived by establishing weak convergence of a sequence of such processes to a Wiener process. For a particular family of two-state stationary Markov chains we show that the corresponding centered integral processes exhibit the Hurst phenomenon to a remarkable degree in their pre-asymptotic behaviour.

Extreme values, range and weak convergence of integrals of Markov chains

1982 ◽  
Vol 19 (2) ◽  
pp. 272-288 ◽  
Author(s):  
P. J. Brockwell ◽  
S. I. Resnick ◽  
N. Pacheco-Santiago
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
State Space ◽  
Extreme Values ◽  
Finite State ◽  
Integral Process ◽  
Maximum Minimum ◽  
Finite State Space

A study is made of the maximum, minimum and range on [0, t] of the integral process where S is a finite state-space Markov chain. Approximate results are derived by establishing weak convergence of a sequence of such processes to a Wiener process. For a particular family of two-state stationary Markov chains we show that the corresponding centered integral processes exhibit the Hurst phenomenon to a remarkable degree in their pre-asymptotic behaviour.

scholarly journals Perturbed Markov chains

2003 ◽  
Vol 40 (1) ◽  
pp. 107-122 ◽  
Author(s):  
Eilon Solan ◽  
Nicolas Vieille
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Matrix Analysis ◽  
Transition Matrices ◽  
Analysis Techniques ◽  
Graph Theoretic ◽  
The Matrix ◽  
Finite State ◽  
Finite State Space

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.

scholarly journals Finite-State-Space Truncations for Infinite Quasi-Birth-Death Processes

2020 ◽  
Vol 2020 ◽  
pp. 1-23
Author(s):  
Hendrik Baumann
Keyword(s):  
Markov Chain ◽  
Lower Bound ◽  
State Space ◽  
Upper Bound ◽  
Speed Of Convergence ◽  
Drift Condition ◽  
Finite State ◽  
Infinite State ◽  
Finite State Space ◽  
Birth Death

For dealing numerically with the infinite-state-space Markov chains, a truncation of the state space is inevitable, that is, an approximation by a finite-state-space Markov chain has to be performed. In this paper, we consider level-dependent quasi-birth-death processes, and we focus on the computation of stationary expectations. In previous literature, efficient methods for computing approximations to these characteristics have been suggested and established. These methods rely on truncating the process at some level N, and for N⟶∞, convergence of the approximation to the desired characteristic is guaranteed. This paper’s main goal is to quantify the speed of convergence. Under the assumption of an f-modulated drift condition, we derive terms for a lower bound and an upper bound on stationary expectations which converge quickly to the same value and which can be efficiently computed.

Perturbed Markov chains

2003 ◽  
Vol 40 (01) ◽  
pp. 107-122 ◽  
Author(s):  
Eilon Solan ◽  
Nicolas Vieille
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Matrix Analysis ◽  
Transition Matrices ◽  
Analysis Techniques ◽  
Graph Theoretic ◽  
The Matrix ◽  
Finite State ◽  
Finite State Space

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.

Asymptotic behaviour of sample weighted circuits representing recurrent Markov chains

1990 ◽  
Vol 27 (03) ◽  
pp. 545-556 ◽  
Author(s):  
S. Kalpazidou
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
Probabilistic Interpretation ◽  
Stationary Markov Chain

The asymptotic behaviour of the sequence (𝒞 n (ω), wc,n (ω)/n), is studied where 𝒞 n (ω) is the class of all cycles c occurring along the trajectory ωof a recurrent strictly stationary Markov chain (ξ n ) until time n and wc,n (ω) is the number of occurrences of the cycle c until time n. The previous sequence of sample weighted classes converges almost surely to a class of directed weighted cycles (𝒞∞, ω c ) which represents uniquely the chain (ξ n ) as a circuit chain, and ω c is given a probabilistic interpretation.

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

An extension of a convergence theorem for Markov chains arising in population genetics

2016 ◽  
Vol 53 (3) ◽  
pp. 953-956 ◽  
Author(s):  
Martin Möhle ◽  
Morihiro Notohara
Keyword(s):  
Population Genetics ◽  
Markov Chains ◽  
Convergence Theorem ◽  
Transition Matrix ◽  
Initial Distribution ◽  
Generator Matrix ◽  
Finite Dimensional ◽  
Finite State ◽  
Initial Distributions ◽  
Finite State Space

AbstractAn extension of a convergence theorem for sequences of Markov chains is derived. For every positive integer N let (XN(r))r be a Markov chain with the same finite state space S and transition matrix ΠN=I+dNBN, where I is the unit matrix, Q a generator matrix, (BN)N a sequence of matrices, limN℩∞cN= limN→∞dN=0 and limN→∞cN∕dN=0. Suppose that the limits P≔limm→∞(I+dNQ)m and G≔limN→∞PBNP exist. If the sequence of initial distributions PXN(0) converges weakly to some probability measure μ, then the finite-dimensional distributions of (XN([t∕cN))t≥0 converge to those of the Markov process (Xt)t≥0 with initial distribution μ, transition matrix PetG and limN→∞(I+dNQ+cNBN)[t∕cN]

scholarly journals Lumpings of Markov Chains, Entropy Rate Preservation, and Higher-Order Lumpability

2014 ◽  
Vol 51 (4) ◽  
pp. 1114-1132 ◽  
Author(s):  
Bernhard C. Geiger ◽  
Christoph Temmel
Keyword(s):  
Growth Rate ◽  
Markov Chain ◽  
Sufficient Conditions ◽  
Entropy Rate ◽  
Transition Graph ◽  
Order Markov Chain ◽  
Finite State ◽  
Random Growth ◽  
Irreducible Markov Chain ◽  
Finite State Space

A lumping of a Markov chain is a coordinatewise projection of the chain. We characterise the entropy rate preservation of a lumping of an aperiodic and irreducible Markov chain on a finite state space by the random growth rate of the cardinality of the realisable preimage of a finite-length trajectory of the lumped chain and by the information needed to reconstruct original trajectories from their lumped images. Both are purely combinatorial criteria, depending only on the transition graph of the Markov chain and the lumping function. A lumping is strongly k-lumpable, if and only if the lumped process is a kth-order Markov chain for each starting distribution of the original Markov chain. We characterise strong k-lumpability via tightness of stationary entropic bounds. In the sparse setting, we give sufficient conditions on the lumping to both preserve the entropy rate and be strongly k-lumpable.

Sign in / Sign up

Export Citation Format

Share Document