scholarly journals Perturbed Markov chains

2003 ◽  
Vol 40 (1) ◽  
pp. 107-122 ◽  
Author(s):  
Eilon Solan ◽  
Nicolas Vieille
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Matrix Analysis ◽  
Transition Matrices ◽  
Analysis Techniques ◽  
Graph Theoretic ◽  
The Matrix ◽  
Finite State ◽  
Finite State Space

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.

Perturbed Markov chains

2003 ◽  
Vol 40 (01) ◽  
pp. 107-122 ◽  
Author(s):  
Eilon Solan ◽  
Nicolas Vieille
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Transition Matrix ◽  
Matrix Analysis ◽  
Transition Matrices ◽  
Analysis Techniques ◽  
Graph Theoretic ◽  
The Matrix ◽  
Finite State ◽  
Finite State Space

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

An extension of a convergence theorem for Markov chains arising in population genetics

2016 ◽  
Vol 53 (3) ◽  
pp. 953-956 ◽  
Author(s):  
Martin Möhle ◽  
Morihiro Notohara
Keyword(s):  
Population Genetics ◽  
Markov Chains ◽  
Convergence Theorem ◽  
Transition Matrix ◽  
Initial Distribution ◽  
Generator Matrix ◽  
Finite Dimensional ◽  
Finite State ◽  
Initial Distributions ◽  
Finite State Space

AbstractAn extension of a convergence theorem for sequences of Markov chains is derived. For every positive integer N let (XN(r))r be a Markov chain with the same finite state space S and transition matrix ΠN=I+dNBN, where I is the unit matrix, Q a generator matrix, (BN)N a sequence of matrices, limN℩∞cN= limN→∞dN=0 and limN→∞cN∕dN=0. Suppose that the limits P≔limm→∞(I+dNQ)m and G≔limN→∞PBNP exist. If the sequence of initial distributions PXN(0) converges weakly to some probability measure μ, then the finite-dimensional distributions of (XN([t∕cN))t≥0 converge to those of the Markov process (Xt)t≥0 with initial distribution μ, transition matrix PetG and limN→∞(I+dNQ+cNBN)[t∕cN]

scholarly journals Similar States in Continuous-Time Markov Chains

2009 ◽  
Vol 46 (02) ◽  
pp. 497-506 ◽  
Author(s):  
V. B. Yap
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Matrix ◽  
Base Substitution ◽  
Continuous Time Markov Chain ◽  
Rate Matrix ◽  
Dna Base ◽  
Substitution Models ◽  
Finite State ◽  
Finite State Space

In a homogeneous continuous-time Markov chain on a finite state space, two states that jump to every other state with the same rate are called similar. By partitioning states into similarity classes, the algebraic derivation of the transition matrix can be simplified, using hidden holding times and lumped Markov chains. When the rate matrix is reversible, the transition matrix is explicitly related in an intuitive way to that of the lumped chain. The theory provides a unified derivation for a whole range of useful DNA base substitution models, and a number of amino acid substitution models.

scholarly journals Similar States in Continuous-Time Markov Chains

2009 ◽  
Vol 46 (2) ◽  
pp. 497-506 ◽  
Author(s):  
V. B. Yap
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Matrix ◽  
Base Substitution ◽  
Continuous Time Markov Chain ◽  
Rate Matrix ◽  
Dna Base ◽  
Substitution Models ◽  
Finite State ◽  
Finite State Space

In a homogeneous continuous-time Markov chain on a finite state space, two states that jump to every other state with the same rate are called similar. By partitioning states into similarity classes, the algebraic derivation of the transition matrix can be simplified, using hidden holding times and lumped Markov chains. When the rate matrix is reversible, the transition matrix is explicitly related in an intuitive way to that of the lumped chain. The theory provides a unified derivation for a whole range of useful DNA base substitution models, and a number of amino acid substitution models.

Coefficients of ergodicity for stochastically monotone Markov chains

1992 ◽  
Vol 29 (4) ◽  
pp. 850-860 ◽  
Author(s):  
G. Ch. Pflug ◽  
W. Schachermayer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Probabilistic Interpretation ◽  
Speed Of Convergence ◽  
Transition Matrices ◽  
Wasserstein Metric ◽  
Ergodic Markov Chain ◽  
Finite State ◽  
Second Eigenvalue ◽  
Finite State Space

In this paper we show that to each distance d defined on the finite state space S of a strongly ergodic Markov chain there corresponds a coefficient ρd of ergodicity based on the Wasserstein metric. For a class of stochastically monotone transition matrices P, the infimum over all such coefficients is given by the spectral radius of P – R, where R = limkPk and is attained. This result has a probabilistic interpretation of a control of the speed of convergence of by the metric d and is linked to the second eigenvalue of P.

scholarly journals Imprecise Discrete-Time Markov Chains

2021 ◽  
pp. 51-65
Author(s):  
Gert de Cooman
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Finite State ◽  
Finite State Space

AbstractI present a short and easy introduction to a number of basic definitions and important results from the theory of imprecise Markov chains in discrete time, with a finite state space. The approach is intuitive and graphical.

Coefficients of ergodicity for stochastically monotone Markov chains

1992 ◽  
Vol 29 (04) ◽  
pp. 850-860 ◽  
Author(s):  
G. Ch. Pflug ◽  
W. Schachermayer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Probabilistic Interpretation ◽  
Speed Of Convergence ◽  
Transition Matrices ◽  
Wasserstein Metric ◽  
Ergodic Markov Chain ◽  
Finite State ◽  
Second Eigenvalue ◽  
Finite State Space

In this paper we show that to each distance d defined on the finite state space S of a strongly ergodic Markov chain there corresponds a coefficient ρd of ergodicity based on the Wasserstein metric. For a class of stochastically monotone transition matrices P, the infimum over all such coefficients is given by the spectral radius of P – R, where R = lim k Pk and is attained. This result has a probabilistic interpretation of a control of the speed of convergence of by the metric d and is linked to the second eigenvalue of P.

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (01) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

scholarly journals Duality and Asymptotics for a Class of Nonneutral Discrete Moran Models

2009 ◽  
Vol 46 (3) ◽  
pp. 866-893 ◽  
Author(s):  
Thierry Huillet ◽  
Martin Möhle
Keyword(s):  
Transition Matrix ◽  
Random Number ◽  
Dual Process ◽  
Probabilistic Interpretation ◽  
Uhlenbeck Process ◽  
Moran Model ◽  
Completely Monotone ◽  
Finite State ◽  
Selection Mechanisms ◽  
Finite State Space

A Markov chain X with finite state space {0,…,N} and tridiagonal transition matrix is considered, where transitions from i to i-1 occur with probability (i/N)(1-p(i/N)) and transitions from i to i+1 occur with probability (1-i/N)p(i/N). Here p:[0,1]→[0,1] is a given function. It is shown that if p is continuous with p(x)≤p(1) for all x∈[0,1] then, for each N, a dual process Y to X (with respect to a specific duality function) exists if and only if 1-p is completely monotone with p(0)=0. A probabilistic interpretation of Y in terms of an ancestral process of a mixed multitype Moran model with a random number of types is presented. It is shown that, under weak conditions on p, the process Y, properly time and space scaled, converges to an Ornstein–Uhlenbeck process as N tends to ∞. The asymptotics of the stationary distribution of Y is studied as N tends to ∞. Examples are presented involving selection mechanisms. results.

Sign in / Sign up

Export Citation Format

Share Document