Some sufficient conditions for non-ergodicity of markov chains

1985 ◽  
Vol 22 (1) ◽  
pp. 138-147 ◽  
Author(s):  
Wojciech Szpankowski
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Lyapunov Functions ◽  
Sufficient Conditions

Some sufficient conditions for non-ergodicity are given for a Markov chain with denumerable state space. These conditions generalize Foster's results, in that unbounded Lyapunov functions are considered. Our criteria directly extend the conditions obtained in Kaplan (1979), in the sense that a class of Lyapunov functions is studied. Applications are presented through some examples; in particular, sufficient conditions for non-ergodicity of a multidimensional Markov chain are given.

Some sufficient conditions for non-ergodicity of markov chains

1985 ◽  
Vol 22 (01) ◽  
pp. 138-147 ◽  
Author(s):  
Wojciech Szpankowski
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Lyapunov Functions ◽  
Sufficient Conditions

Some sufficient conditions for non-ergodicity are given for a Markov chain with denumerable state space. These conditions generalize Foster's results, in that unbounded Lyapunov functions are considered. Our criteria directly extend the conditions obtained in Kaplan (1979), in the sense that a class of Lyapunov functions is studied. Applications are presented through some examples; in particular, sufficient conditions for non-ergodicity of a multidimensional Markov chain are given.

Quasi-ergodicity for non-homogeneous continuous-time Markov chains

1989 ◽  
Vol 26 (3) ◽  
pp. 643-648 ◽  
Author(s):  
A. I. Zeifman
Keyword(s):  
Differential Equation ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

We consider a non-homogeneous continuous-time Markov chain X(t) with countable state space. Definitions of uniform and strong quasi-ergodicity are introduced. The forward Kolmogorov system for X(t) is considered as a differential equation in the space of sequences l1. Sufficient conditions for uniform quasi-ergodicity are deduced from this equation. We consider conditions of uniform and strong ergodicity in the case of proportional intensities.

scholarly journals Necessary and sufficient conditions for recurrence and transience of Markov chains, in terms of inequalities

1978 ◽  
Vol 15 (4) ◽  
pp. 848-851 ◽  
Author(s):  
Jean-François Mertens ◽  
Ester Samuel-Cahn ◽  
Shmuel Zamir
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Bounded Solution ◽  
Sufficient Conditions ◽  
Necessary And Sufficient Conditions ◽  
Existence Of A Solution ◽  
Necessary And Sufficient ◽  
Irreducible Markov Chain

For an aperiodic, irreducible Markov chain with the non-negative integers as state space it is shown that the existence of a solution to in which yi → ∞is necessary and sufficient for recurrence, and the existence of a bounded solution to the same inequalities, with yk < yo, · · ·, yN–1 for some k ≧ N, is necessary and sufficient for transience.

scholarly journals Markov Chains Conditioned Never to Wait Too Long at the Origin

2009 ◽  
Vol 46 (03) ◽  
pp. 812-826
Author(s):  
Saul Jacka
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Sufficient Conditions ◽  
Weak Limit ◽  
The State ◽  
Coin Tossing ◽  
First Time ◽  
Vague Convergence ◽  
Irreducible Markov Chain

Motivated by Feller's coin-tossing problem, we consider the problem of conditioning an irreducible Markov chain never to wait too long at 0. Denoting by τ the first time that the chain,X, waits for at least one unit of time at the origin, we consider conditioning the chain on the event (τ›T). We show that there is a weak limit asT→∞ in the cases where either the state space is finite orXis transient. We give sufficient conditions for the existence of a weak limit in other cases and show that we have vague convergence to a defective limit if the time to hit zero has a lighter tail than τ and τ is subexponential.

Quasi-ergodicity for non-homogeneous continuous-time Markov chains

1989 ◽  
Vol 26 (03) ◽  
pp. 643-648 ◽  
Author(s):  
A. I. Zeifman
Keyword(s):  
Differential Equation ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

We consider a non-homogeneous continuous-time Markov chain X(t) with countable state space. Definitions of uniform and strong quasi-ergodicity are introduced. The forward Kolmogorov system for X(t) is considered as a differential equation in the space of sequences l 1 . Sufficient conditions for uniform quasi-ergodicity are deduced from this equation. We consider conditions of uniform and strong ergodicity in the case of proportional intensities.

scholarly journals Necessary and sufficient conditions for recurrence and transience of Markov chains, in terms of inequalities

1978 ◽  
Vol 15 (04) ◽  
pp. 848-851 ◽  
Author(s):  
Jean-François Mertens ◽  
Ester Samuel-Cahn ◽  
Shmuel Zamir
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Bounded Solution ◽  
Sufficient Conditions ◽  
Necessary And Sufficient Conditions ◽  
Existence Of A Solution ◽  
Necessary And Sufficient ◽  
Irreducible Markov Chain

For an aperiodic, irreducible Markov chain with the non-negative integers as state space it is shown that the existence of a solution to in which yi → ∞is necessary and sufficient for recurrence, and the existence of a bounded solution to the same inequalities, with yk &lt; y o, · · ·, yN –1 for some k ≧ N, is necessary and sufficient for transience.

scholarly journals Markov Chains Conditioned Never to Wait Too Long at the Origin

2009 ◽  
Vol 46 (3) ◽  
pp. 812-826
Author(s):  
Saul Jacka
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Sufficient Conditions ◽  
Weak Limit ◽  
The State ◽  
Coin Tossing ◽  
First Time ◽  
Vague Convergence ◽  
Irreducible Markov Chain

Motivated by Feller's coin-tossing problem, we consider the problem of conditioning an irreducible Markov chain never to wait too long at 0. Denoting by τ the first time that the chain, X, waits for at least one unit of time at the origin, we consider conditioning the chain on the event (τ›T). We show that there is a weak limit as T→∞ in the cases where either the state space is finite or X is transient. We give sufficient conditions for the existence of a weak limit in other cases and show that we have vague convergence to a defective limit if the time to hit zero has a lighter tail than τ and τ is subexponential.

Criteria for classifying general Markov chains

1976 ◽  
Vol 8 (04) ◽  
pp. 737-771 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
General State ◽  
Classical Work ◽  
The Status ◽  
General State Space

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

Reversibility of Markov chains with applications to storage models

1985 ◽  
Vol 22 (01) ◽  
pp. 123-137 ◽  
Author(s):  
Hideo Ōsawa
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Risk Model ◽  
Sufficient Conditions ◽  
Single Server ◽  
General State ◽  
Insurance Risk ◽  
General State Space ◽  
Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

Lumpability and marginalisability for continuous-time Markov chains

1993 ◽  
Vol 30 (3) ◽  
pp. 518-528 ◽  
Author(s):  
Frank Ball ◽  
Geoffrey F. Yeo
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Ion Channel ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Continuous Time Markov Chains ◽  
Probabilistic Proof ◽  
Necessary And Sufficient ◽  
Mutually Independent

We consider lumpability for continuous-time Markov chains and provide a simple probabilistic proof of necessary and sufficient conditions for strong lumpability, valid in circumstances not covered by known theory. We also consider the following marginalisability problem. Let {X{t)} = {(X1(t), X2(t), · ··, Xm(t))} be a continuous-time Markov chain. Under what conditions are the marginal processes {X1(t)}, {X2(t)}, · ··, {Xm(t)} also continuous-time Markov chains? We show that this is related to lumpability and, if no two of the marginal processes can jump simultaneously, then they are continuous-time Markov chains if and only if they are mutually independent. Applications to ion channel modelling and birth–death processes are discussed briefly.

Sign in / Sign up

Export Citation Format

Share Document