Reversibility of Markov chains with applications to storage models

1985 ◽  
Vol 22 (1) ◽  
pp. 123-137 ◽  
Author(s):  
Hideo Ōsawa
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Risk Model ◽  
Sufficient Conditions ◽  
Single Server ◽  
General State ◽  
Insurance Risk ◽  
General State Space ◽  
Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

Reversibility of Markov chains with applications to storage models

1985 ◽  
Vol 22 (01) ◽  
pp. 123-137 ◽  
Author(s):  
Hideo Ōsawa
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Risk Model ◽  
Sufficient Conditions ◽  
Single Server ◽  
General State ◽  
Insurance Risk ◽  
General State Space ◽  
Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

Stability of functionals of perturbed Markov chains under the condition of uniform minorization

2020 ◽  
Vol 28 (4) ◽  
pp. 237-251
Author(s):  
Vitaliy Golomoziy
Keyword(s):  
Markov Chains ◽  
Large Deviations ◽  
State Space ◽  
Discrete Time ◽  
Stability Estimates ◽  
General State ◽  
Whole Space ◽  
Minorization Condition ◽  
General State Space ◽  
The Stability

AbstractIn this paper, we investigate the stability of functionals and trajectories of two different, independent, time-inhomogeneous, discrete-time Markov chains on a general state space. We obtain various stability estimates such as an estimate for a difference in expectations of functionals, {L_{2}} stability, and a probability of large deviations. The key condition that is used is the minorization condition on the whole space. We consider different limitations on the functional and on the proximity of two chains. We use the coupling method as a primary technique in our proofs.

Criteria for classifying general Markov chains

1976 ◽  
Vol 8 (04) ◽  
pp. 737-771 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
General State ◽  
Classical Work ◽  
The Status ◽  
General State Space

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

Quasi-stationary distributions for Markov chains on a general state space

1974 ◽  
Vol 11 (4) ◽  
pp. 726-741 ◽  
Author(s):  
Richard. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Invariant Measure ◽  
Markov Chains ◽  
State Space ◽  
Finiteness Condition ◽  
General State ◽  
Stationary Distributions ◽  
General State Space ◽  
Countably Infinite ◽  
Stationary Behaviour

The quasi-stationary behaviour of a Markov chain which is φ-irreducible when restricted to a subspace of a general state space is investigated. It is shown that previous work on the case where the subspace is finite or countably infinite can be extended to general chains, and the existence of certain quasi-stationary limits as honest distributions is equivalent to the restricted chain being R-positive with the unique R-invariant measure satisfying a certain finiteness condition.

Subgeometric Rates of Convergence of f-Ergodic Markov Chains

1994 ◽  
Vol 26 (03) ◽  
pp. 775-798 ◽  
Author(s):  
Pekka Tuominen ◽  
Richard L. Tweedie
Keyword(s):  
State Space ◽  
Invariant Probability Measure ◽  
Sufficient Conditions ◽  
Rates Of Convergence ◽  
General State ◽  
General State Space ◽  
Drift Conditions ◽  
Image Position ◽  
Regular Sets ◽  
Positive Recurrent

Let Φ = {Φ n } be an aperiodic, positive recurrent Markov chain on a general state space, π its invariant probability measure and f ≧ 1. We consider the rate of (uniform) convergence of E x [g(Φ n )] to the stationary limit π (g) for |g| ≦ f: specifically, we find conditions under which as n →∞, for suitable subgeometric rate functions r. We give sufficient conditions for this convergence to hold in terms of (i) the existence of suitably regular sets, i.e. sets on which (f, r)-modulated hitting time moments are bounded, and (ii) the existence of (f, r)-modulated drift conditions (Foster–Lyapunov conditions). The results are illustrated for random walks and for more general state space models.

Subgeometric Rates of Convergence of f-Ergodic Markov Chains

1994 ◽  
Vol 26 (3) ◽  
pp. 775-798 ◽  
Author(s):  
Pekka Tuominen ◽  
Richard L. Tweedie
Keyword(s):  
State Space ◽  
Invariant Probability Measure ◽  
Sufficient Conditions ◽  
Rates Of Convergence ◽  
General State ◽  
General State Space ◽  
Drift Conditions ◽  
Image Position ◽  
Regular Sets ◽  
Positive Recurrent

Let Φ = {Φ n} be an aperiodic, positive recurrent Markov chain on a general state space, π its invariant probability measure and f ≧ 1. We consider the rate of (uniform) convergence of Ex[g(Φ n)] to the stationary limit π (g) for |g| ≦ f: specifically, we find conditions under which as n →∞, for suitable subgeometric rate functions r. We give sufficient conditions for this convergence to hold in terms of(i) the existence of suitably regular sets, i.e. sets on which (f, r)-modulated hitting time moments are bounded, and(ii) the existence of (f, r)-modulated drift conditions (Foster–Lyapunov conditions).The results are illustrated for random walks and for more general state space models.

scholarly journals Necessary and sufficient conditions for recurrence and transience of Markov chains, in terms of inequalities

1978 ◽  
Vol 15 (4) ◽  
pp. 848-851 ◽  
Author(s):  
Jean-François Mertens ◽  
Ester Samuel-Cahn ◽  
Shmuel Zamir
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Bounded Solution ◽  
Sufficient Conditions ◽  
Necessary And Sufficient Conditions ◽  
Existence Of A Solution ◽  
Necessary And Sufficient ◽  
Irreducible Markov Chain

For an aperiodic, irreducible Markov chain with the non-negative integers as state space it is shown that the existence of a solution to in which yi → ∞is necessary and sufficient for recurrence, and the existence of a bounded solution to the same inequalities, with yk < yo, · · ·, yN–1 for some k ≧ N, is necessary and sufficient for transience.

Sign in / Sign up

Export Citation Format

Share Document