Weak convergence of conditioned processes on a countable state space

1995 ◽  
Vol 32 (4) ◽  
pp. 902-916 ◽  
Author(s):  
S. D. Jacka ◽  
G. O. Roberts
Keyword(s):  
Weak Convergence ◽  
State Space ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Original Process ◽  
Countable State ◽  
Positive Time ◽  
Countably Infinite ◽  
Infinite State

We consider the problem of conditioning a continuous-time Markov chain (on a countably infinite state space) not to hit an absorbing barrier before time T; and the weak convergence of this conditional process as T → ∞. We prove a characterization of convergence in terms of the distribution of the process at some arbitrary positive time, t, introduce a decay parameter for the time to absorption, give an example where weak convergence fails, and give sufficient conditions for weak convergence in terms of the existence of a quasi-stationary limit, and a recurrence property of the original process.

Weak convergence of conditioned processes on a countable state space

1995 ◽  
Vol 32 (04) ◽  
pp. 902-916 ◽  
Author(s):  
S. D. Jacka ◽  
G. O. Roberts
Keyword(s):  
Weak Convergence ◽  
State Space ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Original Process ◽  
Countable State ◽  
Positive Time ◽  
Countably Infinite ◽  
Infinite State

We consider the problem of conditioning a continuous-time Markov chain (on a countably infinite state space) not to hit an absorbing barrier before time T; and the weak convergence of this conditional process as T → ∞. We prove a characterization of convergence in terms of the distribution of the process at some arbitrary positive time, t, introduce a decay parameter for the time to absorption, give an example where weak convergence fails, and give sufficient conditions for weak convergence in terms of the existence of a quasi-stationary limit, and a recurrence property of the original process.

Quasi-ergodicity for non-homogeneous continuous-time Markov chains

1989 ◽  
Vol 26 (3) ◽  
pp. 643-648 ◽  
Author(s):  
A. I. Zeifman
Keyword(s):  
Differential Equation ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

We consider a non-homogeneous continuous-time Markov chain X(t) with countable state space. Definitions of uniform and strong quasi-ergodicity are introduced. The forward Kolmogorov system for X(t) is considered as a differential equation in the space of sequences l1. Sufficient conditions for uniform quasi-ergodicity are deduced from this equation. We consider conditions of uniform and strong ergodicity in the case of proportional intensities.

Quasi-ergodicity for non-homogeneous continuous-time Markov chains

1989 ◽  
Vol 26 (03) ◽  
pp. 643-648 ◽  
Author(s):  
A. I. Zeifman
Keyword(s):  
Differential Equation ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

We consider a non-homogeneous continuous-time Markov chain X(t) with countable state space. Definitions of uniform and strong quasi-ergodicity are introduced. The forward Kolmogorov system for X(t) is considered as a differential equation in the space of sequences l 1 . Sufficient conditions for uniform quasi-ergodicity are deduced from this equation. We consider conditions of uniform and strong ergodicity in the case of proportional intensities.

Markovian couplings staying in arbitrary subsets of the state space

2002 ◽  
Vol 39 (01) ◽  
pp. 197-212 ◽  
Author(s):  
F. Javier López ◽  
Gerardo Sanz
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Transition Rates ◽  
Arbitrary Subset ◽  
State Spaces ◽  
Continuous Time Markov Chains ◽  
Countable State ◽  
Necessary And Sufficient

Let (X t ) and (Y t ) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (X t ) and (Y t ) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.

Markovian couplings staying in arbitrary subsets of the state space

2002 ◽  
Vol 39 (1) ◽  
pp. 197-212 ◽  
Author(s):  
F. Javier López ◽  
Gerardo Sanz
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Transition Rates ◽  
Arbitrary Subset ◽  
State Spaces ◽  
Continuous Time Markov Chains ◽  
Countable State ◽  
Necessary And Sufficient

Let (Xt) and (Yt) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (Xt) and (Yt) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.

scholarly journals Characterization and simplification of optimal strategies in positive stochastic games

2018 ◽  
Vol 55 (3) ◽  
pp. 728-741 ◽  
Author(s):  
János Flesch ◽  
Arkadi Predtetchinski ◽  
William Sudderth
Keyword(s):  
State Space ◽  
Optimal Strategy ◽  
Stochastic Games ◽  
Optimal Strategies ◽  
Action Space ◽  
Positive Zero ◽  
Countable State ◽  
Zero Sum ◽  
Action Spaces

Abstract We consider positive zero-sum stochastic games with countable state and action spaces. For each player, we provide a characterization of those strategies that are optimal in every subgame. These characterizations are used to prove two simplification results. We show that if player 2 has an optimal strategy then he/she also has a stationary optimal strategy, and prove the same for player 1 under the assumption that the state space and player 2's action space are finite.

scholarly journals Efficient Method to Approximately Solve Retrial Systems with Impatience

2012 ◽  
Vol 2012 ◽  
pp. 1-18
Author(s):  
Jose Manuel Gimenez-Guzman ◽  
M. Jose Domenech-Benlloch ◽  
Vicent Pla ◽  
Jorge Martinez-Bauset ◽  
Vicente Casares-Giner
Keyword(s):  
State Space ◽  
Decision Process ◽  
Analytic Solution ◽  
Computational Cost ◽  
Continuous Time Markov Chain ◽  
Kolmogorov Equations ◽  
Novel Technique ◽  
Polynomial Extrapolation ◽  
Markov Decision ◽  
Infinite State

We present a novel technique to solve multiserver retrial systems with impatience. Unfortunately these systems do not present an exact analytic solution, so it is mandatory to resort to approximate techniques. This novel technique does not rely on the numerical solution of the steady-state Kolmogorov equations of the Continuous Time Markov Chain as it is common for this kind of systems but it considers the system in its Markov Decision Process setting. This technique, known as value extrapolation, truncates the infinite state space using a polynomial extrapolation method to approach the states outside the truncated state space. A numerical evaluation is carried out to evaluate this technique and to compare its performance with previous techniques. The obtained results show that value extrapolation greatly outperforms the previous approaches appeared in the literature not only in terms of accuracy but also in terms of computational cost.

Sufficient conditions for regularity, recurrence and ergodicity of Markov processes

1975 ◽  
Vol 78 (1) ◽  
pp. 125-136 ◽  
Author(s):  
Richard L. Tweedie
Keyword(s):  
Markov Process ◽  
Markov Chains ◽  
Markov Processes ◽  
Discrete Time ◽  
Sufficient Conditions ◽  
Process Conditions ◽  
Regular Case ◽  
Competition Process ◽  
Countably Infinite ◽  
Infinite State

AbstractTwo sets of conditions are found on the Q-matrix of an irreducible Markov process on a countably infinite state space which ensure that Q is regular; the first set also implies that the Markov process is ergodic, the second that it is recurrent. If the process is not irreducible, the conditions still imply regularity, and then either non-dissipativity or ‘ultimate recurrence’ (reducible analogues of ergodicity and recurrence) of the process. Conditions sufficient for ergodicity or recurrence of the process in the non-regular and regular case are also given. The results parallel (and use) results for discrete time Markov chains, and the known discrete time recurrence condition is extended to the reducible case. The conditions are illustrated by a competition process example.

scholarly journals On a New Geometric Constant Related to the Euler-Lagrange Type Identity in Banach Spaces

Mathematics ◽  
2021 ◽  
Vol 9 (2) ◽  
pp. 116
Author(s):  
Qi Liu ◽  
Yongjin Li
Keyword(s):  
Banach Spaces ◽  
Product Space ◽  
Sufficient Conditions ◽  
Normal Structure ◽  
The Other ◽  
Inner Product ◽  
Equivalent Characterization ◽  
Geometric Constant ◽  
Geometric Constants

In this paper, we will introduce a new geometric constant LYJ(λ,μ,X) based on an equivalent characterization of inner product space, which was proposed by Moslehian and Rassias. We first discuss some equivalent forms of the proposed constant. Next, a characterization of uniformly non-square is given. Moreover, some sufficient conditions which imply weak normal structure are presented. Finally, we obtain some relationship between the other well-known geometric constants and LYJ(λ,μ,X). Also, this new coefficient is computed for X being concrete space.

Sign in / Sign up

Export Citation Format

Share Document