scholarly journals A Fast Newton-Shamanskii Iteration for a Matrix Equation Arising from M/G/1-Type Markov Chains

2017 ◽  
Vol 2017 ◽  
pp. 1-8
Author(s):  
Pei-Chang Guo
Keyword(s):  
Markov Chains ◽  
Nonnegative Solution ◽  
Initial Guess ◽  
Monotone Convergence ◽  
Nonnegative Matrices ◽  
Schur Decomposition ◽  
Minimal Nonnegative Solution ◽  
Convergence Results ◽  
The Minimal Nonnegative Solution ◽  
Nonlinear Matrix

For the nonlinear matrix equations arising in the analysis of M/G/1-type and GI/M/1-type Markov chains, the minimal nonnegative solution G or R can be found by Newton-like methods. We prove monotone convergence results for the Newton-Shamanskii iteration for this class of equations. Starting with zero initial guess or some other suitable initial guess, the Newton-Shamanskii iteration provides a monotonically increasing sequence of nonnegative matrices converging to the minimal nonnegative solution. A Schur decomposition method is used to accelerate the Newton-Shamanskii iteration. Numerical examples illustrate the effectiveness of the Newton-Shamanskii iteration.

Newton-Shamanskii Method for a Quadratic Matrix Equation Arising in Quasi-Birth-Death Problems

2014 ◽  
Vol 4 (4) ◽  
pp. 386-395
Author(s):  
Pei-Chang Guo
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Matrix Equation ◽  
Nonnegative Solution ◽  
Minimal Nonnegative Solution ◽  
The Minimal Nonnegative Solution ◽  
Quadratic Matrix Equation ◽  
Quadratic Matrix ◽  
Birth Death

AbstractIn order to determine the stationary distribution for discrete time quasi-birth-death Markov chains, it is necessary to find the minimal nonnegative solution of a quadratic matrix equation. The Newton-Shamanskii method is applied to solve this equation, and the sequence of matrices produced is monotonically increasing and converges to its minimal nonnegative solution. Numerical results illustrate the effectiveness of this procedure.

On the convergence and limits of certain matrix sequences arising in quasi-birth-and-death Markov chains

2001 ◽  
Vol 38 (2) ◽  
pp. 519-541 ◽  
Author(s):  
Qi-Ming He ◽  
Marcel F. Neuts
Keyword(s):  
Markov Chains ◽  
Nonnegative Solution ◽  
Nonlinear Matrix Equation ◽  
The Matrix ◽  
Boundary States ◽  
The Minimal Nonnegative Solution ◽  
Stochastic Solution ◽  
Matrix Geometric Solution ◽  
Technical Issues ◽  
Nonlinear Matrix

We study the convergence of certain matrix sequences that arise in quasi-birth-and-death (QBD) Markov chains and we identify their limits. In particular, we focus on a sequence of matrices whose elements are absorption probabilities into some boundary states of the QBD. We prove that, under certain technical conditions, that sequence converges. Its limit is either the minimal nonnegative solution G of the standard nonlinear matrix equation, or it is a stochastic solution that can be explicitly expressed in terms of G. Similar results are obtained relative to the standard matrix R that arises in the matrix-geometric solution of the QBD. We present numerical examples that clarify some of the technical issues of interest.

On the convergence and limits of certain matrix sequences arising in quasi-birth-and-death Markov chains

2001 ◽  
Vol 38 (02) ◽  
pp. 519-541 ◽  
Author(s):  
Qi-Ming He ◽  
Marcel F. Neuts
Keyword(s):  
Markov Chains ◽  
Nonnegative Solution ◽  
Nonlinear Matrix Equation ◽  
The Matrix ◽  
Boundary States ◽  
The Minimal Nonnegative Solution ◽  
Stochastic Solution ◽  
Matrix Geometric Solution ◽  
Technical Issues ◽  
Nonlinear Matrix

We study the convergence of certain matrix sequences that arise in quasi-birth-and-death (QBD) Markov chains and we identify their limits. In particular, we focus on a sequence of matrices whose elements are absorption probabilities into some boundary states of the QBD. We prove that, under certain technical conditions, that sequence converges. Its limit is either the minimal nonnegative solution G of the standard nonlinear matrix equation, or it is a stochastic solution that can be explicitly expressed in terms of G. Similar results are obtained relative to the standard matrix R that arises in the matrix-geometric solution of the QBD. We present numerical examples that clarify some of the technical issues of interest.

scholarly journals Additive Functionals for Discrete-Time Markov Chains with Applications to Birth-Death Processes

2011 ◽  
Vol 48 (04) ◽  
pp. 925-937 ◽  
Author(s):  
Yuanyuan Liu
Keyword(s):  
Markov Chains ◽  
Central Limit ◽  
Discrete Time ◽  
Nonnegative Solution ◽  
Return Time ◽  
Additive Functionals ◽  
The Minimal Nonnegative Solution ◽  
First Return Time ◽  
Do So ◽  
Birth Death

In this paper we are interested in bounding or calculating the additive functionals of the first return time on a set for discrete-time Markov chains on a countable state space, which is motivated by investigating ergodic theory and central limit theorems. To do so, we introduce the theory of the minimal nonnegative solution. This theory combined with some other techniques is proved useful for investigating the additive functionals. This method is used to study the functionals for discrete-time birth-death processes, and the polynomial convergence and a central limit theorem are derived.

scholarly journals Additive Functionals for Discrete-Time Markov Chains with Applications to Birth-Death Processes

2011 ◽  
Vol 48 (4) ◽  
pp. 925-937 ◽  
Author(s):  
Yuanyuan Liu
Keyword(s):  
Markov Chains ◽  
Central Limit ◽  
Discrete Time ◽  
Nonnegative Solution ◽  
Return Time ◽  
Additive Functionals ◽  
The Minimal Nonnegative Solution ◽  
First Return Time ◽  
Do So ◽  
Birth Death

In this paper we are interested in bounding or calculating the additive functionals of the first return time on a set for discrete-time Markov chains on a countable state space, which is motivated by investigating ergodic theory and central limit theorems. To do so, we introduce the theory of the minimal nonnegative solution. This theory combined with some other techniques is proved useful for investigating the additive functionals. This method is used to study the functionals for discrete-time birth-death processes, and the polynomial convergence and a central limit theorem are derived.

Convergence of simulated annealing using Foster-Lyapunov criteria

2001 ◽  
Vol 38 (04) ◽  
pp. 975-994 ◽  
Author(s):  
Christophe Andrieu ◽  
Laird A. Breyer ◽  
Arnaud Doucet
Keyword(s):  
Simulated Annealing ◽  
Markov Chains ◽  
State Spaces ◽  
Compact Spaces ◽  
Convergence Results

Simulated annealing is a popular and much studied method for maximizing functions on finite or compact spaces. For noncompact state spaces, the method is still sound, but convergence results are scarce. We show here how to prove convergence in such cases, for Markov chains satisfying suitable drift and minorization conditions.

scholarly journals Using systematic sampling selection for Monte Carlo solutions of Feynman-Kac equations

2008 ◽  
Vol 40 (2) ◽  
pp. 454-472 ◽  
Author(s):  
Ivan Gentil ◽  
Bruno Rémillard
Keyword(s):  
Monte Carlo ◽  
Markov Chains ◽  
Selection Method ◽  
Systematic Sampling ◽  
Selection Methods ◽  
Convergence Properties ◽  
Main Motivation ◽  
Empirical Basis ◽  
Convergence Results ◽  
Selection For

While the convergence properties of many sampling selection methods can be proven, there is one particular sampling selection method introduced in Baker (1987), closely related to ‘systematic sampling’ in statistics, that has been exclusively treated on an empirical basis. The main motivation of the paper is to start to study formally its convergence properties, since in practice it is by far the fastest selection method available. We will show that convergence results for the systematic sampling selection method are related to properties of peculiar Markov chains.

scholarly journals Extinction Probabilities of Supercritical Decomposable Branching Processes

2012 ◽  
Vol 49 (03) ◽  
pp. 639-651 ◽  
Author(s):  
Sophie Hautphenne
Keyword(s):  
Nonnegative Solution ◽  
Branching Processes ◽  
Sufficient Conditions ◽  
Equivalence Classes ◽  
Specific Class ◽  
Initial Particle ◽  
Whole Process ◽  
Extinction Probabilities ◽  
The Minimal Nonnegative Solution ◽  
Multitype Branching Processes

We focus on supercritical decomposable (reducible) multitype branching processes. Types are partitioned into irreducible equivalence classes. In this context, extinction of some classes is possible without the whole process becoming extinct. We derive criteria for the almost-sure extinction of the whole process, as well as of a specific class, conditionally given the class of the initial particle. We give sufficient conditions under which the extinction of a class implies the extinction of another class or of the whole process. Finally, we show that the extinction probability of a specific class is the minimal nonnegative solution of the usual extinction equation but with added constraints.

Sign in / Sign up

Export Citation Format

Share Document