Global and Asymptotic Convergence Rate Estimates for a Class of Projected Gradient Processes

1981 ◽  
Vol 19 (3) ◽  
pp. 368-400 ◽  
Author(s):  
J. C. Dunn
Keyword(s):  
Convergence Rate ◽  
Asymptotic Convergence ◽  
Projected Gradient ◽  
Gradient Processes ◽  
Asymptotic Convergence Rate

Asymptotic Convergence Rate of the EM Algorithm for Gaussian Mixtures

2000 ◽  
Vol 12 (12) ◽  
pp. 2881-2907 ◽  
Author(s):  
Jinwen Ma ◽  
Lei Xu ◽  
Michael I. Jordan
Keyword(s):  
Convergence Rate ◽  
Em Algorithm ◽  
Iterative Algorithms ◽  
Asymptotic Convergence ◽  
Gaussian Mixtures ◽  
True Solution ◽  
First Order ◽  
The Em Algorithm ◽  
The Given ◽  
Asymptotic Convergence Rate

It is well known that the convergence rate of the expectation-maximization (EM) algorithm can be faster than those of convention first-order iterative algorithms when the overlap in the given mixture is small. But this argument has not been mathematically proved yet. This article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of Xu and Jordan (1996). It has been proved that the asymptotic convergence rate of the EM algorithm for gaussian mixtures locally around the true solution Θ* is o(e0.5−ε(Θ*)), where ε > 0 is an arbitrarily small number, o(x) means that it is a higher-order infinitesimal as x → 0, and e(Θ*) is a measure of the average overlap of gaussians in the mixture. In other words, the large sample local convergence rate for the EM algorithm tends to be asymptotically superlinear when e(Θ*) tends to zero.

Asymptotic Convergence Properties of the EM Algorithm for Mixture of Experts

2011 ◽  
Vol 23 (8) ◽  
pp. 2140-2168 ◽  
Author(s):  
Yan Yang ◽  
Jinwen Ma
Keyword(s):  
Convergence Rate ◽  
Em Algorithm ◽  
Upper Bound ◽  
Asymptotic Convergence ◽  
Convergence Properties ◽  
Mixture Of Experts ◽  
Approach To Learning ◽  
The Em Algorithm ◽  
Newton Raphson ◽  
Asymptotic Convergence Rate

Mixture of experts (ME) is a modular neural network architecture for supervised classification. The double-loop expectation-maximization (EM) algorithm has been developed for learning the parameters of the ME architecture, and the iteratively reweighted least squares (IRLS) algorithm and the Newton-Raphson algorithm are two popular schemes for learning the parameters in the inner loop or gating network. In this letter, we investigate asymptotic convergence properties of the EM algorithm for ME using either the IRLS or Newton-Raphson approach. With the help of an overlap measure for the ME model, we obtain an upper bound of the asymptotic convergence rate of the EM algorithm in each case. Moreover, we find that for the Newton approach as a specific Newton-Raphson approach to learning the parameters in the inner loop, the upper bound of asymptotic convergence rate of the EM algorithm locally around the true solution Θ* is [Formula: see text], where ϵ>0 is an arbitrarily small number, o(x) means that it is a higher-order infinitesimal as x → 0, and e(Θ*) is a measure of the average overlap of the ME model. That is, as the average overlap of the true ME model with large sample tends to zero, the EM algorithm with the Newton approach to learning the parameters in the inner loop tends to be asymptotically superlinear. Finally, we substantiate our theoretical results by simulation experiments.

Convergence Rates of a Class of Predictor-Corrector Iterations for the Nonsymmetric Algebraic Riccati Equation Arising in Transport Theory

2017 ◽  
Vol 9 (4) ◽  
pp. 944-963 ◽  
Author(s):  
Ning Dong ◽  
Jicheng Jin ◽  
Bo Yu
Keyword(s):  
Riccati Equation ◽  
Numerical Experiments ◽  
Convergence Rates ◽  
Transport Theory ◽  
Algebraic Riccati Equation ◽  
Asymptotic Convergence ◽  
Comparison Theory ◽  
Nonsymmetric Algebraic Riccati Equation ◽  
Predictor Corrector ◽  
Asymptotic Convergence Rate

AbstractIn this paper, we analyse the convergence rates of several different predictor-corrector iterations for computing the minimal positive solution of the nonsymmetric algebraic Riccati equation arising in transport theory. We have shown theoretically that the new predictor-corrector iteration given in [Numer. Linear Algebra Appl., 21 (2014), pp. 761–780] will converge no faster than the simple predictor-corrector iteration and the nonlinear block Jacobi predictor-corrector iteration. Moreover the last two have the same asymptotic convergence rate with the nonlinear block Gauss-Seidel iteration given in [SIAM J. Sci. Comput., 30 (2008), pp. 804–818]. Preliminary numerical experiments have been reported for the validation of the developed comparison theory.

Sign in / Sign up

Export Citation Format

Share Document