Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General Processes

1993 ◽  
Vol 31 (4) ◽  
pp. 1045-1062 ◽  
Author(s):  
Harold J. Kushner ◽  
Jichuan Yang
Keyword(s):  
Rate Of Convergence ◽  
Stochastic Approximation ◽  
Asymptotic Rate

scholarly journals Fast, Asymptotically Efficient, Recursive Estimation in a Riemannian Manifold

Entropy ◽  
2019 ◽  
Vol 21 (10) ◽  
pp. 1021 ◽  
Author(s):  
Jialun Zhou ◽  
Salem Said
Keyword(s):  
Riemannian Manifold ◽  
Rate Of Convergence ◽  
Statistical Parameter ◽  
Recursive Estimation ◽  
Numerical Application ◽  
Stochastic Optimisation ◽  
Asymptotic Rate ◽  
Fisher Information Metric ◽  
Convexity Assumptions ◽  
Asymptotically Efficient

Stochastic optimisation in Riemannian manifolds, especially the Riemannian stochastic gradient method, has attracted much recent attention. The present work applies stochastic optimisation to the task of recursive estimation of a statistical parameter which belongs to a Riemannian manifold. Roughly, this task amounts to stochastic minimisation of a statistical divergence function. The following problem is considered: how to obtain fast, asymptotically efficient, recursive estimates, using a Riemannian stochastic optimisation algorithm with decreasing step sizes. In solving this problem, several original results are introduced. First, without any convexity assumptions on the divergence function, we proved that, with an adequate choice of step sizes, the algorithm computes recursive estimates which achieve a fast non-asymptotic rate of convergence. Second, the asymptotic normality of these recursive estimates is proved by employing a novel linearisation technique. Third, it is proved that, when the Fisher information metric is used to guide the algorithm, these recursive estimates achieve an optimal asymptotic rate of convergence, in the sense that they become asymptotically efficient. These results, while relatively familiar in the Euclidean context, are here formulated and proved for the first time in the Riemannian context. In addition, they are illustrated with a numerical application to the recursive estimation of elliptically contoured distributions.

Sign in / Sign up

Export Citation Format

Share Document