scholarly journals On Stochastic Subgradient Mirror-Descent Algorithm with Weighted Averaging

2014 ◽  
Vol 24 (1) ◽  
pp. 84-107 ◽  
Author(s):  
Angelia Nedić ◽  
Soomin Lee
Keyword(s):  
Weighted Averaging ◽  
Descent Algorithm ◽  
Mirror Descent

scholarly journals Recursive Aggregation of Estimators by the Mirror Descent Algorithm with Averaging

2005 ◽  
Vol 41 (4) ◽  
pp. 368-384 ◽  
Author(s):  
A. B. Juditsky ◽  
A. V. Nazin ◽  
A. B. Tsybakov ◽  
N. Vayatis
Keyword(s):  
Descent Algorithm ◽  
Mirror Descent

A mirror descent algorithm for minimization of mean Poisson flow driven losses

2014 ◽  
Vol 75 (6) ◽  
pp. 1010-1016
Author(s):  
A. V. Nazin ◽  
S. V. Anulova ◽  
A. A. Tremba
Keyword(s):  
Descent Algorithm ◽  
Mirror Descent

Analysis of Online Composite Mirror Descent Algorithm

2017 ◽  
Vol 29 (3) ◽  
pp. 825-860 ◽  
Author(s):  
Yunwen Lei ◽  
Ding-Xuan Zhou
Keyword(s):  
Error Analysis ◽  
Objective Function ◽  
Convergence Rates ◽  
Hölder Continuous ◽  
Descent Algorithm ◽  
Strongly Convex ◽  
Mirror Descent ◽  
One Step ◽  
Error Decomposition ◽  
Mirror Map

We study the convergence of the online composite mirror descent algorithm, which involves a mirror map to reflect the geometry of the data and a convex objective function consisting of a loss and a regularizer possibly inducing sparsity. Our error analysis provides convergence rates in terms of properties of the strongly convex differentiable mirror map and the objective function. For a class of objective functions with Hölder continuous gradients, the convergence rates of the excess (regularized) risk under polynomially decaying step sizes have the order [Formula: see text] after [Formula: see text] iterates. Our results improve the existing error analysis for the online composite mirror descent algorithm by avoiding averaging and removing boundedness assumptions, and they sharpen the existing convergence rates of the last iterate for online gradient descent without any boundedness assumptions. Our methodology mainly depends on a novel error decomposition in terms of an excess Bregman distance, refined analysis of self-bounding properties of the objective function, and the resulting one-step progress bounds.

Sign in / Sign up

Export Citation Format

Share Document