scholarly journals Online Primal-Dual Mirror Descent under Stochastic Constraints

2020 ◽  
Vol 4 (2) ◽  
pp. 1-36
Author(s):  
Xiaohan Wei ◽  
Hao Yu ◽  
Michael J. Neely
Keyword(s):  
Mirror Descent ◽  
Primal Dual ◽  
Stochastic Constraints

scholarly journals Online Primal-Dual Mirror Descent under Stochastic Constraints

2020 ◽  
Vol 48 (1) ◽  
pp. 3-4
Author(s):  
Xiaohan Wei ◽  
Hao Yu ◽  
Michael J. Neely
Keyword(s):  
Mirror Descent ◽  
Primal Dual ◽  
Stochastic Constraints

Primal–Dual Mirror Descent Method for Constraint Stochastic Optimization Problems

2018 ◽  
Vol 58 (11) ◽  
pp. 1728-1736 ◽  
Author(s):  
A. S. Bayandina ◽  
A. V. Gasnikov ◽  
E. V. Gasnikova ◽  
S. V. Matsievskii
Keyword(s):  
Stochastic Optimization ◽  
Optimization Problems ◽  
Descent Method ◽  
Mirror Descent Method ◽  
Mirror Descent ◽  
Primal Dual

scholarly journals Efficient Adaptive Online Learning via Frequent Directions

2018 ◽  
Author(s):  
Yuanyu Wan ◽  
Nan Wei ◽  
Lijun Zhang
Keyword(s):  
Online Learning ◽  
Large Scale ◽  
Descent Method ◽  
Low Rank ◽  
Subgradient Methods ◽  
Outer Product ◽  
Mirror Descent Method ◽  
Mirror Descent ◽  
Primal Dual ◽  
Frequent Directions

By employing time-varying proximal functions, adaptive subgradient methods (ADAGRAD) have improved the regret bound and been widely used in online learning and optimization. However, ADAGRAD with full matrix proximal functions (ADA-FULL) cannot deal with large-scale problems due to the impractical time and space complexities, though it has better performance when gradients are correlated. In this paper, we propose ADA-FD, an efficient variant of ADA-FULL based on a deterministic matrix sketching technique called frequent directions. Following ADA-FULL, we incorporate our ADA-FD into both primal-dual subgradient method and composite mirror descent method to develop two efficient methods. By maintaining and manipulating low-rank matrices, at each iteration, the space complexity is reduced from $O(d^2)$ to $O(\tau d)$ and the time complexity is reduced from $O(d^3)$ to $O(\tau^2d)$, where $d$ is the dimensionality of the data and $\tau \ll d$ is the sketching size. Theoretical analysis reveals that the regret of our methods is close to that of ADA-FULL as long as the outer product matrix of gradients is approximately low-rank. Experimental results show that our ADA-FD is comparable to ADA-FULL and outperforms other state-of-the-art algorithms in online convex optimization as well as in training convolutional neural networks (CNN).

Routing Optimization with Efficient Second Order Distributed Approach Using Congestion Control Rules

2017 ◽  
Vol 7 (7) ◽  
pp. 363
Author(s):  
Jaya Pratha Sebastiyar ◽  
Martin Sahayaraj Joseph
Keyword(s):  
Congestion Control ◽  
Second Order ◽  
Back Pressure ◽  
Order Method ◽  
Delay Performance ◽  
Routing Optimization ◽  
Distributed Approach ◽  
Control Rules ◽  
Primal Dual ◽  
Queue Stability

Distributed joint congestion control and routing optimization has received a significant amount of attention recently. To date, however, most of the existing schemes follow a key idea called the back-pressure algorithm. Despite having many salient features, the first-order sub gradient nature of the back-pressure based schemes results in slow convergence and poor delay performance. To overcome these limitations, the present study was made as first attempt at developing a second-order joint congestion control and routing optimization framework that offers utility-optimality, queue-stability, fast convergence, and low delay.  Contributions in this project are three-fold. The present study propose a new second-order joint congestion control and routing framework based on a primal-dual interior-point approach and established utility-optimality and queue-stability of the proposed second-order method. The results of present study showed that how to implement the proposed second-order method in a distributed fashion.

Sign in / Sign up

Export Citation Format

Share Document