Strictly convex loss functions for port-Hamiltonian based optimization algorithm for MTDC networks

2016 ◽  
Author(s):  
Ernest Benedito ◽  
Dunstano del Puerto-Flores ◽  
Arnau Doria-Cerezo ◽  
Olivier van der Feltz ◽  
Jacquelien M.A. Scherpen
Keyword(s):  
Optimization Algorithm ◽  
Loss Functions ◽  
Strictly Convex ◽  
Convex Loss

scholarly journals Differentially Private Empirical Risk Minimization with Smooth Non-Convex Loss Functions: A Non-Stationary View

2019 ◽  
Vol 33 ◽  
pp. 1182-1189
Author(s):  
Di Wang ◽  
Jinhui Xu
Keyword(s):  
Dimensional Space ◽  
Upper Bounds ◽  
Loss Functions ◽  
Log P ◽  
Empirical Risk Minimization ◽  
Risk Minimization ◽  
Projected Gradient ◽  
Empirical Risk ◽  
Convex Loss ◽  
Real World Datasets

In this paper, we study the Differentially Private Empirical Risk Minimization (DP-ERM) problem with non-convex loss functions and give several upper bounds for the utility in different settings. We first consider the problem in low-dimensional space. For DP-ERM with non-smooth regularizer, we generalize an existing work by measuring the utility using ℓ2 norm of the projected gradient. Also, we extend the error bound measurement, for the first time, from empirical risk to population risk by using the expected ℓ2 norm of the gradient. We then investigate the problem in high dimensional space, and show that by measuring the utility with Frank-Wolfe gap, it is possible to bound the utility by the Gaussian Width of the constraint set, instead of the dimensionality p of the underlying space. We further demonstrate that the advantages of this result can be achieved by the measure of ℓ2 norm of the projected gradient. A somewhat surprising discovery is that although the two kinds of measurements are quite different, their induced utility upper bounds are asymptotically the same under some assumptions. We also show that the utility of some special non-convex loss functions can be reduced to a level (i.e., depending only on log p) similar to that of convex loss functions. Finally, we test our proposed algorithms on both synthetic and real world datasets and the experimental results confirm our theoretical analysis.

Robust statistical learning with Lipschitz and convex loss functions

2019 ◽  
Vol 176 (3-4) ◽  
pp. 897-940 ◽  
Author(s):  
Geoffrey Chinot ◽  
Guillaume Lecué ◽  
Matthieu Lerasle
Keyword(s):  
Statistical Learning ◽  
Loss Functions ◽  
Convex Loss

Online pairwise learning algorithms with convex loss functions

2017 ◽  
Vol 406-407 ◽  
pp. 57-70 ◽  
Author(s):  
Junhong Lin ◽  
Yunwen Lei ◽  
Bo Zhang ◽  
Ding-Xuan Zhou
Keyword(s):  
Learning Algorithms ◽  
Loss Functions ◽  
Pairwise Learning ◽  
Convex Loss

scholarly journals Learning With Subquadratic Regularization : A Primal-Dual Approach

2020 ◽  
Author(s):  
Raman Sankaran ◽  
Francis Bach ◽  
Chiranjib Bhattacharyya
Keyword(s):  
Covariance Estimation ◽  
Loss Functions ◽  
Learning Problems ◽  
Structured Sparsity ◽  
Sparse Models ◽  
Dual Approach ◽  
Primal Dual ◽  
Computational Aspects ◽  
Convex Loss ◽  
General Setup

Subquadratic norms have been studied recently in the context of structured sparsity, which has been shown to be more beneficial than conventional regularizers in applications such as image denoising, compressed sensing, banded covariance estimation, etc. While existing works have been successful in learning structured sparse models such as trees, graphs, their associated optimization procedures have been inefficient because of hard-to-evaluate proximal operators of the norms. In this paper, we study the computational aspects of learning with subquadratic norms in a general setup. Our main contributions are two proximal-operator based algorithms ADMM-η and CP-η, which generically apply to these learning problems with convex loss functions, and achieve a proven rate of convergence of O(1/T) after T iterations. These algorithms are derived in a primal-dual framework, which have not been examined for subquadratic norms. We illustrate the efficiency of the algorithms developed in the context of tree-structured sparsity, where they comprehensively outperform relevant baselines.

Sign in / Sign up

Export Citation Format

Share Document