scholarly journals Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss function

2021 ◽  
Vol 29 (10) ◽  
pp. 15089
Author(s):  
Chun Chen ◽  
Byounghyo Lee ◽  
Nan-Nan Li ◽  
Minseok Chae ◽  
Di Wang ◽  
...  
Keyword(s):  
Loss Function ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Descent Algorithm ◽  
Gradient Descent Algorithm

scholarly journals Stochastic Subgradient for Large-Scale Support Vector Machine Using the Generalized Pinball Loss Function

Symmetry ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1652
Author(s):  
Wanida Panup ◽  
Rabian Wangkeeree
Keyword(s):  
Support Vector Machine ◽  
Loss Function ◽  
Gradient Descent ◽  
Large Scale ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Hinge Loss ◽  
Gradient Descent Algorithm ◽  
Pinball Loss

In this paper, we propose a stochastic gradient descent algorithm, called stochastic gradient descent method-based generalized pinball support vector machine (SG-GPSVM), to solve data classification problems. This approach was developed by replacing the hinge loss function in the conventional support vector machine (SVM) with a generalized pinball loss function. We show that SG-GPSVM is convergent and that it approximates the conventional generalized pinball support vector machine (GPSVM). Further, the symmetric kernel method was adopted to evaluate the performance of SG-GPSVM as a nonlinear classifier. Our suggested algorithm surpasses existing methods in terms of noise insensitivity, resampling stability, and accuracy for large-scale data scenarios, according to the experimental results.

scholarly journals Parameter calibration with stochastic gradient descent for interacting particle systems driven by neural networks

2021 ◽  
Author(s):  
Simone Göttlich ◽  
Claudia Totzeck
Keyword(s):  
Gradient Descent ◽  
Interacting Particle Systems ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Force Model ◽  
Data Sets ◽  
Optimal Controls ◽  
Parameter Calibration ◽  
Descent Algorithm ◽  
Gradient Descent Algorithm

AbstractWe propose a neural network approach to model general interaction dynamics and an adjoint-based stochastic gradient descent algorithm to calibrate its parameters. The parameter calibration problem is considered as optimal control problem that is investigated from a theoretical and numerical point of view. We prove the existence of optimal controls, derive the corresponding first-order optimality system and formulate a stochastic gradient descent algorithm to identify parameters for given data sets. To validate the approach, we use real data sets from traffic and crowd dynamics to fit the parameters. The results are compared to forces corresponding to well-known interaction models such as the Lighthill–Whitham–Richards model for traffic and the social force model for crowd motion.

scholarly journals On the Convergence Properties of a K-step Averaging Stochastic Gradient Descent Algorithm for Nonconvex Optimization

2018 ◽  
Author(s):  
Fan Zhou ◽  
Guojing Cong
Keyword(s):  
Gradient Descent ◽  
Large Scale ◽  
Stochastic Gradient ◽  
Learning Problems ◽  
Stochastic Gradient Descent ◽  
Convergence Properties ◽  
Descent Algorithm ◽  
Convergence Results ◽  
Gradient Descent Algorithm ◽  
Parallel Stochastic Gradient Descent

We adopt and analyze a synchronous K-step averaging stochastic gradient descent algorithm which we call K-AVG  for solving large scale machine learning problems. We establish the convergence results of K-AVG for nonconvex objectives. Our analysis of K-AVG applies to many existing variants of synchronous SGD.  We explain why the K-step delay is necessary and leads to better performance than traditional parallel stochastic gradient descent which is equivalent to K-AVG with $K=1$. We also show that K-AVG scales better with the number of learners than asynchronous stochastic gradient descent (ASGD). Another advantage of K-AVG over ASGD is that it allows larger stepsizes and facilitates faster convergence. On a cluster of $128$ GPUs, K-AVG is faster than ASGD implementations and achieves better accuracies and faster convergence for training with the CIFAR-10 dataset.

Sign in / Sign up

Export Citation Format

Share Document