scholarly journals Stochastic Subgradient for Large-Scale Support Vector Machine Using the Generalized Pinball Loss Function

Symmetry ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1652
Author(s):  
Wanida Panup ◽  
Rabian Wangkeeree
Keyword(s):  
Support Vector Machine ◽  
Loss Function ◽  
Gradient Descent ◽  
Large Scale ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Hinge Loss ◽  
Gradient Descent Algorithm ◽  
Pinball Loss

In this paper, we propose a stochastic gradient descent algorithm, called stochastic gradient descent method-based generalized pinball support vector machine (SG-GPSVM), to solve data classification problems. This approach was developed by replacing the hinge loss function in the conventional support vector machine (SVM) with a generalized pinball loss function. We show that SG-GPSVM is convergent and that it approximates the conventional generalized pinball support vector machine (GPSVM). Further, the symmetric kernel method was adopted to evaluate the performance of SG-GPSVM as a nonlinear classifier. Our suggested algorithm surpasses existing methods in terms of noise insensitivity, resampling stability, and accuracy for large-scale data scenarios, according to the experimental results.

scholarly journals On the Convergence Properties of a K-step Averaging Stochastic Gradient Descent Algorithm for Nonconvex Optimization

2018 ◽  
Author(s):  
Fan Zhou ◽  
Guojing Cong
Keyword(s):  
Gradient Descent ◽  
Large Scale ◽  
Stochastic Gradient ◽  
Learning Problems ◽  
Stochastic Gradient Descent ◽  
Convergence Properties ◽  
Descent Algorithm ◽  
Convergence Results ◽  
Gradient Descent Algorithm ◽  
Parallel Stochastic Gradient Descent

We adopt and analyze a synchronous K-step averaging stochastic gradient descent algorithm which we call K-AVG  for solving large scale machine learning problems. We establish the convergence results of K-AVG for nonconvex objectives. Our analysis of K-AVG applies to many existing variants of synchronous SGD.  We explain why the K-step delay is necessary and leads to better performance than traditional parallel stochastic gradient descent which is equivalent to K-AVG with $K=1$. We also show that K-AVG scales better with the number of learners than asynchronous stochastic gradient descent (ASGD). Another advantage of K-AVG over ASGD is that it allows larger stepsizes and facilitates faster convergence. On a cluster of $128$ GPUs, K-AVG is faster than ASGD implementations and achieves better accuracies and faster convergence for training with the CIFAR-10 dataset.

scholarly journals Perbandingan Teknik Klasifikasi Dalam Data Mining Untuk Bank Direct Marketing

2018 ◽  
Vol 5 (5) ◽  
pp. 567 ◽  
Author(s):  
Irvi Oktanisa ◽  
Ahmad Afif Supianto
Keyword(s):  
Data Mining ◽  
Support Vector Machine ◽  
Random Forest ◽  
Gradient Descent ◽  
Naive Bayes ◽  
Direct Marketing ◽  
Naïve Bayes ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Support Vector

<p class="Abstrak">Klasifikasi merupakan teknik dalam <em>data mining</em> untuk mengelompokkan data berdasarkan keterikatan data terhadap  data sampel. Pada penelitian ini, kami melakukan perbandingan 9 teknik klasifikasi untuk mengklasifikasi respon pelanggan pada <em>dataset Bank Direct Marketing</em>. Perbandingan teknik klasifikasi ini dilakukan untuk mengetahui model dalam teknik klasfikasi yang paling efektif untuk mengklasifikasi target pada <em>dataset Bank Direct Marketing</em>. Teknik klasifikasi yang digunakan yaitu <em>Support Vector Machine</em>, <em>AdaBoost</em>, <em>Naïve Bayes</em>, <em>Constant, KNN, Tree, Random Forest, Stochastic Gradient Descent</em>, dan <em>CN2 Rule</em>. Proses klasifikasi diawali dengan <em>preprocessing</em> data untuk melakukan penghilangan <em>missing value</em> dan pemilihan fitur pada <em>dataset</em>. Pada tahap evaluasi digunakan teknik <em>10 fold cross validation</em>. Setelah dilakukan pengujian, didapatkan bahwa hasil klasifikasi menunjukkan akurasi terbaik diperoleh oleh model <em>Tree, Constant</em>, <em>Naive Bayes</em>, dan <em>Stochastic Gardient Descent</em>. Kemudian diikuti oleh model <em>Random Forest</em>, <em>K-Nearest Neighbor</em>, <em>CN-2 Rule</em>, <em>AdaBoost</em> dan <em>Support Vector Machine</em>. Dari keempat model yang menunjukkan hasil akurasi terbaik, untuk kasus ini <em>Stochastic Gradient Descent</em> terpilih sebagai model yang memiliki akurasi terbaik dengan nilai akurasi sebesar 0,972 dan hasil visualisasi yang dihasilkan lebih jelas untuk mengklasifikasi target pada <em>dataset Bank Direct Marketing</em>.</p><p class="Abstrak"><em><strong><br /></strong></em></p><p class="Abstrak"><em><strong>Abstract</strong></em></p>Classification is a technique in data mining to classify data based on the attachment of data to the sample data.. In this paper, we present the comparison of  9 classification techniques performed to classify customer response on the dataset of Bank Direct Marketing. The techniques performed to find out the effectiveness model in the classification technique used to classify targets on the dataset of Bank Direct Marketing. The techniques used are Support Vector Machine, AdaBoost, Naïve Bayes, Constant, KNN, Tree, Random Forest, Stochastic Gradient Descent, and CN2 Rule. The classification process begins with preprocessing data to perform missing value omissions and feature selection on the dataset. Cross validation technique, with k value is 10, used in the evaluation stage. After testing, it was found that the classification results showed the best accuracy obtained when using the Tree model, Constant, Naive Bayes and Stochastic Gradient Descent. Afterwards the Random Forest model, K-Nearest Neighbor, CN-2 Rule, AdaBoost, and Support Vector Machine are followed. Of the four models with the high accuracy results, in this case Stochastic Gradient Descent was selected as the best accuracy model with an accuracy value of 0.972 and resulting visualization more clearly to classify targets on the dataset of Bank Direct Marketing.

scholarly journals Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss function

2021 ◽  
Vol 29 (10) ◽  
pp. 15089
Author(s):  
Chun Chen ◽  
Byounghyo Lee ◽  
Nan-Nan Li ◽  
Minseok Chae ◽  
Di Wang ◽  
...  
Keyword(s):  
Loss Function ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Descent Algorithm ◽  
Gradient Descent Algorithm

Pinball Loss Twin Support Vector Clustering

2021 ◽  
Vol 17 (2s) ◽  
pp. 1-23
Author(s):  
M. Tanveer ◽  
Tarun Gupta ◽  
Miten Shah ◽  
Keyword(s):  
Loss Function ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Structural Mri ◽  
Twin Support Vector Machine ◽  
Support Vector ◽  
Support Vector Clustering ◽  
Hinge Loss ◽  
Pinball Loss ◽  
Vector Clustering

Twin Support Vector Clustering (TWSVC) is a clustering algorithm inspired by the principles of Twin Support Vector Machine (TWSVM). TWSVC has already outperformed other traditional plane based clustering algorithms. However, TWSVC uses hinge loss, which maximizes shortest distance between clusters and hence suffers from noise-sensitivity and low re-sampling stability. In this article, we propose Pinball loss Twin Support Vector Clustering (pinTSVC) as a clustering algorithm. The proposed pinTSVC model incorporates the pinball loss function in the plane clustering formulation. Pinball loss function introduces favorable properties such as noise-insensitivity and re-sampling stability. The time complexity of the proposed pinTSVC remains equivalent to that of TWSVC. Extensive numerical experiments on noise-corrupted benchmark UCI and artificial datasets have been provided. Results of the proposed pinTSVC model are compared with TWSVC, Twin Bounded Support Vector Clustering (TBSVC) and Fuzzy c-means clustering (FCM). Detailed and exhaustive comparisons demonstrate the better performance and generalization of the proposed pinTSVC for noise-corrupted datasets. Further experiments and analysis on the performance of the above-mentioned clustering algorithms on structural MRI (sMRI) images taken from the ADNI database, face clustering, and facial expression clustering have been done to demonstrate the effectiveness and feasibility of the proposed pinTSVC model.

Sign in / Sign up

Export Citation Format

Share Document