scholarly journals Support Vector Machines for Unbalanced Multicategory Classification

2015 ◽  
Vol 2015 ◽  
pp. 1-7 ◽  
Author(s):  
Kang-Mo Jung
Keyword(s):  
Penalty Function ◽  
Theoretical Background ◽  
Support Vector ◽  
Important Research ◽  
Absolute Deviation ◽  
Hinge Loss ◽  
Important Research Topic ◽  
Vector Machines ◽  
Smoothly Clipped Absolute Deviation ◽  
Treat All

Classification is a very important research topic and its applications are various, because data can be easily obtained in these days. Among many techniques of classification the support vector machine (SVM) is widely applied to bioinformatics or genetic analysis, because it gives sound theoretical background and its performance is superior to other methods. The SVM can be rewritten by a combination of the hinge loss function and the penalty function. The smoothly clipped absolute deviation penalty function satisfies desirably statistical properties. Since standard SVM techniques typically treat all classes equally, it is not well suited to unbalanced proportion data. We propose a robust method to treat unbalanced cases based on the weights of the class. Simulation and a numerical example show that the proposed method is effective to analyze unbalanced proportion data.

scholarly journals Robust Support Vector Machines for Classification with Nonconvex and Smooth Losses

2016 ◽  
Vol 28 (6) ◽  
pp. 1217-1247 ◽  
Author(s):  
Yunlong Feng ◽  
Yuning Yang ◽  
Xiaolin Huang ◽  
Siamak Mehrkanoon ◽  
Johan A. K. Suykens
Keyword(s):  
Support Vector Machines ◽  
Robust Statistics ◽  
Real Data ◽  
Constructive Proof ◽  
Support Vector ◽  
Data Sets ◽  
Large Margin ◽  
Hinge Loss ◽  
Vector Machines ◽  
Smooth Classification

This letter addresses the robustness problem when learning a large margin classifier in the presence of label noise. In our study, we achieve this purpose by proposing robustified large margin support vector machines. The robustness of the proposed robust support vector classifiers (RSVC), which is interpreted from a weighted viewpoint in this work, is due to the use of nonconvex classification losses. Besides the robustness, we also show that the proposed RSCV is simultaneously smooth, which again benefits from using smooth classification losses. The idea of proposing RSVC comes from M-estimation in statistics since the proposed robust and smooth classification losses can be taken as one-sided cost functions in robust statistics. Its Fisher consistency property and generalization ability are also investigated. Besides the robustness and smoothness, another nice property of RSVC lies in the fact that its solution can be obtained by solving weighted squared hinge loss–based support vector machine problems iteratively. We further show that in each iteration, it is a quadratic programming problem in its dual space and can be solved by using state-of-the-art methods. We thus propose an iteratively reweighted type algorithm and provide a constructive proof of its convergence to a stationary point. Effectiveness of the proposed classifiers is verified on both artificial and real data sets.

scholarly journals A Study on L2-Loss (Squared Hinge-Loss) Multiclass SVM

2013 ◽  
Vol 25 (5) ◽  
pp. 1302-1323 ◽  
Author(s):  
Ching-Pei Lee ◽  
Chih-Jen Lin
Keyword(s):  
Support Vector Machines ◽  
Optimization Problem ◽  
Support Vector ◽  
Hinge Loss ◽  
Vector Machines ◽  
Multiclass Svm ◽  
L1 And L2 ◽  
Multiclass Support Vector Machines ◽  
Tedious Process

Crammer and Singer's method is one of the most popular multiclass support vector machines (SVMs). It considers L1 loss (hinge loss) in a complicated optimization problem. In SVM, squared hinge loss (L2 loss) is a common alternative to L1 loss, but surprisingly we have not seen any paper studying the details of Crammer and Singer's method using L2 loss. In this letter, we conduct a thorough investigation. We show that the derivation is not trivial and has some subtle differences from the L1 case. Details provided in this work can be a useful reference for those who intend to use Crammer and Singer's method with L2 loss. They do not need a tedious process to derive everything by themselves. Furthermore, we present some new results on and discussion of both L1- and L2-loss formulations.

scholarly journals HEARTBEAT CLASSIFICATION USING SUPPORT VECTOR MACHINES (SVMs) WITH AN EMBEDDED REJECT OPTION

2012 ◽  
Vol 26 (01) ◽  
pp. 1250001 ◽  
Author(s):  
ZAHIA ZIDELMAL ◽  
AHMED AMIROU ◽  
ADEL BELOUCHRANI
Keyword(s):  
Support Vector Machines ◽  
Performance Enhancement ◽  
Support Vector ◽  
Heartbeat Classification ◽  
Medical Diagnoses ◽  
Hinge Loss ◽  
Beat Classification ◽  
Vector Machines ◽  
Significant Performance ◽  
The Cost

In this paper, we introduce a new system for ECG beat classification using support vector machines classifier with a double hinge loss. The proposed classifier rejects samples that cannot be classified with enough confidence. Specifically in medical diagnoses, the consequence of a wrong classification can be so harmful that it is convenient to reject such sample. After ECG preprocessing, feature selection and extraction, our decision rule uses dynamic reject thresholds according to the cost of rejecting or misclassifying a sample. Significant performance enhancement is observed when the proposed approach is tested with the MIT-BIH arrythmia database. The achieved results are represented by the error reject tradeoff. We obtained 98.2% of sensitivity with no rejection and more than 99% of sensitivity for the optimal classification cost being competitive to other published studies.

scholarly journals K Important Neighbors: A Novel Approach to Binary Classification in High Dimensional Data

2017 ◽  
Vol 2017 ◽  
pp. 1-9 ◽  
Author(s):  
Hadi Raeisi Shahraki ◽  
Saeedeh Pourahmad ◽  
Najaf Zare
Keyword(s):  
Euclidean Distance ◽  
Binary Classification ◽  
Penalized Regression ◽  
Dissimilarity Measure ◽  
High Dimensional ◽  
Support Vector ◽  
Absolute Deviation ◽  
Novel Approach ◽  
Initial Stage ◽  
Smoothly Clipped Absolute Deviation

K nearest neighbors (KNN) are known as one of the simplest nonparametric classifiers but in high dimensional setting accuracy of KNN are affected by nuisance features. In this study, we proposed the K important neighbors (KIN) as a novel approach for binary classification in high dimensional problems. To avoid the curse of dimensionality, we implemented smoothly clipped absolute deviation (SCAD) logistic regression at the initial stage and considered the importance of each feature in construction of dissimilarity measure with imposing features contribution as a function of SCAD coefficients on Euclidean distance. The nature of this hybrid dissimilarity measure, which combines information of both features and distances, enjoys all good properties of SCAD penalized regression and KNN simultaneously. In comparison to KNN, simulation studies showed that KIN has a good performance in terms of both accuracy and dimension reduction. The proposed approach was found to be capable of eliminating nearly all of the noninformative features because of utilizing oracle property of SCAD penalized regression in the construction of dissimilarity measure. In very sparse settings, KIN also outperforms support vector machine (SVM) and random forest (RF) as the best classifiers.

scholarly journals Robust relative margin support vector machines

2016 ◽  
Vol 11 (2) ◽  
pp. 186-191 ◽  
Author(s):  
Yunyan Song ◽  
Wenxin Zhu ◽  
Yingyuan Xiao ◽  
Ping Zhong
Keyword(s):  
Computational Complexity ◽  
Binary Classification ◽  
Support Vector ◽  
Decision Boundary ◽  
Large Margin ◽  
Hinge Loss ◽  
Shortest Distance ◽  
Vector Machines ◽  
Pinball Loss ◽  
Real World Problems

Recently, a class of classifiers, called relative margin machine, has been developed. Relative margin machine has shown significant improvements over the large margin counterparts on real-world problems. In binary classification, the most widely used loss function is the hinge loss, which results in the hinge loss relative margin machine. The hinge loss relative margin machine is sensitive to outliers. In this article, we proposed to change maximizing the shortest distance used in relative margin machine into maximizing the quantile distance, the pinball loss which is related to quantiles was used in classification. The proposed method is less sensitive to noise, especially the feature noise around the decision boundary. Meanwhile, the computational complexity of the proposed method is similar to that of the relative margin machine.

Robust support vector machines based on the rescaled hinge loss function

2017 ◽  
Vol 63 ◽  
pp. 139-148 ◽  
Author(s):  
Guibiao Xu ◽  
Zheng Cao ◽  
Bao-Gang Hu ◽  
Jose C. Principe
Keyword(s):  
Support Vector Machines ◽  
Loss Function ◽  
Support Vector ◽  
Hinge Loss ◽  
Vector Machines
Sign in / Sign up

Export Citation Format

Share Document