scholarly journals SVM-Based Multiple Instance Classification via DC Optimization

Algorithms ◽  
2019 ◽  
Vol 12 (12) ◽  
pp. 249 ◽  
Author(s):  
Annabella Astorino ◽  
Antonio Fuduli ◽  
Giovanni Giallombardo ◽  
Giovanna Miglionico
Keyword(s):  
Binary Classification ◽  
Error Function ◽  
Multiple Instance Learning ◽  
Classification Error ◽  
Support Vector ◽  
Data Sets ◽  
Training Set ◽  
Dc Algorithm ◽  
Difference Of Convex ◽  
Dc Decomposition

A multiple instance learning problem consists of categorizing objects, each represented as a set (bag) of points. Unlike the supervised classification paradigm, where each point of the training set is labeled, the labels are only associated with bags, while the labels of the points inside the bags are unknown. We focus on the binary classification case, where the objective is to discriminate between positive and negative bags using a separating surface. Adopting a support vector machine setting at the training level, the problem of minimizing the classification-error function can be formulated as a nonconvex nonsmooth unconstrained program. We propose a difference-of-convex (DC) decomposition of the nonconvex function, which we face using an appropriate nonsmooth DC algorithm. Some of the numerical results on benchmark data sets are reported.

Online Learning Based on Online DCA and Application to Online Classification

2020 ◽  
Vol 32 (4) ◽  
pp. 759-793 ◽  
Author(s):  
Hoai An Le Thi ◽  
Vinh Thanh Ho
Keyword(s):  
Online Learning ◽  
State Of The Art ◽  
Classification Algorithms ◽  
Data Sets ◽  
Prediction Problem ◽  
Dc Algorithm ◽  
Difference Of Convex Functions ◽  
Online Classification ◽  
Learning Techniques ◽  
Difference Of Convex

We investigate an approach based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) for online learning techniques. The prediction problem of an online learner can be formulated as a DC program for which online DCA is applied. We propose the two so-called complete/approximate versions of online DCA scheme and prove their logarithmic/sublinear regrets. Six online DCA-based algorithms are developed for online binary linear classification. Numerical experiments on a variety of benchmark classification data sets show the efficiency of our proposed algorithms in comparison with the state-of-the-art online classification algorithms.

A hybrid of fast K-nearest neighbor and improved directed acyclic graph support vector machine for large-scale supersonic inlet flow pattern recognition

2021 ◽  
pp. 095441002110086
Author(s):  
Huan Wu ◽  
Yong-Ping Zhao ◽  
Tan Hui-Jun
Keyword(s):  
Pattern Recognition ◽  
Flow Pattern ◽  
Directed Acyclic Graph ◽  
Large Scale ◽  
Computational Cost ◽  
Classification Error ◽  
Support Vector ◽  
Training Set ◽  
Inlet Flow ◽  
Supersonic Inlet

Inlet flow pattern recognition is one of the most crucial issues and also the foundation of protection control for supersonic air-breathing propulsion systems. This article proposes a hybrid algorithm of fast K-nearest neighbors (F-KNN) and improved directed acyclic graph support vector machine (I-DAGSVM) to solve this issue based on a large amount of experimental data. The basic idea behind the proposed algorithm is combining F-KNN and I-DAGSVM together to reduce the classification error and computational cost when dealing with big data. The proposed algorithm first finds a small set of nearest samples from the training set quickly by F-KNN and then trains a local I-DAGSVM classifier based on these nearest samples. Compared with standard KNN which needs to compare each test sample with the entire training set, F-KNN uses an efficient index-based strategy to quickly find nearest samples, but there also exists misclassification when the number of nearest samples belonging to different classes is the same. To cope with this, I-DAGSVM is adopted, and its tree structure is improved by a measure of class separability to overcome the sequential randomization in classifier generation and to reduce the classification error. In addition, the proposed algorithm compensates for the expensive computational cost of I-DAGSVM because it only needs to train a local classifier based on a small number of samples found by F-KNN instead of all training samples. With all these strategies, the proposed algorithm combines the advantages of both F-KNN and I-DAGSVM and can be applied to the issue of large-scale supersonic inlet flow pattern recognition. The experimental results demonstrate the effectiveness of the proposed algorithm in terms of classification accuracy and test time.

A STUDY ON THE ERROR OF DISTRIBUTED ALGORITHMS FOR BIG DATA CLASSIFICATION WITH SVM

2017 ◽  
Vol 58 (3-4) ◽  
pp. 231-237
Author(s):  
CHENG WANG ◽  
FEILONG CAO
Keyword(s):  
Support Vector Machine ◽  
Big Data ◽  
Distributed Algorithm ◽  
Data Classification ◽  
Classification Error ◽  
Support Vector ◽  
Data Sets ◽  
Gaussian Kernels ◽  
Big Data Classification ◽  
The Given

The error of a distributed algorithm for big data classification with a support vector machine (SVM) is analysed in this paper. First, the given big data sets are divided into small subsets, on which the classical SVM with Gaussian kernels is used. Then, the classification error of the SVM for each subset is analysed based on the Tsybakov exponent, geometric noise, and width of the Gaussian kernels. Finally, the whole error of the distributed algorithm is estimated in terms of the error of each subset.

scholarly journals THE USE OF MACHINE LEARNING METHODS FOR BINARY CLASSIFICATION OF THE WORKING CONDITION OF BEARINGS USING THE SIGNALS OF VIBRATION ACCELERATION

2021 ◽  
pp. 15-22
Author(s):  
Ruslan Babudzhan ◽  
Konstantyn Isaienkov ◽  
Danilo Krasiy ◽  
Oleksii Vodka ◽  
Ivan Zadorozhny ◽  
...  
Keyword(s):  
Machine Learning ◽  
Binary Classification ◽  
Fractal Dimensions ◽  
Feature Space ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Support Vector ◽  
Data Sets ◽  
Vibration Acceleration ◽  
K Nearest Neighbors

The paper investigates the relationship between vibration acceleration of bearings with their operational state. To determine these dependencies, a testbench was built and 112 experiments were carried out with different bearings: 100 bearings that developed an internal defect during operation and 12bearings without a defect. From the obtained records, a dataset was formed, which was used to build classifiers. Dataset is freely available. A methodfor classifying new and used bearings was proposed, which consists in searching for dependencies and regularities of the signal using descriptive functions: statistical, entropy, fractal dimensions and others. In addition to processing the signal itself, the frequency domain of the bearing operationsignal was also used to complement the feature space. The paper considered the possibility of generalizing the classification for its application on thosesignals that were not obtained in the course of laboratory experiments. An extraneous dataset was found in the public domain. This dataset was used todetermine how accurate a classifier was when it was trained and tested on significantly different signals. Training and validation were carried out usingthe bootstrapping method to eradicate the effect of randomness, given the small amount of training data available. To estimate the quality of theclassifiers, the F1-measure was used as the main metric due to the imbalance of the data sets. The following supervised machine learning methodswere chosen as classifier models: logistic regression, support vector machine, random forest, and K nearest neighbors. The results are presented in theform of plots of density distribution and diagrams.

scholarly journals Reduced Dilation-Erosion Perceptron for Binary Classification

Mathematics ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. 512
Author(s):  
Marcos Eduardo Valle
Keyword(s):  
Mathematical Morphology ◽  
Computing Methodology ◽  
Convex Combination ◽  
Binary Classification ◽  
Feature Space ◽  
Support Vector ◽  
Training Set ◽  
Linear Lattice ◽  
Hinge Loss ◽  
Classification Tasks

Dilation and erosion are two elementary operations from mathematical morphology, a non-linear lattice computing methodology widely used for image processing and analysis. The dilation-erosion perceptron (DEP) is a morphological neural network obtained by a convex combination of a dilation and an erosion followed by the application of a hard-limiter function for binary classification tasks. A DEP classifier can be trained using a convex-concave procedure along with the minimization of the hinge loss function. As a lattice computing model, the DEP classifier assumes the feature and class spaces are partially ordered sets. In many practical situations, however, there is no natural ordering for the feature patterns. Using concepts from multi-valued mathematical morphology, this paper introduces the reduced dilation-erosion (r-DEP) classifier. An r-DEP classifier is obtained by endowing the feature space with an appropriate reduced ordering. Such reduced ordering can be determined using two approaches: one based on an ensemble of support vector classifiers (SVCs) with different kernels and the other based on a bagging of similar SVCs trained using different samples of the training set. Using several binary classification datasets from the OpenML repository, the ensemble and bagging r-DEP classifiers yielded mean higher balanced accuracy scores than the linear, polynomial, and radial basis function (RBF) SVCs as well as their ensemble and a bagging of RBF SVCs.

scholarly journals Two New Decomposition Algorithms for Training Bound-Constrained Support Vector Machines*

2015 ◽  
Vol 40 (1) ◽  
pp. 67-86 ◽  
Author(s):  
Lingfeng Niu ◽  
Ruizhi Zhou ◽  
Xi Zhao ◽  
Yong Shi
Keyword(s):  
Binary Classification ◽  
Gradient Algorithm ◽  
Support Vector ◽  
Data Sets ◽  
Decomposition Algorithms ◽  
Public Data ◽  
Vector Machines ◽  
Working Set ◽  
Derivative Information ◽  
New Algorithms

Abstract Bound-constrained Support Vector Machine(SVM) is one of the stateof- art model for binary classification. The decomposition method is currently one of the major methods for training SVMs, especially when the nonlinear kernel is used. In this paper, we proposed two new decomposition algorithms for training bound-constrained SVMs. Projected gradient algorithm and interior point method are combined together to solve the quadratic subproblem effciently. The main difference between the two algorithms is the way of choosing working set. The first one only uses first order derivative information of the model for simplicity. The second one incorporate part of second order information into the process of working set selection, besides the gradient. Both algorithms are proved to be global convergent in theory. New algorithms is compared with the famous package BSVM. Numerical experiments on several public data sets validate the effciency of the proposed methods.

scholarly journals Multi-choice wavelet thresholding based binary classification method

Methodology ◽  
2020 ◽  
Vol 16 (2) ◽  
pp. 127-146 ◽  
Author(s):  
Seung Hyun Baek ◽  
Alberto Garcia-Diaz ◽  
Yuanshun Dai
Keyword(s):  
Data Mining ◽  
Binary Classification ◽  
High Dimensional ◽  
Support Vector ◽  
Published Data ◽  
Data Sets ◽  
Wavelet Thresholding ◽  
Reliability Of Results ◽  
Vector Machines ◽  
Cognition Model

Data mining is one of the most effective statistical methodologies to investigate a variety of problems in areas including pattern recognition, machine learning, bioinformatics, chemometrics, and statistics. In particular, statistically-sophisticated procedures that emphasize on reliability of results and computational efficiency are required for the analysis of high-dimensional data. Optimization principles can play a significant role in the rationalization and validation of specialized data mining procedures. This paper presents a novel methodology which is Multi-Choice Wavelet Thresholding (MCWT) based three-step methodology consists of three processes: perception (dimension reduction), decision (feature ranking), and cognition (model selection). In these steps three concepts known as wavelet thresholding, support vector machines for classification and information complexity are integrated to evaluate learning models. Three published data sets are used to illustrate the proposed methodology. Additionally, performance comparisons with recent and widely applied methods are shown.

scholarly journals Reduction from Cost-Sensitive Ordinal Ranking to Weighted Binary Classification

2012 ◽  
Vol 24 (5) ◽  
pp. 1329-1367 ◽  
Author(s):  
Hsuan-Tien Lin ◽  
Ling Li
Keyword(s):  
Binary Classification ◽  
Upper Bounds ◽  
Support Vector ◽  
Data Sets ◽  
Binary Classifier ◽  
Generalization Bounds ◽  
Ordinal Ranking ◽  
Benchmark Data ◽  
Ranking Algorithms ◽  
Ranking Performance

We present a reduction framework from ordinal ranking to binary classification. The framework consists of three steps: extracting extended examples from the original examples, learning a binary classifier on the extended examples with any binary classification algorithm, and constructing a ranker from the binary classifier. Based on the framework, we show that a weighted 0/1 loss of the binary classifier upper-bounds the mislabeling cost of the ranker, both error-wise and regret-wise. Our framework allows not only the design of good ordinal ranking algorithms based on well-tuned binary classification approaches, but also the derivation of new generalization bounds for ordinal ranking from known bounds for binary classification. In addition, our framework unifies many existing ordinal ranking algorithms, such as perceptron ranking and support vector ordinal regression. When compared empirically on benchmark data sets, some of our newly designed algorithms enjoy advantages in terms of both training speed and generalization performance over existing algorithms. In addition, the newly designed algorithms lead to better cost-sensitive ordinal ranking performance, as well as improved listwise ranking performance.

scholarly journals Evolutionary multiple instance boosting framework for weakly supervised learning

2021 ◽  
Author(s):  
Kamanasish Bhattacharjee ◽  
Millie Pant ◽  
Shilpa Srivastava
Keyword(s):  
Differential Evolution ◽  
Supervised Learning ◽  
Binary Classification ◽  
Confusion Matrix ◽  
Multiple Instance Learning ◽  
Classification Error ◽  
Weak Classifier ◽  
Inexact Data ◽  
Weakly Supervised ◽  
Boosting Technique

AbstractMultiple instance boosting (MILBoost) is a framework which uses multiple instance learning (MIL) with boosting technique to solve the problems regarding weakly labeled inexact data. This paper proposes an enhanced multiple boosting framework—evolutionary MILBoost (EMILBoost) which utilizes differential evolution (DE) to optimize the combination of weak classifier or weak estimator weights in the framework. A standard MIL dataset MUSK and a binary classification dataset Hastie_10_2 are used to evaluate the results. Results are presented in terms of bag and instance classification error and also confusion matrix of test data.

A pruned support vector data description-based outlier detection method: Applied to robust process monitoring

2020 ◽  
Vol 42 (11) ◽  
pp. 2113-2126 ◽  
Author(s):  
Ping Yuan ◽  
Zhizhong Mao ◽  
Biao Wang
Keyword(s):  
Process Monitoring ◽  
Support Vector ◽  
Support Vector Data Description ◽  
Data Sets ◽  
Vector Data ◽  
Training Set ◽  
Data Set ◽  
Data Description ◽  
One Class Classifier ◽  
Comparative Results

Support vector data description (SVDD) is a boundary-based one-class classifier that has been widely used for process monitoring during recent years. However, in some applications where databases are often contaminated by outliers, the performance of SVDD would become deteriorated, leading to low detection rate. To this end, this paper proposes a pruned SVDD model in order to improve its robustness. In contrast to other robust SVDD models that are developed from the algorithmic level, we prune the basic SVDD from a data level. The rationale is to exclude outlier examples from the final training set as many as possible. Specifically, three different SVDD models are constructed successively with different training sets. The first model is used to extract target points by means of rejecting more suspect outlier examples. The second model is constructed using those extracted target points, and is used to recover some false outlier examples labeled by the first model. We build the third (final) model with the final training set consisting of target examples by the first model and false outlier examples by the second model. We validate our proposed method on 20 benchmark data sets and TE data set. Comparative results show that our pruned model could improve the robustness of SVDD more efficiently.

Sign in / Sign up

Export Citation Format

Share Document