A Robust Regression Framework with Laplace Kernel-Induced Loss

2017 ◽  
Vol 29 (11) ◽  
pp. 3014-3039 ◽  
Author(s):  
Liming Yang ◽  
Zhuo Ren ◽  
Yidan Wang ◽  
Hongwei Dong
Keyword(s):  
Loss Function ◽  
Near Infrared ◽  
Robust Regression ◽  
Continuous Optimization ◽  
Optimization Method ◽  
Support Vector ◽  
Data Sets ◽  
Difference Of Convex Functions ◽  
Noisy Input ◽  
Regression Framework

This work proposes a robust regression framework with nonconvex loss function. Two regression formulations are presented based on the Laplace kernel-induced loss (LK-loss). Moreover, we illustrate that the LK-loss function is a nice approximation for the zero-norm. However, nonconvexity of the LK-loss makes it difficult to optimize. A continuous optimization method is developed to solve the proposed framework. The problems are formulated as DC (difference of convex functions) programming. The corresponding DC algorithms (DCAs) converge linearly. Furthermore, the proposed algorithms are applied directly to determine the hardness of licorice seeds using near-infrared spectral data with noisy input. Experiments in eight spectral regions show that the proposed methods improve generalization compared with the traditional support vector regressions (SVR), especially in high-frequency regions. Experiments on several benchmark data sets demonstrate that the proposed methods achieve better results than the traditional regression methods in most of data sets we have considered.

Bayesian Trigonometric Support Vector Classifier

2003 ◽  
Vol 15 (9) ◽  
pp. 2227-2254 ◽  
Author(s):  
Wei Chu ◽  
S. Sathiya Keerthi ◽  
Chong Jin Ong
Keyword(s):  
Loss Function ◽  
Gaussian Processes ◽  
Likelihood Function ◽  
Support Vector ◽  
Data Sets ◽  
Model Adaptation ◽  
Bayesian Techniques ◽  
Benchmark Data ◽  
Support Vector Classifier ◽  
Set Up

This letter describes Bayesian techniques for support vector classification. In particular, we propose a novel differentiable loss function, called the trigonometric loss function, which has the desirable characteristic of natural normalization in the likelihood function, and then follow standard gaussian processes techniques to set up a Bayesian framework. In this framework, Bayesian inference is used to implement model adaptation, while keeping the merits of support vector classifier, such as sparseness and convex programming. This differs from standard gaussian processes for classification. Moreover, we put forward class probability in making predictions. Experimental results on benchmark data sets indicate the usefulness of this approach.

scholarly journals Least Absolute Deviation Support Vector Regression

2014 ◽  
Vol 2014 ◽  
pp. 1-8
Author(s):  
Kuaini Wang ◽  
Jingjing Zhang ◽  
Yanyan Chen ◽  
Ping Zhong
Keyword(s):  
Support Vector Regression ◽  
Loss Function ◽  
Robust Regression ◽  
Support Vector ◽  
Regression Estimation ◽  
Least Absolute Deviation ◽  
Absolute Deviation ◽  
Robust Model ◽  
Benchmark Datasets ◽  
Classification And Regression

Least squares support vector machine (LS-SVM) is a powerful tool for pattern classification and regression estimation. However, LS-SVM is sensitive to large noises and outliers since it employs the squared loss function. To solve the problem, in this paper, we propose an absolute deviation loss function to reduce the effects of outliers and derive a robust regression model termed as least absolute deviation support vector regression (LAD-SVR). The proposed loss function is not differentiable. We approximate it by constructing a smooth function and develop a Newton algorithm to solve the robust model. Numerical experiments on both artificial datasets and benchmark datasets demonstrate the robustness and effectiveness of the proposed method.

Robust sparse principal component analysis by DC programming algorithm

2020 ◽  
Vol 39 (3) ◽  
pp. 3183-3193
Author(s):  
Jieya Li ◽  
Liming Yang
Keyword(s):  
Principal Component Analysis ◽  
Continuous Optimization ◽  
Principal Component ◽  
Optimization Method ◽  
Component Analysis ◽  
Programming Algorithm ◽  
Data Sets ◽  
Sparse Principal Component Analysis ◽  
Data Types ◽  
L2 Norm

The classical principal component analysis (PCA) is not sparse enough since it is based on the L2-norm that is also prone to be adversely affected by the presence of outliers and noises. In order to address the problem, a sparse robust PCA framework is proposed based on the min of zero-norm regularization and the max of Lp-norm (0 < p ≤ 2) PCA. Furthermore, we developed a continuous optimization method, DC (difference of convex functions) programming algorithm (DCA), to solve the proposed problem. The resulting algorithm (called DC-LpZSPCA) is convergent linearly. In addition, when choosing different p values, the model can keep robust and is applicable to different data types. Numerical simulations are simulated in artificial data sets and Yale face data sets. Experiment results show that the proposed method can maintain good sparsity and anti-outlier ability.

scholarly journals Watermelon ripeness detector using near infrared spectroscopy

2020 ◽  
Vol 8 (4) ◽  
pp. 317-322
Author(s):  
Edwin R. Arboleda ◽  
Kimberly M. Parazo ◽  
Christle M. Pareja
Keyword(s):  
Infrared Spectroscopy ◽  
Near Infrared Spectroscopy ◽  
High Speed ◽  
Near Infrared ◽  
Agricultural Sector ◽  
Research Problem ◽  
Training Dataset ◽  
Raspberry Pi ◽  
Support Vector ◽  
Data Sets

This study aimed to design and develop a watermelon ripeness detector using Near-Infrared Spectroscopy (NIRS). The research problem being solved in this study is developing a prototype wherein the watermelon ripeness can be detected without the need to open it. This detector will save customers from buying unripe watermelon and the farmers from harvesting an unripe watermelon. The researchers attempted to use the NIRS technique in determining the ripeness level of watermelon as it is widely used in the agricultural sector with high-speed analysis. The project was composed of Raspberry Pi Zero W as the microprocessor unit connected to input and output devices, such as the NIR spectral sensor and the OLED display. It was programmed by Python 3 IDLE. The detector scanned a total of 200 watermelon samples. These samples were grouped as 60 % for the training dataset, 20 % for testing, and another 20 % for evaluation. The data sets were collected and are subjected to the Support Vector Machine (SVM) algorithm. Overall, experimental results showed that the detector could correctly classify both unripe and ripe watermelons with 92.5 % accuracy.

Parameter Selection Based on Support Vector Machines for Tongue Spectral Identification Model

2014 ◽  
Vol 678 ◽  
pp. 242-251
Author(s):  
Wen Juan Yan ◽  
Guo Quan He ◽  
Shi Jian Huang ◽  
Lin Qin
Keyword(s):  
Near Infrared ◽  
Principal Component ◽  
Kernel Functions ◽  
Support Vector ◽  
Data Sets ◽  
Spectral Identification ◽  
Kernel Parameter ◽  
Two Samples ◽  
Infrared Spectral ◽  
Identification Model

Support Vector Machine (SVM) method is suitable for machine learning. In order to detect pathological information from tongue diagnosis rapidly, noninvasively and objectively, a near infrared spectral identification model is proposed based on SVM. The tongue spectral data of healthy people and hepatitis patients were collected. Twenty two samples were obtained for individual groups, and for each group, fifteen samples were randomly selected and used as the training sets, while the other seven were taken as the prediction sets. For the data sets, The effects of the principal component number, kernel parameters, and kernel functions on the identification model were investigated respectively. The results showed that the penalty parameter c was always 0.25, not related to the values of the principal component number and kernel parameter g. The kernel parameter g decreased along with the increased number of principal components, and ultimately reached a relatively stable value. When the Radial Basis Function (RBF) was applied, the established model was the best, indicating that the SVM approach is feasible to classify and recognize tongue near infrared spectroscopy, as along as right parameters are selected. This can provide a novel tongue spectral analysis method to distinguish healthy individuals from hepatitis patients.

scholarly journals Sinkhorn Regression

2020 ◽  
Author(s):  
Lei Luo ◽  
Jian Pei ◽  
Heng Huang
Keyword(s):  
Loss Function ◽  
Robust Regression ◽  
Real Data ◽  
Mass Spectrometry Data ◽  
Data Sets ◽  
Data Set ◽  
Practical Applications ◽  
Lp Norm ◽  
Leibler Divergence ◽  
Geometric Knowledge

This paper introduces a novel Robust Regression (RR) model, named Sinkhorn regression, which imposes Sinkhorn distances on both loss function and regularization. Traditional RR methods target at searching for an element-wise loss function (e.g., Lp-norm) to characterize the errors such that outlying data have a relatively smaller influence on the regression estimator. Due to the neglect of the geometric information, they often lead to the suboptimal results in the practical applications. To address this problem, we use a cross-bin distance function, i.e., Sinkhorn distances, to capture the geometric knowledge of real data. Sinkhorn distances is invariant in movement, rotation and zoom. Thus, our method is more robust to variations of data than traditional regression models. Meanwhile, we leverage Kullback-Leibler divergence to relax the proposed model with marginal constraints into its unbalanced formulation to adapt more types of features. In addition, we propose an efficient algorithm to solve the relaxed model and establish its complete statistical guarantees under mild conditions. Experiments on the five publicly available microarray data sets and one mass spectrometry data set demonstrate the effectiveness and robustness of our method.

Identification of Candidate Genetic Markers and a Novel 4-genes Diagnostic Model in Osteoarthritis through Integrating Multiple Microarray Data

2020 ◽  
Vol 23 (8) ◽  
pp. 805-813
Author(s):  
Ai Jiang ◽  
Peng Xu ◽  
Zhenda Zhao ◽  
Qizhao Tan ◽  
Shang Sun ◽  
...  
Keyword(s):  
Signaling Pathway ◽  
Microarray Data ◽  
Differential Expression Analysis ◽  
Enrichment Analysis ◽  
Mapk Signaling ◽  
Functional Enrichment ◽  
Joint Disease ◽  
Support Vector ◽  
Diagnostic Model ◽  
Data Sets

Background: Osteoarthritis (OA) is a joint disease that leads to a high disability rate and a low quality of life. With the development of modern molecular biology techniques, some key genes and diagnostic markers have been reported. However, the etiology and pathogenesis of OA are still unknown. Objective: To develop a gene signature in OA. Method: In this study, five microarray data sets were integrated to conduct a comprehensive network and pathway analysis of the biological functions of OA related genes, which can provide valuable information and further explore the etiology and pathogenesis of OA. Results and Discussion: Differential expression analysis identified 180 genes with significantly expressed expression in OA. Functional enrichment analysis showed that the up-regulated genes were associated with rheumatoid arthritis (p < 0.01). Down-regulated genes regulate the biological processes of negative regulation of kinase activity and some signaling pathways such as MAPK signaling pathway (p < 0.001) and IL-17 signaling pathway (p < 0.001). In addition, the OA specific protein-protein interaction (PPI) network was constructed based on the differentially expressed genes. The analysis of network topological attributes showed that differentially upregulated VEGFA, MYC, ATF3 and JUN genes were hub genes of the network, which may influence the occurrence and development of OA through regulating cell cycle or apoptosis, and were potential biomarkers of OA. Finally, the support vector machine (SVM) method was used to establish the diagnosis model of OA, which not only had excellent predictive power in internal and external data sets (AUC > 0.9), but also had high predictive performance in different chip platforms (AUC > 0.9) and also had effective ability in blood samples (AUC > 0.8). Conclusion: The 4-genes diagnostic model may be of great help to the early diagnosis and prediction of OA.

Rapid Determining Contents of the Rhubarb Anthraquinones Compounds by Support Vector Machines Modeling based on Near Infrared Spectra

2020 ◽  
Vol 16 ◽  
Author(s):  
Linqi Liu ◽  
JInhua Luo ◽  
Chenxi Zhao ◽  
Bingxue Zhang ◽  
Wei Fan ◽  
...  
Keyword(s):  
Infrared Spectra ◽  
Near Infrared ◽  
Mean Squared Error ◽  
Rapid Determination ◽  
Partial Least Square ◽  
Least Square ◽  
Support Vector ◽  
Near Infrared Spectra ◽  
Aloe Emodin ◽  
Relative Differences

BACKGROUND: Measuring medicinal compounds to evaluate their quality and efficacy has been recognized as a useful approach in treatment. Rhubarb anthraquinones compounds (mainly including aloe-emodin, rhein, emodin, chrysophanol and physcion) are its main effective components as purgating drug. In the current Chinese Pharmacopoeia, the total anthraquinones content is designated as its quantitative quality and control index while the content of each compound has not been specified. METHODS: On the basis of forty rhubarb samples, the correlation models between the near infrared spectra and UPLC analysis data were constructed using support vector machine (SVM) and partial least square (PLS) methods according to Kennard and Stone algorithm for dividing the calibration/prediction datasets. Good models mean they have high correlation coefficients (R2) and low root mean squared error of prediction (RMSEP) values. RESULTS: The models constructed by SVM have much better performance than those by PLS methods. The SVM models have high R2 of 0.8951, 0.9738, 0.9849, 0.9779, 0.9411 and 0.9862 that correspond to aloe-emodin, rhein, emodin, chrysophanol, physcion and total anthraquinones contents, respectively. The corresponding RMSEPs are 0.3592, 0.4182, 0.4508, 0.7121, 0.8365 and 1.7910, respectively. 75% of the predicted results have relative differences being lower than 10%. As for rhein and total anthraquinones, all of the predicted results have relative differences being lower than 10%. CONCLUSION: The nonlinear models constructed by SVM showed good performances with predicted values close to the experimental values. This can perform the rapid determination of the main medicinal ingredients in rhubarb medicinal materials.

Fault Diagnosis Method of Mine Motor Based on Support Vector Machine

2019 ◽  
Vol 13 ◽  
Author(s):  
Yan Zhang ◽  
Ren Sheng
Keyword(s):  
Support Vector Machine ◽  
Fault Diagnosis ◽  
Optimization Method ◽  
Support Vector ◽  
Optimal Parameters ◽  
Instantaneous Power ◽  
Practical Usefulness ◽  
Sample Data ◽  
Diagnosis Method ◽  
Fault Characteristic

Background: In order to improve the efficiency of fault treatment of mining motor, the method of model construction is used to construct the type of kernel function based on the principle of vector machine classification and the optimization method of parameters. Methodology: One-to-many algorithm is used to establish two kinds of support vector machine models for fault diagnosis of motor rotor of crusher. One of them is to obtain the optimal parameters C and g based on the input samples of the instantaneous power fault characteristic data of some motor rotors which have not been processed by rough sets. Patents on machine learning have also shows their practical usefulness in the selction of the feature for fault detection. Results: The results show that the instantaneous power fault feature extracted from the rotor of the crusher motor is obtained by the cross validation method of grid search k-weights (where k is 3) and the final data of the applied Gauss radial basis penalty parameter C and the nuclear parameter g are obtained. Conclusion: The model established by the optimal parameters is used to classify and diagnose the sample of instantaneous power fault characteristic measurement of motor rotor. Therefore, the classification accuracy of the sample data processed by rough set is higher.

scholarly journals Comparison of Spectroscopic Techniques Combined with Chemometrics for Cocaine Powder Analysis

2020 ◽  
Vol 44 (8) ◽  
pp. 851-860
Author(s):  
Joy Eliaerts ◽  
Natalie Meert ◽  
Pierre Dardenne ◽  
Vincent Baeten ◽  
Juan-Antonio Fernandez Pierna ◽  
...  
Keyword(s):  
Gas Chromatography ◽  
Near Infrared ◽  
Evaluation Criteria ◽  
Classification Model ◽  
Support Vector ◽  
Spectroscopic Techniques ◽  
Data Set ◽  
Promising Tool ◽  
Powder Analysis ◽  
Mir Spectra

Abstract Spectroscopic techniques combined with chemometrics are a promising tool for analysis of seized drug powders. In this study, the performance of three spectroscopic techniques [Mid-InfraRed (MIR), Raman and Near-InfraRed (NIR)] was compared. In total, 364 seized powders were analyzed and consisted of 276 cocaine powders (with concentrations ranging from 4 to 99 w%) and 88 powders without cocaine. A classification model (using Support Vector Machines [SVM] discriminant analysis) and a quantification model (using SVM regression) were constructed with each spectral dataset in order to discriminate cocaine powders from other powders and quantify cocaine in powders classified as cocaine positive. The performances of the models were compared with gas chromatography coupled with mass spectrometry (GC–MS) and gas chromatography with flame-ionization detection (GC–FID). Different evaluation criteria were used: number of false negatives (FNs), number of false positives (FPs), accuracy, root mean square error of cross-validation (RMSECV) and determination coefficients (R2). Ten colored powders were excluded from the classification data set due to fluorescence background observed in Raman spectra. For the classification, the best accuracy (99.7%) was obtained with MIR spectra. With Raman and NIR spectra, the accuracy was 99.5% and 98.9%, respectively. For the quantification, the best results were obtained with NIR spectra. The cocaine content was determined with a RMSECV of 3.79% and a R2 of 0.97. The performance of MIR and Raman to predict cocaine concentrations was lower than NIR, with RMSECV of 6.76% and 6.79%, respectively and both with a R2 of 0.90. The three spectroscopic techniques can be applied for both classification and quantification of cocaine, but some differences in performance were detected. The best classification was obtained with MIR spectra. For quantification, however, the RMSECV of MIR and Raman was twice as high in comparison with NIR. Spectroscopic techniques combined with chemometrics can reduce the workload for confirmation analysis (e.g., chromatography based) and therefore save time and resources.

Sign in / Sign up

Export Citation Format

Share Document