scholarly journals Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model

2016 ◽  
Vol 20 ◽  
pp. 309-331 ◽  
Author(s):  
Marius Kwemou
Keyword(s):  
Logistic Model ◽  
Group Lasso ◽  
High Dimensional ◽  
Oracle Inequalities

scholarly journals Oracle inequalities for weighted group lasso in high-dimensional misspecified Cox models

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Yijun Xiao ◽  
Ting Yan ◽  
Huiming Zhang ◽  
Yuanyuan Zhang
Keyword(s):  
Estimation Error ◽  
Group Lasso ◽  
High Dimensional ◽  
Group Sparsity ◽  
Oracle Inequalities ◽  
Cox Models ◽  
Special Cases ◽  
Coefficient Vector ◽  
General Norm ◽  
Time Dependent Covariates

AbstractWe study the nonasymptotic properties of a general norm penalized estimator, which include Lasso, weighted Lasso, and group Lasso as special cases, for sparse high-dimensional misspecified Cox models with time-dependent covariates. Under suitable conditions on the true regression coefficients and random covariates, we provide oracle inequalities for prediction and estimation error based on the group sparsity of the true coefficient vector. The nonasymptotic oracle inequalities show that the penalized estimator has good sparse approximation of the true model and enables to select a few meaningful structure variables among the set of features.

scholarly journals Simultaneous Channel and Feature Selection of Fused EEG Features Based on Sparse Group Lasso

2015 ◽  
Vol 2015 ◽  
pp. 1-13 ◽  
Author(s):  
Jin-Jia Wang ◽  
Fang Xue ◽  
Hui Li
Keyword(s):  
Feature Selection ◽  
Feature Selection Method ◽  
Group Lasso ◽  
High Dimensional ◽  
Test Accuracy ◽  
Gradient Descent Method ◽  
Feature Subset ◽  
Eeg Signals ◽  
Sparse Group Lasso ◽  
Selection Of

Feature extraction and classification of EEG signals are core parts of brain computer interfaces (BCIs). Due to the high dimension of the EEG feature vector, an effective feature selection algorithm has become an integral part of research studies. In this paper, we present a new method based on a wrapped Sparse Group Lasso for channel and feature selection of fused EEG signals. The high-dimensional fused features are firstly obtained, which include the power spectrum, time-domain statistics, AR model, and the wavelet coefficient features extracted from the preprocessed EEG signals. The wrapped channel and feature selection method is then applied, which uses the logistical regression model with Sparse Group Lasso penalized function. The model is fitted on the training data, and parameter estimation is obtained by modified blockwise coordinate descent and coordinate gradient descent method. The best parameters and feature subset are selected by using a 10-fold cross-validation. Finally, the test data is classified using the trained model. Compared with existing channel and feature selection methods, results show that the proposed method is more suitable, more stable, and faster for high-dimensional feature fusion. It can simultaneously achieve channel and feature selection with a lower error rate. The test accuracy on the data used from international BCI Competition IV reached 84.72%.

scholarly journals Seagull: lasso, group lasso and sparse-group lasso regularization for linear regression models via proximal gradient descent

2020 ◽  
Vol 21 (1) ◽  
Author(s):  
Jan Klosa ◽  
Noah Simon ◽  
Pål Olof Westermark ◽  
Volkmar Liebscher ◽  
Dörte Wittenburg
Keyword(s):  
Linear Regression ◽  
Regression Models ◽  
Gradient Descent ◽  
Methylation Status ◽  
R Package ◽  
Group Lasso ◽  
High Dimensional ◽  
Linear Regression Models ◽  
Sparse Group Lasso ◽  
Proximal Gradient Descent

Abstract Background Statistical analyses of biological problems in life sciences often lead to high-dimensional linear models. To solve the corresponding system of equations, penalization approaches are often the methods of choice. They are especially useful in case of multicollinearity, which appears if the number of explanatory variables exceeds the number of observations or for some biological reason. Then, the model goodness of fit is penalized by some suitable function of interest. Prominent examples are the lasso, group lasso and sparse-group lasso. Here, we offer a fast and numerically cheap implementation of these operators via proximal gradient descent. The grid search for the penalty parameter is realized by warm starts. The step size between consecutive iterations is determined with backtracking line search. Finally, seagull -the R package presented here- produces complete regularization paths. Results Publicly available high-dimensional methylation data are used to compare seagull to the established R package SGL. The results of both packages enabled a precise prediction of biological age from DNA methylation status. But even though the results of seagull and SGL were very similar (R2 > 0.99), seagull computed the solution in a fraction of the time needed by SGL. Additionally, seagull enables the incorporation of weights for each penalized feature. Conclusions The following operators for linear regression models are available in seagull: lasso, group lasso, sparse-group lasso and Integrative LASSO with Penalty Factors (IPF-lasso). Thus, seagull is a convenient envelope of lasso variants.

scholarly journals Oracle inequalities for high-dimensional prediction

Bernoulli ◽  
2019 ◽  
Vol 25 (2) ◽  
pp. 1225-1255 ◽  
Author(s):  
Johannes Lederer ◽  
Lu Yu ◽  
Irina Gaynanova
Keyword(s):  
High Dimensional ◽  
Oracle Inequalities

scholarly journals Prediction and Variable Selection in High-Dimensional Misspecified Binary Classification

Entropy ◽  
2020 ◽  
Vol 22 (5) ◽  
pp. 543 ◽  
Author(s):  
Konrad Furmańczyk ◽  
Wojciech Rejchel
Keyword(s):  
Logistic Regression ◽  
Variable Selection ◽  
Logistic Model ◽  
Binary Classification ◽  
Model Misspecification ◽  
High Dimensional ◽  
Classification Models ◽  
Computationally Efficient ◽  
Class Labels ◽  
Penalized Logistic Regression

In this paper, we consider prediction and variable selection in the misspecified binary classification models under the high-dimensional scenario. We focus on two approaches to classification, which are computationally efficient, but lead to model misspecification. The first one is to apply penalized logistic regression to the classification data, which possibly do not follow the logistic model. The second method is even more radical: we just treat class labels of objects as they were numbers and apply penalized linear regression. In this paper, we investigate thoroughly these two approaches and provide conditions, which guarantee that they are successful in prediction and variable selection. Our results hold even if the number of predictors is much larger than the sample size. The paper is completed by the experimental results.

Sign in / Sign up

Export Citation Format

Share Document