scholarly journals Nested model averaging on solution path for high‐dimensional linear regression

Stat ◽  
2020 ◽  
Vol 9 (1) ◽  
Author(s):  
Yang Feng ◽  
Qingfeng Liu
Keyword(s):  
Linear Regression ◽  
Model Averaging ◽  
High Dimensional ◽  
Solution Path ◽  
Nested Model

Sequential Model Averaging for High Dimensional Linear Regression Models

2017 ◽  
Author(s):  
Wei Lan ◽  
Yingying Ma ◽  
Junlong Zhao ◽  
Hansheng Wang ◽  
Chih-Ling Tsai
Keyword(s):  
Linear Regression ◽  
Regression Models ◽  
Model Averaging ◽  
High Dimensional ◽  
Linear Regression Models ◽  
Sequential Model

Sequential Model Averaging for High Dimensional Linear Regression Models

2018 ◽  
Author(s):  
WEI LAN ◽  
Yingying Ma ◽  
Junlong Zhao ◽  
Hansheng Wang ◽  
Chih-Ling Tsai
Keyword(s):  
Linear Regression ◽  
Regression Models ◽  
Model Averaging ◽  
High Dimensional ◽  
Linear Regression Models ◽  
Sequential Model

scholarly journals Improved two-stage model averaging for high-dimensional linear regression, with application to Riboflavin data analysis

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Juming Pan
Keyword(s):  
Linear Regression ◽  
Cross Validation ◽  
Linear Models ◽  
Model Averaging ◽  
Alternative Methods ◽  
High Dimensional ◽  
Stage Model ◽  
Two Stage ◽  
Stat Assoc ◽  
Riboflavin Production

Abstract Background Model averaging has attracted increasing attention in recent years for the analysis of high-dimensional data. By weighting several competing statistical models suitably, model averaging attempts to achieve stable and improved prediction. In this paper, we develop a two-stage model averaging procedure to enhance accuracy and stability in prediction for high-dimensional linear regression. First we employ a high-dimensional variable selection method such as LASSO to screen redundant predictors and construct a class of candidate models, then we apply the jackknife cross-validation to optimize model weights for averaging. Results In simulation studies, the proposed technique outperforms commonly used alternative methods under high-dimensional regression setting, in terms of minimizing the mean of the squared prediction error. We apply the proposed method to a riboflavin data, the result show that such method is quite efficient in forecasting the riboflavin production rate, when there are thousands of genes and only tens of subjects. Conclusions Compared with a recent high-dimensional model averaging procedure (Ando and Li in J Am Stat Assoc 109:254–65, 2014), the proposed approach enjoys three appealing features thus has better predictive performance: (1) More suitable methods are applied for model constructing and weighting. (2) Computational flexibility is retained since each candidate model and its corresponding weight are determined in the low-dimensional setting and the quadratic programming is utilized in the cross-validation. (3) Model selection and averaging are combined in the procedure thus it makes full use of the strengths of both techniques. As a consequence, the proposed method can achieve stable and accurate predictions in high-dimensional linear models, and can greatly help practical researchers analyze genetic data in medical research.

scholarly journals SPReM: Sparse Projection Regression Model For High-Dimensional Linear Regression

2015 ◽  
Vol 110 (509) ◽  
pp. 289-302 ◽  
Author(s):  
Qiang Sun ◽  
Hongtu Zhu ◽  
Yufeng Liu ◽  
Joseph G. Ibrahim
Keyword(s):  
Linear Regression ◽  
Regression Model ◽  
High Dimensional ◽  
Sparse Projection

scholarly journals Seagull: lasso, group lasso and sparse-group lasso regularization for linear regression models via proximal gradient descent

2020 ◽  
Vol 21 (1) ◽  
Author(s):  
Jan Klosa ◽  
Noah Simon ◽  
Pål Olof Westermark ◽  
Volkmar Liebscher ◽  
Dörte Wittenburg
Keyword(s):  
Linear Regression ◽  
Regression Models ◽  
Gradient Descent ◽  
Methylation Status ◽  
R Package ◽  
Group Lasso ◽  
High Dimensional ◽  
Linear Regression Models ◽  
Sparse Group Lasso ◽  
Proximal Gradient Descent

Abstract Background Statistical analyses of biological problems in life sciences often lead to high-dimensional linear models. To solve the corresponding system of equations, penalization approaches are often the methods of choice. They are especially useful in case of multicollinearity, which appears if the number of explanatory variables exceeds the number of observations or for some biological reason. Then, the model goodness of fit is penalized by some suitable function of interest. Prominent examples are the lasso, group lasso and sparse-group lasso. Here, we offer a fast and numerically cheap implementation of these operators via proximal gradient descent. The grid search for the penalty parameter is realized by warm starts. The step size between consecutive iterations is determined with backtracking line search. Finally, seagull -the R package presented here- produces complete regularization paths. Results Publicly available high-dimensional methylation data are used to compare seagull to the established R package SGL. The results of both packages enabled a precise prediction of biological age from DNA methylation status. But even though the results of seagull and SGL were very similar (R2 > 0.99), seagull computed the solution in a fraction of the time needed by SGL. Additionally, seagull enables the incorporation of weights for each penalized feature. Conclusions The following operators for linear regression models are available in seagull: lasso, group lasso, sparse-group lasso and Integrative LASSO with Penalty Factors (IPF-lasso). Thus, seagull is a convenient envelope of lasso variants.

Sign in / Sign up

Export Citation Format

Share Document