scholarly journals A non-convex optimization framework for large-scale low-rank matrix factorization

2020 ◽  
Author(s):  
Sajad Fathi Hafshejani ◽  
Saeed Vahidian ◽  
Zahra Moaberfard ◽  
Reza Alikhani ◽  
Bill Lin
Keyword(s):  
Matrix Factorization ◽  
Large Scale ◽  
Significant Loss ◽  
Low Rank ◽  
Function Evaluation ◽  
Dimensional Manifold ◽  
Step Size ◽  
Rank Matrix ◽  
Real World Datasets ◽  
Low Rank Matrix

Low-rank matrix factorization problems such as non negative matrix factorization (NMF) can be categorized as a clustering or dimension reduction technique. The latter denotes techniques designed to find representations of some high dimensional dataset in a lower dimensional manifold without a significant loss of information. If such a representation exists, the features ought to contain the most relevant features of the dataset. Many linear dimensionality reduction techniques can be formulated as a matrix factorization. In this paper, we combine the conjugate gradient (CG) method with the Barzilai and Borwein (BB) gradient method, and propose a BB scaling CG method for NMF problems. The new method does not require to compute and store matrices associated with Hessian of the objective functions. Moreover, adopting a suitable BB step size along with a proper nonmonotone strategy which comes by the size convex parameter $\eta_k$, results in a new algorithm that can significantly improve the CPU time, efficiency, the number of function evaluation. Convergence result is established and numerical comparisons of methods on both synthetic and real-world datasets show that the proposed method is efficient in comparison with existing methods and demonstrate the superiority of our algorithms.

scholarly journals A Non-Convex Optimization Framework for Large-Scale Low-rank Matrix Factorization

2021 ◽  
Author(s):  
Sajad Fathi Hafshejani ◽  
Saeed Vahidian ◽  
Zahra Moaberfard ◽  
Bill Lin
Keyword(s):  
Matrix Factorization ◽  
Large Scale ◽  
Significant Loss ◽  
Low Rank ◽  
Function Evaluation ◽  
Dimensional Manifold ◽  
Step Size ◽  
Rank Matrix ◽  
Real World Datasets ◽  
Low Rank Matrix

Low-rank matrix factorization problems such as non negative matrix factorization (NMF) can be categorized as a clustering or dimension reduction technique. The latter denotes techniques designed to find representations of some high dimensional dataset in a lower dimensional manifold without a significant loss of information. If such a representation exists, the features ought to contain the most relevant features of the dataset. Many linear dimensionality reduction techniques can be formulated as a matrix factorization. In this paper, we combine the conjugate gradient (CG) method with the Barzilai and Borwein (BB) gradient method, and propose a BB scaling CG method for NMF problems. The new method does not require to compute and store matrices associated with Hessian of the objective functions. Moreover, adopting a suitable BB step size along with a proper nonmonotone strategy which comes by the size convex parameter $\eta_k$, results in a new algorithm that can significantly improve the CPU time, efficiency, the number of function evaluation. Convergence result is established and numerical comparisons of methods on both synthetic and real-world datasets show that the proposed method is efficient in comparison with existing methods and demonstrate the superiority of our algorithms.

scholarly journals A non-convex optimization framework for large-scale low-rank matrix factorization

2020 ◽  
Author(s):  
Sajad Fathi Hafshejani ◽  
Saeed Vahidian ◽  
Zahra Moaberfard ◽  
Reza Alikhani ◽  
Bill Lin
Keyword(s):  
Matrix Factorization ◽  
Large Scale ◽  
Significant Loss ◽  
Low Rank ◽  
Function Evaluation ◽  
Dimensional Manifold ◽  
Step Size ◽  
Rank Matrix ◽  
Real World Datasets ◽  
Low Rank Matrix

Low-rank matrix factorization problems such as non negative matrix factorization (NMF) can be categorized as a clustering or dimension reduction technique. The latter denotes techniques designed to find representations of some high dimensional dataset in a lower dimensional manifold without a significant loss of information. If such a representation exists, the features ought to contain the most relevant features of the dataset. Many linear dimensionality reduction techniques can be formulated as a matrix factorization. In this paper, we combine the conjugate gradient (CG) method with the Barzilai and Borwein (BB) gradient method, and propose a BB scaling CG method for NMF problems. The new method does not require to compute and store matrices associated with Hessian of the objective functions. Moreover, adopting a suitable BB step size along with a proper nonmonotone strategy which comes by the size convex parameter $\eta_k$, results in a new algorithm that can significantly improve the CPU time, efficiency, the number of function evaluation. Convergence result is established and numerical comparisons of methods on both synthetic and real-world datasets show that the proposed method is efficient in comparison with existing methods and demonstrate the superiority of our algorithms.

Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks

2018 ◽  
Author(s):  
Daniel Povey ◽  
Gaofeng Cheng ◽  
Yiming Wang ◽  
Ke Li ◽  
Hainan Xu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Matrix Factorization ◽  
Deep Neural Networks ◽  
Low Rank ◽  
Rank Matrix ◽  
Low Rank Matrix

scholarly journals Single-cell data clustering based on sparse optimization and low-rank matrix factorization

2021 ◽  
Author(s):  
Yinlei Hu ◽  
Bin Li ◽  
Falai Chen ◽  
Kun Qu
Keyword(s):  
Single Cell ◽  
Rna Sequencing ◽  
Matrix Factorization ◽  
Data Clustering ◽  
Cell Types ◽  
Low Rank ◽  
Sequencing Data ◽  
Rank Matrix ◽  
Single Cell Rna Sequencing ◽  
Low Rank Matrix

Abstract Unsupervised clustering is a fundamental step of single-cell RNA sequencing data analysis. This issue has inspired several clustering methods to classify cells in single-cell RNA sequencing data. However, accurate prediction of the cell clusters remains a substantial challenge. In this study, we propose a new algorithm for single-cell RNA sequencing data clustering based on Sparse Optimization and low-rank matrix factorization (scSO). We applied our scSO algorithm to analyze multiple benchmark datasets and showed that the cluster number predicted by scSO was close to the number of reference cell types and that most cells were correctly classified. Our scSO algorithm is available at https://github.com/QuKunLab/scSO. Overall, this study demonstrates a potent cell clustering approach that can help researchers distinguish cell types in single-cell RNA sequencing data.

Structural Identifiability in Low-Rank Matrix Factorization

Algorithmica ◽  
2009 ◽  
Vol 56 (3) ◽  
pp. 313-332 ◽  
Author(s):  
Epameinondas Fritzilas ◽  
Martin Milanič ◽  
Sven Rahmann ◽  
Yasmin A. Rios-Solis
Keyword(s):  
Matrix Factorization ◽  
Low Rank ◽  
Structural Identifiability ◽  
Rank Matrix ◽  
Low Rank Matrix

scholarly journals A Boosting Framework of Factorization Machine

2021 ◽  
pp. 2159036
Author(s):  
Jun Zhou ◽  
Longfei Li ◽  
Ziqi Liu ◽  
Chaochao Chen
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Experimental Results ◽  
Low Rank ◽  
Inner Product ◽  
Adaptive Boosting ◽  
Rank Matrix ◽  
Factorization Machine ◽  
Fixed Rank ◽  
Low Rank Matrix

Recently, Factorization Machine (FM) has become more and more popular for recommendation systems due to its effectiveness in finding informative interactions between features. Usually, the weights for the interactions are learned as a low rank weight matrix, which is formulated as an inner product of two low rank matrices. This low rank matrix can help improve the generalization ability of Factorization Machine. However, to choose the rank properly, it usually needs to run the algorithm for many times using different ranks, which clearly is inefficient for some large-scale datasets. To alleviate this issue, we propose an Adaptive Boosting framework of Factorization Machine (AdaFM), which can adaptively search for proper ranks for different datasets without re-training. Instead of using a fixed rank for FM, the proposed algorithm will gradually increase its rank according to its performance until the performance does not grow. Extensive experiments are conducted to validate the proposed method on multiple large-scale datasets. The experimental results demonstrate that the proposed method can be more effective than the state-of-the-art Factorization Machines.

Multi-view low-rank matrix factorization using multiple manifold regularization

2019 ◽  
Vol 335 ◽  
pp. 143-152 ◽  
Author(s):  
Shengxiang Gao ◽  
Zhengtao Yu ◽  
Taisong Jin ◽  
Ming Yin
Keyword(s):  
Matrix Factorization ◽  
Low Rank ◽  
Manifold Regularization ◽  
Rank Matrix ◽  
Low Rank Matrix
Sign in / Sign up

Export Citation Format

Share Document