An Efficient Orthonormalization-Free Approach for Sparse Dictionary Learning and Dual Principal Component Pursuit

Xiaoyin Hu; Xin Liu

doi:10.3390/s20113041

An Efficient Orthonormalization-Free Approach for Sparse Dictionary Learning and Dual Principal Component Pursuit

Sensors ◽

10.3390/s20113041 ◽

2020 ◽

Vol 20 (11) ◽

pp. 3041

Author(s):

Xiaoyin Hu ◽

Xin Liu

Keyword(s):

Dictionary Learning ◽

Optimization Problem ◽

High Efficiency ◽

Principal Component ◽

Representation Learning ◽

First Order ◽

Orthogonality Constraints ◽

Sparse Dictionary Learning ◽

Principal Component Pursuit ◽

Order Algorithm

Sparse dictionary learning (SDL) is a classic representation learning method and has been widely used in data analysis. Recently, the ℓ m -norm ( m ≥ 3 , m ∈ N ) maximization has been proposed to solve SDL, which reshapes the problem to an optimization problem with orthogonality constraints. In this paper, we first propose an ℓ m -norm maximization model for solving dual principal component pursuit (DPCP) based on the similarities between DPCP and SDL. Then, we propose a smooth unconstrained exact penalty model and show its equivalence with the ℓ m -norm maximization model. Based on our penalty model, we develop an efficient first-order algorithm for solving our penalty model (PenNMF) and show its global convergence. Extensive experiments illustrate the high efficiency of PenNMF when compared with the other state-of-the-art algorithms on solving the ℓ m -norm maximization with orthogonality constraints.

Download Full-text

Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learning

2019 53rd Asilomar Conference on Signals, Systems, and Computers ◽

10.1109/ieeeconf44664.2019.9048840 ◽

2019 ◽

Author(s):

Shixiang Chen ◽

Zengde Deng ◽

Shiqian Ma ◽

Anthony Man-Cho So

Keyword(s):

Dictionary Learning ◽

Principal Component ◽

Proximal Point ◽

Proximal Point Algorithms ◽

Principal Component Pursuit

Download Full-text

Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learning

IEEE Transactions on Signal Processing ◽

10.1109/tsp.2021.3099643 ◽

2021 ◽

pp. 1-1

Author(s):

Shixiang Chen ◽

Zengde Deng ◽

Shiqian Ma ◽

Anthony Man-Cho So

Keyword(s):

Dictionary Learning ◽

Principal Component ◽

Proximal Point ◽

Proximal Point Algorithms ◽

Principal Component Pursuit

Download Full-text

Hyperspectral Image Classification with Localized Graph Convolutional Filtering

Remote Sensing ◽

10.3390/rs13030526 ◽

2021 ◽

Vol 13 (3) ◽

pp. 526

Author(s):

Shengliang Pu ◽

Yuanfeng Wu ◽

Xu Sun ◽

Xiaotong Sun

Keyword(s):

Hyperspectral Image ◽

Principal Component ◽

Representation Learning ◽

Classification Performance ◽

Hyperspectral Data ◽

Spectral Graph Theory ◽

Feature Reduction ◽

Graph Representation ◽

Novel Method ◽

Local Graph

The nascent graph representation learning has shown superiority for resolving graph data. Compared to conventional convolutional neural networks, graph-based deep learning has the advantages of illustrating class boundaries and modeling feature relationships. Faced with hyperspectral image (HSI) classification, the priority problem might be how to convert hyperspectral data into irregular domains from regular grids. In this regard, we present a novel method that performs the localized graph convolutional filtering on HSIs based on spectral graph theory. First, we conducted principal component analysis (PCA) preprocessing to create localized hyperspectral data cubes with unsupervised feature reduction. These feature cubes combined with localized adjacent matrices were fed into the popular graph convolution network in a standard supervised learning paradigm. Finally, we succeeded in analyzing diversified land covers by considering local graph structure with graph convolutional filtering. Experiments on real hyperspectral datasets demonstrated that the presented method offers promising classification performance compared with other popular competitors.

Download Full-text

Efficient Graph Collaborative Filtering via Contrastive Learning

Sensors ◽

10.3390/s21144666 ◽

2021 ◽

Vol 21 (14) ◽

pp. 4666

Author(s):

Zhiqiang Pan ◽

Honghui Chen

Keyword(s):

Collaborative Filtering ◽

State Of The Art ◽

Representation Learning ◽

First Order ◽

Satisfactory Performance ◽

Potential Applications ◽

Benchmark Datasets ◽

Bayesian Personalized Ranking ◽

Graph Neural Networks ◽

Training Efficiency

Collaborative filtering (CF) aims to make recommendations for users by detecting user’s preference from the historical user–item interactions. Existing graph neural networks (GNN) based methods achieve satisfactory performance by exploiting the high-order connectivity between users and items, however they suffer from the poor training efficiency problem and easily introduce bias for information propagation. Moreover, the widely applied Bayesian personalized ranking (BPR) loss is insufficient to provide supervision signals for training due to the extremely sparse observed interactions. To deal with the above issues, we propose the Efficient Graph Collaborative Filtering (EGCF) method. Specifically, EGCF adopts merely one-layer graph convolution to model the collaborative signal for users and items from the first-order neighbors in the user–item interactions. Moreover, we introduce contrastive learning to enhance the representation learning of users and items by deriving the self-supervisions, which is jointly trained with the supervised learning. Extensive experiments are conducted on two benchmark datasets, i.e., Yelp2018 and Amazon-book, and the experimental results demonstrate that EGCF can achieve the state-of-the-art performance in terms of Recall and normalized discounted cumulative gain (NDCG), especially on ranking the target items at right positions. In addition, EGCF shows obvious advantages in the training efficiency compared with the competitive baselines, making it practicable for potential applications.

Download Full-text

Building Electronic Commerce Recommendation System Based on Ontology Learning and BP Neural Network

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.718-720.1961 ◽

2013 ◽

Vol 718-720 ◽

pp. 1961-1966

Author(s):

Hong Sheng Xu ◽

Qing Tan

Keyword(s):

Neural Network ◽

Electronic Commerce ◽

Bp Neural Network ◽

Optimization Problem ◽

Recommendation System ◽

High Efficiency ◽

Ontology Learning ◽

The Neural Network ◽

Network Process ◽

Hidden Layer

Electronic commerce recommendation system can effectively retain user, prevent users from erosion, and improve e-commerce system sales. BP neural network using iterative operation, solving the weights of the neural network and close values to corresponding network process of learning and memory, to join the hidden layer nodes of the optimization problem of adjustable parameters increase. Ontology learning is the use of machine learning and statistical techniques, with automatic or semi-automatic way, from the existing data resources and obtaining desired body. The paper presents building electronic commerce recommendation system based on ontology learning and BP neural network. Experimental results show that the proposed algorithm has high efficiency.

Download Full-text