scholarly journals Alternating Direction Multiplier Method for Matrix l2,1-Norm Optimization in Multitask Feature Learning Problems

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Yaping Hu ◽  
Liying Liu ◽  
Yujie Wang
Keyword(s):  
Numerical Experiments ◽  
Optimization Problem ◽  
Feature Learning ◽  
Learning Problems ◽  
Multiplier Method ◽  
Feature Selection Problem ◽  
Norm Minimization ◽  
Alternating Direction ◽  
The Matrix ◽  
Spectral Gradient Method

The joint feature selection problem can be resolved by solving a matrix l2,1-norm minimization problem. For l2,1-norm regularization, one of the most fascinating features is that some similar sparsity structures can be employed by multiple predictors. However, the nonsmooth nature of the problem brings great challenges to the problem. In this paper, an alternating direction multiplier method combined with the spectral gradient method is proposed for solving the matrix l2,1-norm optimization problem involved with multitask feature learning. Numerical experiments show the effectiveness of the proposed algorithm.

scholarly journals Discretization and machine learning approximation of BSDEs with a constraint on the Gains-process

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Idris Kharroubi ◽  
Thomas Lim ◽  
Xavier Warin
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Differential Equations ◽  
Numerical Experiments ◽  
Optimization Problem ◽  
Learning Approach ◽  
The Neural Network ◽  
Machine Learning Approach ◽  
Mesh Grid

AbstractWe study the approximation of backward stochastic differential equations (BSDEs for short) with a constraint on the gains process. We first discretize the constraint by applying a so-called facelift operator at times of a grid. We show that this discretely constrained BSDE converges to the continuously constrained one as the mesh grid converges to zero. We then focus on the approximation of the discretely constrained BSDE. For that we adopt a machine learning approach. We show that the facelift can be approximated by an optimization problem over a class of neural networks under constraints on the neural network and its derivative. We then derive an algorithm converging to the discretely constrained BSDE as the number of neurons goes to infinity. We end by numerical experiments.

Accelerated 3D blind separation of convolved mixtures based on the fast iterative shrinkage thresholding algorithm for adaptive multiple subtraction

Geophysics ◽  
2018 ◽  
Vol 83 (2) ◽  
pp. V99-V113 ◽  
Author(s):  
Zhong-Xiao Li ◽  
Zhen-Chun Li
Keyword(s):  
Field Data ◽  
Optimization Problem ◽  
Computational Cost ◽  
Computation Time ◽  
Blind Separation ◽  
L1 Norm ◽  
Norm Minimization ◽  
Iterative Shrinkage ◽  
Thresholding Algorithm ◽  
Iterative Shrinkage Thresholding

After multiple prediction, adaptive multiple subtraction is essential for the success of multiple removal. The 3D blind separation of convolved mixtures (3D BSCM) method, which is effective in conducting adaptive multiple subtraction, needs to solve an optimization problem containing L1-norm minimization constraints on primaries by the iterative reweighted least-squares (IRLS) algorithm. The 3D BSCM method can better separate primaries and multiples than the 1D/2D BSCM method and the method with energy minimization constraints on primaries. However, the 3D BSCM method has high computational cost because the IRLS algorithm achieves nonquadratic optimization with an LS optimization problem solved in each iteration. In general, it is good to have a faster 3D BSCM method. To improve the adaptability of field data processing, the fast iterative shrinkage thresholding algorithm (FISTA) is introduced into the 3D BSCM method. The proximity operator of FISTA can solve the L1-norm minimization problem efficiently. We demonstrate that our FISTA-based 3D BSCM method achieves similar accuracy of estimating primaries as that of the reference IRLS-based 3D BSCM method. Furthermore, our FISTA-based 3D BSCM method reduces computation time by approximately 60% compared with the reference IRLS-based 3D BSCM method in the synthetic and field data examples.

Fast Algorithms for LS and LAD-Collaborative Regression

2021 ◽  
Author(s):  
Jun Sun ◽  
Lingchen Kong ◽  
Mei Li
Keyword(s):  
Numerical Experiments ◽  
Modern Science ◽  
Alternating Direction Method ◽  
Least Square ◽  
High Dimensional ◽  
Statistical Interpretation ◽  
Absolute Deviation ◽  
Linear Rate ◽  
Alternating Direction ◽  
High Dimensional Datasets

With the development of modern science and technology, it is easy to obtain a large number of high-dimensional datasets, which are related but different. Classical unimodel analysis is less likely to capture potential links between the different datasets. Recently, a collaborative regression model based on least square (LS) method for this problem has been proposed. In this paper, we propose a robust collaborative regression based on the least absolute deviation (LAD). We give the statistical interpretation of the LS-collaborative regression and LAD-collaborative regression. Then we design an efficient symmetric Gauss–Seidel-based alternating direction method of multipliers algorithm to solve the two models, which has the global convergence and the Q-linear rate of convergence. Finally we report numerical experiments to illustrate the efficiency of the proposed methods.

scholarly journals Pedestrian Walking Behavior Revealed through a Random Walk Model

2012 ◽  
Vol 2012 ◽  
pp. 1-13
Author(s):  
Hui Xiong ◽  
Liya Yao ◽  
Huachun Tan ◽  
Wuhong Wang
Keyword(s):  
Numerical Experiments ◽  
Flow Simulation ◽  
Two Dimensional ◽  
Pedestrian Flow ◽  
Alternating Direction Implicit Method ◽  
Walking Behavior ◽  
Alternating Direction Implicit ◽  
Alternating Direction ◽  
Continuous Time Random Walks ◽  
High Order Compact Scheme

This paper applies method of continuous-time random walks for pedestrian flow simulation. In the model, pedestrians can walk forward or backward and turn left or right if there is no block. Velocities of pedestrian flow moving forward or diffusing are dominated by coefficients. The waiting time preceding each jump is assumed to follow an exponential distribution. To solve the model, a second-order two-dimensional partial differential equation, a high-order compact scheme with the alternating direction implicit method, is employed. In the numerical experiments, the walking domain of the first one is two-dimensional with two entrances and one exit, and that of the second one is two-dimensional with one entrance and one exit. The flows in both scenarios are one way. Numerical results show that the model can be used for pedestrian flow simulation.

scholarly journals OPTIMIZATION OF LONG-RANGE TRAJECTORY FOR AN UNPOWERED FLIGHT VEHICLE

2020 ◽  
Vol 57 (6A) ◽  
pp. 43
Author(s):  
Tuan Hung Pham ◽  
Duc Cuong Nguyen ◽  
Duc Thanh Nguyen
Keyword(s):  
Long Range ◽  
Numerical Experiments ◽  
Optimization Problem ◽  
Normal Load ◽  
Load Factor ◽  
Flight Vehicle ◽  
Maximum Range ◽  
Normal Acceleration

This report presents problems of optimization of long-range trajectory for an unpowered flight vehicle at subsonic and transonic speed. The results may be recommended to have a new long range trajectory. The optimization problem is solved by numerical experiments while  the normal load factor (normal acceleration) is used as optimization variables with compliance to flight constraints. The focus problem of this study is the investigation of the possibility of trajectory expansion according to the criteria of the maximum range in the first stage of the trajectory.  

scholarly journals Learning Discriminative Correlation Subspace for Heterogeneous Domain Adaptation

2017 ◽  
Author(s):  
Yuguang Yan ◽  
Wen Li ◽  
Michael Ng ◽  
Mingkui Tan ◽  
Hanrui Wu ◽  
...  
Keyword(s):  
Optimization Problem ◽  
Domain Adaptation ◽  
Data Sets ◽  
Target Domain ◽  
Real World Data ◽  
Discriminative Ability ◽  
Convex Optimization Problem ◽  
Alternating Direction ◽  
Feature Spaces ◽  
Target Data

Domain adaptation aims to reduce the effort on collecting and annotating target data by leveraging knowledge from a different source domain. The domain adaptation problem will become extremely challenging when the feature spaces of the source and target domains are different, which is also known as the heterogeneous domain adaptation (HDA) problem. In this paper, we propose a novel HDA method to find the optimal discriminative correlation subspace for the source and target data. The discriminative correlation subspace is inherited from the canonical correlation subspace between the source and target data, and is further optimized to maximize the discriminative ability for the target domain classifier. We formulate a joint objective in order to simultaneously learn the discriminative correlation subspace and the target domain classifier. We then apply an alternating direction method of multiplier (ADMM) algorithm to address the resulting non-convex optimization problem. Comprehensive experiments on two real-world data sets demonstrate the effectiveness of the proposed method compared to the state-of-the-art methods.

Feeder Ship Size Optimization in Maritime Markets with Uncertainty

2021 ◽  
Vol 38 (03) ◽  
pp. 2040014
Author(s):  
Hongtao Hu ◽  
Jiao Mo ◽  
Lufei Huang
Keyword(s):  
Dynamic Programming ◽  
Stochastic Dynamic Programming ◽  
Numerical Experiments ◽  
Optimization Problem ◽  
Planning Horizon ◽  
Dynamic Programming Method ◽  
Size Optimization ◽  
Stochastic Dynamic ◽  
Shipping Company ◽  
Decision Behaviors

This paper considers the ship size optimization problem for a liner shipping company that provides feeder service between one hub port and one feeder port. In the maritime market with uncertainty, this problem becomes more challenging. This research first analyzes the decision behaviors of the shipping company. Then, a stochastic dynamic programming method is proposed to calculate the expected total volume of containers transported within the planning horizon. Using the calculated volumes as input parameters calculate the profit of each ship sizes and then determine the suitable ship size for the feeder route. Numerical experiments are performed to validate the effectiveness of the proposed method and the efficiency of the proposed algorithm.

scholarly journals Superresolution 2D DOA Estimation for a Rectangular Array via Reweighted Decoupled Atomic Norm Minimization

2019 ◽  
Vol 2019 ◽  
pp. 1-13
Author(s):  
Ming-Ming Liu ◽  
Chun-Xi Dong ◽  
Yang-Yang Dong ◽  
Guo-Qing Zhao
Keyword(s):  
Computational Complexity ◽  
Optimization Problem ◽  
Toeplitz Matrices ◽  
Doa Estimation ◽  
Rank Minimization ◽  
Rectangular Array ◽  
Angle Estimation ◽  
Norm Minimization ◽  
Atomic Norm ◽  
Surrogate Function

This paper proposes a superresolution two-dimensional (2D) direction of arrival (DOA) estimation algorithm for a rectangular array based on the optimization of the atomic l0 norm and a series of relaxation formulations. The atomic l0 norm of the array response describes the minimum number of sources, which is derived from the atomic norm minimization (ANM) problem. However, the resolution is restricted and high computational complexity is incurred by using ANM for 2D angle estimation. Although an improved algorithm named decoupled atomic norm minimization (DAM) has a reduced computational burden, the resolution is still relatively low in terms of angle estimation. To overcome these limitations, we propose the direct minimization of the atomic l0 norm, which is demonstrated to be equivalent to a decoupled rank optimization problem in the positive semidefinite (PSD) form. Our goal is to solve this rank minimization problem and recover two decoupled Toeplitz matrices in which the azimuth-elevation angles of interest are encoded. Since rank minimization is an NP-hard problem, a novel sparse surrogate function is further proposed to effectively approximate the two decoupled rank functions. Then, the new optimization problem obtained through the above relaxation can be implemented via the majorization-minimization (MM) method. The proposed algorithm offers greatly improved resolution while maintaining the same computational complexity as the DAM algorithm. Moreover, it is possible to use a single snapshot for angle estimation without prior information on the number of sources, and the algorithm is robust to noise due to its iterative nature. In addition, the proposed surrogate function can achieve local convergence faster than existing functions.

A Heuristic Algorithm for Feature Selection Based on Optimization Techniques

2011 ◽  
pp. 13-26 ◽  
Author(s):  
A. M. Bagirov ◽  
A. M. Rubinov ◽  
J. Yearwood
Keyword(s):  
Feature Selection ◽  
Heuristic Algorithm ◽  
Real World ◽  
Numerical Experiments ◽  
Optimization Techniques ◽  
Selection Problem ◽  
Feature Selection Problem ◽  
Selection Of

The feature selection problem involves the selection of a subset of features that will be sufficient for the determination of structures or clusters in a given dataset and in making predictions. This chapter presents an algorithm for feature selection, which is based on the methods of optimization. To verify the effectiveness of the proposed algorithm we applied it to a number of publicly available real-world databases. The results of numerical experiments are presented and discussed. These results demonstrate that the algorithm performs well on the datasets considered.

scholarly journals ECCM Schemes against Deception Jamming Using OFDM Radar with Low Global PAPR

Sensors ◽  
2020 ◽  
Vol 20 (7) ◽  
pp. 2071
Author(s):  
Xinhai Wang ◽  
Gong Zhang ◽  
Xiangmin Wang ◽  
Qingqing Song ◽  
Fangqing Wen
Keyword(s):  
Orthogonal Frequency Division Multiplexing ◽  
Optimization Problem ◽  
Average Power ◽  
Doppler Spectrum ◽  
Frequency Division ◽  
Ofdm Symbol ◽  
Alternating Direction ◽  
Parseval’S Theorem ◽  
Joint Range ◽  
Huge Challenge

In this paper, a type of effective electronic counter-countermeasures (ECCM) technique for suppressing the high-power deception jamming using an orthogonal frequency division multiplexing (OFDM) radar is proposed. Concerning the velocity deception jamming, the initial phases of the pulses transmitted in a coherent processing interval (CPI) are designed to minimize the jamming power within a specific range, forming a notch around the jamming in the Doppler spectrum. For the purpose of suppressing the range deception jamming and the joint range-velocity deception jamming, the phase codes of the subcarriers belonging to the OFDM pulses are optimized to minimize the jamming power, distributing some specific bands in the range and the range-velocity domain, respectively. According to Parseval’s theorem, the phase encoding, acting as the coding manner of the OFDM subcarriers can ensure that the energy of each OFDM symbol stays the same. It is worth noticing that the phase codes of the OFDM subcarriers can influence the peak-to-average power ratio (PAPR). Thus, an optimization problem is formulated to optimize the phase codes of the subcarriers under the constraint of global PAPR, which can regulate the PAPRs of multiple OFDM symbols at the same time. The proposed problem is non-convex; therefore, it is a huge challenge to tackle. Then we present a method named by the phase-only alternating direction method multipliers (POADMM) to solve the aforementioned optimization problem. Some necessary simulation results are provided to demonstrate the effectiveness of the proposed radar signaling strategy

Sign in / Sign up

Export Citation Format

Share Document