scholarly journals Projected Newton Methods for Optimization Problems with Simple Constraints

1982 ◽  
Vol 20 (2) ◽  
pp. 221-246 ◽  
Author(s):  
Dimitri P. Bertsekas
Keyword(s):  
Optimization Problems ◽  
Newton Methods

scholarly journals Exact and inexact subsampled Newton methods for optimization

2018 ◽  
Vol 39 (2) ◽  
pp. 545-578 ◽  
Author(s):  
Raghu Bollapragada ◽  
Richard H Byrd ◽  
Jorge Nocedal
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Rate Of Convergence ◽  
Conjugate Gradient ◽  
Complexity Analysis ◽  
Optimization Problems ◽  
Inexact Newton Method ◽  
Newton Methods ◽  
Machine Learning Applications

Abstract The paper studies the solution of stochastic optimization problems in which approximations to the gradient and Hessian are obtained through subsampling. We first consider Newton-like methods that employ these approximations and discuss how to coordinate the accuracy in the gradient and Hessian to yield a superlinear rate of convergence in expectation. The second part of the paper analyzes an inexact Newton method that solves linear systems approximately using the conjugate gradient (CG) method, and that samples the Hessian and not the gradient (the gradient is assumed to be exact). We provide a complexity analysis for this method based on the properties of the CG iteration and the quality of the Hessian approximation, and compare it with a method that employs a stochastic gradient iteration instead of the CG method. We report preliminary numerical results that illustrate the performance of inexact subsampled Newton methods on machine learning applications based on logistic regression.

Quasi-Newton methods for multiobjective optimization problems

4OR ◽  
2017 ◽  
Vol 16 (3) ◽  
pp. 261-294 ◽  
Author(s):  
Vahid Morovati ◽  
Hadi Basirzadeh ◽  
Latif Pourkarimi
Keyword(s):  
Multiobjective Optimization ◽  
Optimization Problems ◽  
Newton Methods ◽  
Multiobjective Optimization Problems ◽  
Quasi Newton
Sign in / Sign up

Export Citation Format

Share Document