scholarly journals A Numerical Study of the Limited Memory BFGS Method and the Truncated-Newton Method for Large Scale Optimization

1991 ◽  
Vol 1 (3) ◽  
pp. 358-372 ◽  
Author(s):  
Stephen G. Nash ◽  
Jorge Nocedal
Keyword(s):  
Newton Method ◽  
Large Scale ◽  
Numerical Study ◽  
Large Scale Optimization ◽  
Bfgs Method ◽  
Limited Memory Bfgs Method ◽  
Limited Memory ◽  
Truncated Newton Method ◽  
Truncated Newton ◽  
Scale Optimization

scholarly journals Implementing and modifying Broyden class updates for large scale optimization

2020 ◽  
Author(s):  
Martin Buhmann ◽  
Dirk Siegel
Keyword(s):  
Large Scale ◽  
Optimization Problems ◽  
Substantial Improvement ◽  
Gradient Algorithm ◽  
Large Scale Optimization ◽  
Fixed Amount ◽  
Limited Memory ◽  
Broyden Class ◽  
Scale Optimization ◽  
Number Of Iterations

Abstract We consider Broyden class updates for large scale optimization problems in n dimensions, restricting attention to the case when the initial second derivative approximation is the identity matrix. Under this assumption we present an implementation of the Broyden class based on a coordinate transformation on each iteration. It requires only $$2nk + O(k^{2}) + O(n)$$ 2 n k + O ( k 2 ) + O ( n ) multiplications on the kth iteration and stores $$nK+ O(K^2) + O(n)$$ n K + O ( K 2 ) + O ( n ) numbers, where K is the total number of iterations. We investigate a modification of this algorithm by a scaling approach and show a substantial improvement in performance over the BFGS method. We also study several adaptations of the new implementation to the limited memory situation, presenting algorithms that work with a fixed amount of storage independent of the number of iterations. We show that one such algorithm retains the property of quadratic termination. The practical performance of the new methods is compared with the performance of Nocedal’s (Math Comput 35:773--782, 1980) method, which is considered the benchmark in limited memory algorithms. The tests show that the new algorithms can be significantly more efficient than Nocedal’s method. Finally, we show how a scaling technique can significantly improve both Nocedal’s method and the new generalized conjugate gradient algorithm.

scholarly journals A Stochastic Quasi-Newton Method for Large-Scale Optimization

2016 ◽  
Vol 26 (2) ◽  
pp. 1008-1031 ◽  
Author(s):  
R. H. Byrd ◽  
S. L. Hansen ◽  
Jorge Nocedal ◽  
Y. Singer
Keyword(s):  
Newton Method ◽  
Large Scale ◽  
Large Scale Optimization ◽  
Scale Optimization ◽  
Quasi Newton

A Modified Limited Memory BFGS Method for Non-Convex Minimization

2014 ◽  
Vol 530-531 ◽  
pp. 367-371
Author(s):  
Ting Feng Li ◽  
Yu Ting Zhang ◽  
Sheng Hui Yan
Keyword(s):  
Large Scale ◽  
Optimization Problems ◽  
Test Problems ◽  
Bfgs Method ◽  
Remarkable Feature ◽  
Limited Memory Bfgs Method ◽  
Limited Memory ◽  
Unconstrained Optimization Problems ◽  
Global Convergence Property ◽  
Large Scale Unconstrained Optimization

In this paper, a modified limited memory BFGS method for solving large-scale unconstrained optimization problems is proposed. A remarkable feature of the proposed method is that it possesses a global convergence property even without convexity assumption on the objective function. The implementations of the algorithm on CUTE test problems are reported, which suggest that a slight improvement has been achieved.

Sign in / Sign up

Export Citation Format

Share Document