scholarly journals A hybrid method for nonlinear least squares that uses quasi-Newton updates applied to an approximation of the Jacobian matrix

2019 ◽  
Author(s):  
Ladislav Lukšan ◽  
Jan Vlček
Keyword(s):  
Least Squares ◽  
Hybrid Method ◽  
Jacobian Matrix ◽  
Nonlinear Least Squares ◽  
Quasi Newton

scholarly journals A new limited memory method for unconstrained nonlinear least squares

2021 ◽  
Author(s):  
Morteza Kimiaei ◽  
Arnold Neumaier
Keyword(s):  
Least Squares ◽  
Jacobian Matrix ◽  
Trust Region ◽  
Nonlinear Least Squares ◽  
Black Box ◽  
Least Squares Problems ◽  
Trust Region Algorithm ◽  
Limited Memory ◽  
Adaptive Radius ◽  
Quasi Newton

AbstractThis paper suggests a new limited memory trust region algorithm for large unconstrained black box least squares problems, called LMLS. Main features of LMLS are a new non-monotone technique, a new adaptive radius strategy, a new Broyden-like algorithm based on the previous good points, and a heuristic estimation for the Jacobian matrix in a subspace with random basis indices. Our numerical results show that LMLS is robust and efficient, especially in comparison with solvers using traditional limited memory and standard quasi-Newton approximations.

scholarly journals A Review of Well-Known Robust Line Search and Trust Region Numerical Optimization Algorithms for Solving Nonlinear Least-Squares Problems

2021 ◽  
Vol 2 (3) ◽  
pp. 1-17
Author(s):  
Jacques SABITI KISETA ◽  
Roger LIENDI AKUMOSO
Keyword(s):  
Least Squares ◽  
Numerical Optimization ◽  
Line Search ◽  
Trust Region ◽  
Nonlinear Least Squares ◽  
Likelihood Estimation ◽  
Nonlinear Regression Models ◽  
Line Search Techniques ◽  
Online Searches ◽  
Quasi Newton

The conditional, unconditional, or the exact maximum likelihood estimation and the least-squares estimation involve minimizing either the conditional or the unconditional residual sum of squares. The maximum likelihood estimation (MLE) approach and the nonlinear least squares (NLS) procedure involve an iterative search technique for obtaining global rather than local optimal estimates. Several authors have presented brief overviews of algorithms for solving NLS problems. Snezana S. Djordjevic (2019) presented a review of some unconstrained optimization methods based on the line search techniques. Mahaboob et al. (2017) proposed a different approach to estimate nonlinear regression models using numerical methods also based on the line search techniques. Mohammad, Waziri, and Santos (2019) have briefly reviewed methods for solving NLS problems, paying special attention to the structured quasi-Newton methods which are the family of the search line techniques. Ya-Xiang Yuan (2011) reviewed some recent results on numerical methods for nonlinear equations and NLS problems based on online searches and trust regions techniques, particularly on Levenberg-Marquardt type methods, quasi-Newton type methods, and trust regions algorithms. The purpose of this paper is to review some online searches and trust region's more well-known robust numerical optimization algorithms and the most used in practice for the estimation of time series models and other nonlinear regression models. The line searches algorithms considered are: Gradient algorithm, Steepest Descent (SD) algorithm, Newton-Raphson (NR) algorithm, Murray’s algorithm, Quasi-Newton (QN) algorithm, Gauss-Newton (GN) algorithm, Fletcher and Powell algorithm (FP), Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. While the only trust-region algorithm considered is the Levenberg-Marquardt (LM) algorithm. We also give some main advantages and disadvantages of these different algorithms.

scholarly journals A Superlinearly Convergent Penalty Method with Nonsmooth Line Search for Constrained Nonlinear Least Squares

2012 ◽  
Vol 16 ◽  
pp. 103 ◽  
Author(s):  
Nezam Mahdavi-Amiri ◽  
Mohammad Reza Ansari
Keyword(s):  
Least Squares ◽  
Penalty Method ◽  
Line Search ◽  
Search Strategy ◽  
Nonlinear Least Squares ◽  
Strategy Implementation ◽  
Test Problems ◽  
Least Squares Problems ◽  
Comparative Results ◽  
Quasi Newton

Recently, we have presented a projected structured algorithm for solving constrained nonlinear least squares problems, and established its local two-step Q-superlinear convergence. The approach is based on an adaptive structured scheme due to Mahdavi-Amiri and Bartels of the exact penalty method. The structured adaptation also makes use of the ideas of Nocedal and Overton for handling the quasi-Newton updates of projected Hessians and appropriates the structuring scheme of Dennis, Martinez and Tapia. Here, for robustness, we present a specific nonsmooth line search strategy, taking account of the least squares objective. We also discuss the details of our new nonsmooth line search strategy, implementation details of the algorithm, and provide comparative results obtained by the testing of our program and three nonlinear programming codes from KNITRO on test problems (both small and large residuals) from Hock and Schittkowski, Lukšan and Vlček and some randomly generated ones due to Bartels and Mahdavi-Amiri. The results indeed affirm the practical relevance of our special considerations for the inherent structure of the least squares.    

Sign in / Sign up

Export Citation Format

Share Document