Verified Bounds for Least Squares Problems and Underdetermined Linear Systems

2012 ◽  
Vol 33 (1) ◽  
pp. 130-148 ◽  
Author(s):  
Siegfried M. Rump
Keyword(s):  
Least Squares ◽  
Linear Systems ◽  
Least Squares Problems ◽  
Underdetermined Linear Systems ◽  
Verified Bounds

Fast line searches for the robust solution of linear systems in the hybrid ℓ1 ∕ ℓ2 and Huber norms

Geophysics ◽  
2007 ◽  
Vol 72 (2) ◽  
pp. A13-A17 ◽  
Author(s):  
Kenneth P. Bube ◽  
Tamas Nemeth
Keyword(s):  
Least Squares ◽  
Linear Systems ◽  
Nonlinear Problems ◽  
Least Squares Problems ◽  
Robust Solution ◽  
Conjugate Gradient Algorithms ◽  
Line Searches ◽  
Gradient Algorithms ◽  
Linear Behavior ◽  
Nonlinear Minimization

Linear systems of equations arise in traveltime tomography, deconvolution, and many other geophysical applications. Nonlinear problems are often solved by successive linearization, leading to a sequence of linear systems. Overdetermined linear systems are solved by minimizing some measure of the size of the misfit or residual. The most commonly used measure is the [Formula: see text] norm (squared), leading to least squares problems. The advantage of least squares problems for linear systems is that they can be solved by methods (for example, [Formula: see text] factorization) that retain the linear behavior of the problem. The disadvantage of least squares solutions is that the solution is sensitive to outliers. More robust norms, approximating the [Formula: see text] norm, can be used to reduce the sensitivity to outliers. Unfortunately, these more robustnorms lead to nonlinear minimization problems, even for linear systems, and many efficient algorithms for nonlinear minimiza-tion problems require line searches. One iterative method for solving linear problems in these more robust norms is iteratively reweighted least squares (IRLS). Recently, the limited-memory Broyden, Fletcher, Goldfarb, and Shanno (BFGS) algorithm (LBFGS) has been applied efficiently to these problems. A vari-ety of nonlinear conjugate gradient algorithms (NLCG) can also be applied. LBFGS and NLCG methods require a line search in each iteration. We show that exact line searches for these meth-ods can be performed very efficiently for computing solutions to linear systems in these robust norms, thereby promoting fast con--vergence of these methods. We also compare LBFGS and NLCG (with and without exact line searches) to IRLS for a small number of iterations.

scholarly journals Alternative structured spectral gradient algorithms for solving nonlinear least-squares problems

Heliyon ◽  
2021 ◽  
pp. e07499
Author(s):  
Mahmoud Muhammad Yahaya ◽  
Poom Kumam ◽  
Aliyu Muhammed Awwal ◽  
Sani Aji
Keyword(s):  
Least Squares ◽  
Nonlinear Least Squares ◽  
Least Squares Problems ◽  
Gradient Algorithms
Sign in / Sign up

Export Citation Format

Share Document