Row elimination for solving sparse linear systems and least squares problems

1976 ◽  
pp. 122-133 ◽  
Author(s):  
W. Morven Gentleman
Keyword(s):  
Least Squares ◽  
Linear Systems ◽  
Sparse Linear Systems ◽  
Least Squares Problems

Fast line searches for the robust solution of linear systems in the hybrid ℓ1 ∕ ℓ2 and Huber norms

Geophysics ◽  
2007 ◽  
Vol 72 (2) ◽  
pp. A13-A17 ◽  
Author(s):  
Kenneth P. Bube ◽  
Tamas Nemeth
Keyword(s):  
Least Squares ◽  
Linear Systems ◽  
Nonlinear Problems ◽  
Least Squares Problems ◽  
Robust Solution ◽  
Conjugate Gradient Algorithms ◽  
Line Searches ◽  
Gradient Algorithms ◽  
Linear Behavior ◽  
Nonlinear Minimization

Linear systems of equations arise in traveltime tomography, deconvolution, and many other geophysical applications. Nonlinear problems are often solved by successive linearization, leading to a sequence of linear systems. Overdetermined linear systems are solved by minimizing some measure of the size of the misfit or residual. The most commonly used measure is the [Formula: see text] norm (squared), leading to least squares problems. The advantage of least squares problems for linear systems is that they can be solved by methods (for example, [Formula: see text] factorization) that retain the linear behavior of the problem. The disadvantage of least squares solutions is that the solution is sensitive to outliers. More robust norms, approximating the [Formula: see text] norm, can be used to reduce the sensitivity to outliers. Unfortunately, these more robustnorms lead to nonlinear minimization problems, even for linear systems, and many efficient algorithms for nonlinear minimiza-tion problems require line searches. One iterative method for solving linear problems in these more robust norms is iteratively reweighted least squares (IRLS). Recently, the limited-memory Broyden, Fletcher, Goldfarb, and Shanno (BFGS) algorithm (LBFGS) has been applied efficiently to these problems. A vari-ety of nonlinear conjugate gradient algorithms (NLCG) can also be applied. LBFGS and NLCG methods require a line search in each iteration. We show that exact line searches for these meth-ods can be performed very efficiently for computing solutions to linear systems in these robust norms, thereby promoting fast con--vergence of these methods. We also compare LBFGS and NLCG (with and without exact line searches) to IRLS for a small number of iterations.

A survey of direct methods for sparse linear systems

Acta Numerica ◽  
2016 ◽  
Vol 25 ◽  
pp. 383-566 ◽  
Author(s):  
Timothy A. Davis ◽  
Sivasankaran Rajamanickam ◽  
Wissam M. Sid-Lakhdar
Keyword(s):  
Linear Systems ◽  
Sparse Matrix ◽  
Direct Methods ◽  
Theory And Practice ◽  
Sparse Linear Systems ◽  
Least Squares Problems ◽  
Know How ◽  
Survey Article ◽  
Wide Range ◽  
Matrix Problems

Wilkinson defined a sparse matrix as one with enough zeros that it pays to take advantage of them.1This informal yet practical definition captures the essence of the goal of direct methods for solving sparse matrix problems. They exploit the sparsity of a matrix to solve problems economically: much faster and using far less memory than if all the entries of a matrix were stored and took part in explicit computations. These methods form the backbone of a wide range of problems in computational science. A glimpse of the breadth of applications relying on sparse solvers can be seen in the origins of matrices in published matrix benchmark collections (Duff and Reid 1979a, Duff, Grimes and Lewis 1989a, Davis and Hu 2011). The goal of this survey article is to impart a working knowledge of the underlying theory and practice of sparse direct methods for solving linear systems and least-squares problems, and to provide an overview of the algorithms, data structures, and software available to solve these problems, so that the reader can both understand the methods and know how best to use them.

Sign in / Sign up

Export Citation Format

Share Document