Lower bounds for the multiplicative complexity of matrix multiplication

M. Bläser

doi:10.1007/s000370050028

Communication lower bounds and optimal algorithms for numerical linear algebra

Acta Numerica ◽

10.1017/s0962492914000038 ◽

2014 ◽

Vol 23 ◽

pp. 1-155 ◽

Cited By ~ 38

Author(s):

G. Ballard ◽

E. Carson ◽

J. Demmel ◽

M. Hoemmen ◽

N. Knight ◽

...

Keyword(s):

Linear Algebra ◽

Lower Bounds ◽

Krylov Subspace ◽

Sparse Matrices ◽

Arithmetic Operation ◽

Matrix Multiplication ◽

Direct Methods ◽

Numerical Linear Algebra ◽

Theory And Practice ◽

Large Speed

The traditional metric for the efficiency of a numerical algorithm has been the number of arithmetic operations it performs. Technological trends have long been reducing the time to perform an arithmetic operation, so it is no longer the bottleneck in many algorithms; rather, communication, or moving data, is the bottleneck. This motivates us to seek algorithms that move as little data as possible, either between levels of a memory hierarchy or between parallel processors over a network. In this paper we summarize recent progress in three aspects of this problem. First we describe lower bounds on communication. Some of these generalize known lower bounds for dense classical (O(n3)) matrix multiplication to all direct methods of linear algebra, to sequential and parallel algorithms, and to dense and sparse matrices. We also present lower bounds for Strassen-like algorithms, and for iterative methods, in particular Krylov subspace methods applied to sparse matrices. Second, we compare these lower bounds to widely used versions of these algorithms, and note that these widely used algorithms usually communicate asymptotically more than is necessary. Third, we identify or invent new algorithms for most linear algebra problems that do attain these lower bounds, and demonstrate large speed-ups in theory and practice.

Download Full-text

Lower bounds for depth 4 formulas computing iterated matrix multiplication

Proceedings of the 46th Annual ACM Symposium on Theory of Computing - STOC '14 ◽

10.1145/2591796.2591824 ◽

2014 ◽

Cited By ~ 19

Author(s):

Hervé Fournier ◽

Nutan Limaye ◽

Guillaume Malod ◽

Srikanth Srinivasan

Keyword(s):

Lower Bounds ◽

Matrix Multiplication ◽

Iterated Matrix Multiplication

Download Full-text

Lower Bounds for Depth-4 Formulas Computing Iterated Matrix Multiplication

SIAM Journal on Computing ◽

10.1137/140990280 ◽

2015 ◽

Vol 44 (5) ◽

pp. 1173-1201 ◽

Cited By ~ 3

Author(s):

Hervé Fournier ◽

Nutan Limaye ◽

Guillaume Malod ◽

Srikanth Srinivasan

Keyword(s):

Lower Bounds ◽

Matrix Multiplication ◽

Iterated Matrix Multiplication

Download Full-text

Algorithms meeting the lower bounds on the multiplicative complexity of length-2/sup n/ DFTs and their connection with practical algorithms

IEEE Transactions on Acoustics Speech and Signal Processing ◽

10.1109/29.60070 ◽

1990 ◽

Vol 38 (9) ◽

pp. 1504-1511 ◽

Cited By ~ 32

Author(s):

P. Duhamel

Keyword(s):

Lower Bounds ◽

Multiplicative Complexity ◽

Practical Algorithms

Download Full-text

Lower Bounds for Dynamic Algebraic Problems

BRICS Report Series ◽

10.7146/brics.v5i11.19283 ◽

1998 ◽

Vol 5 (11) ◽

Author(s):

Gudmund Skovbjerg Frandsen ◽

Johan P. Hansen ◽

Peter Bro Miltersen

Keyword(s):

Fourier Transform ◽

Lower Bound ◽

Lower Bounds ◽

Communication Complexity ◽

Symmetric Functions ◽

Matrix Multiplication ◽

Worst Case ◽

Dynamic Matrix ◽

Wide Range ◽

On Line

We consider dynamic evaluation of algebraic functions (matrix multiplication, determinant, convolution, Fourier transform, etc.) in the model of Reif and Tate; i.e., if f(x1, . . . , xn) = (y1, . . . , ym) is an algebraic problem, we consider serving on-line requests of the form "change input xi to value v" or "what is the value of output yi?". We present techniques for showing lower bounds on the worst case time complexity per operation for such problems. The first gives lower bounds in a wide range of rather powerful models (for instance history dependent<br />algebraic computation trees over any infinite subset of a field, the integer RAM, and the generalized real RAM model of Ben-Amram and Galil). Using this technique, we show optimal Omega(n) bounds for dynamic matrix-vector product, dynamic matrix multiplication and dynamic discriminant and an <br />Omega(sqrt(n)) lower bound for dynamic polynomial multiplication (convolution), providing a good match with Reif and<br />Tate's O(sqrt(n log n)) upper bound. We also show linear lower bounds for dynamic determinant, matrix adjoint and matrix inverse and an Omega(sqrt(n)) lower bound for the elementary symmetric functions. The second technique is the communication complexity technique of Miltersen, Nisan, Safra, and Wigderson which we apply to the setting<br />of dynamic algebraic problems, obtaining similar lower bounds in the word RAM model. The third technique gives lower bounds in the weaker straight line program model. Using this technique, we show an ((log n)2= log log n) lower bound for dynamic discrete Fourier transform. Technical ingredients of our techniques are the incompressibility technique of Ben-Amram and Galil and the lower bound for depth-two superconcentrators of Radhakrishnan and Ta-Shma. The incompressibility technique is extended to arithmetic computation in arbitrary fields.

Download Full-text