AbstractIn matrix-vector multiplication, matrix symmetry does not permit a straightforward reduction in computational cost.
More generally, in contractions of symmetric tensors, the symmetries are not preserved in the usual algebraic form of contraction algorithms.
We introduce an algorithm that reduces the bilinear complexity (number of computed elementwise products) for most types of symmetric tensor contractions.
In particular, it lowers the bilinear complexity of symmetrized contractions of symmetric tensors of order {s+v} and {v+t} by a factor of {\frac{(s+t+v)!}{s!t!v!}} to leading order.
The algorithm computes a symmetric tensor of bilinear products, then subtracts unwanted parts of its partial sums.
Special cases of this algorithm provide improvements to the bilinear complexity of the multiplication of a symmetric matrix and a vector, the symmetrized vector outer product, and the symmetrized product of symmetric matrices.
While the algorithm requires more additions for each elementwise product, the total number of operations is in some cases less than classical algorithms, for tensors of any size.
We provide a round-off error analysis of the algorithm and demonstrate that the error is not too large in practice.
Finally, we provide an optimized implementation for one variant of the symmetry-preserving algorithm, which achieves speedups of up to 4.58\times for a particular tensor contraction, relative to a classical approach that casts the problem as a matrix-matrix multiplication.
Batched Sparse Matrix Multiplication for Accelerating Graph Convolutional Networks
