A survey of numerical linear algebra methods utilizing mixed-precision arithmetic

2021 ◽  
pp. 109434202110033 ◽  
Author(s):  
Ahmad Abdelfattah ◽  
Hartwig Anzt ◽  
Erik G Boman ◽  
Erin Carson ◽  
Terry Cojean ◽  
...  
Keyword(s):  
Linear Algebra ◽  
Special Function ◽  
Numerical Algorithms ◽  
Theoretical Background ◽  
Numerical Linear Algebra ◽  
Efficient Utilization ◽  
Sparse Linear Algebra ◽  
Mixed Precision ◽  
Machine Learning Applications ◽  
Comprehensive Survey

The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.

Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software

2021 ◽  
Vol 47 (2) ◽  
pp. 1-28
Author(s):  
Goran Flegar ◽  
Hartwig Anzt ◽  
Terry Cojean ◽  
Enrique S. Quintana-Ortí
Keyword(s):  
Linear Algebra ◽  
Graphics Processing Units ◽  
High Performance ◽  
Numerical Algorithms ◽  
Mixed Precision ◽  
Before And After ◽  
Memory Accesses ◽  
Specialized Hardware ◽  
The Individual ◽  
Graphics Processing

The use of mixed precision in numerical algorithms is a promising strategy for accelerating scientific applications. In particular, the adoption of specialized hardware and data formats for low-precision arithmetic in high-end GPUs (graphics processing units) has motivated numerous efforts aiming at carefully reducing the working precision in order to speed up the computations. For algorithms whose performance is bound by the memory bandwidth, the idea of compressing its data before (and after) memory accesses has received considerable attention. One idea is to store an approximate operator–like a preconditioner–in lower than working precision hopefully without impacting the algorithm output. We realize the first high-performance implementation of an adaptive precision block-Jacobi preconditioner which selects the precision format used to store the preconditioner data on-the-fly, taking into account the numerical properties of the individual preconditioner blocks. We implement the adaptive block-Jacobi preconditioner as production-ready functionality in the Ginkgo linear algebra library, considering not only the precision formats that are part of the IEEE standard, but also customized formats which optimize the length of the exponent and significand to the characteristics of the preconditioner blocks. Experiments run on a state-of-the-art GPU accelerator show that our implementation offers attractive runtime savings.

scholarly journals Some algorithms for maximum volume and cross approximation of symmetric semidefinite matrices

2021 ◽  
Author(s):  
Stefano Massei
Keyword(s):  
Computer Science ◽  
Recent Work ◽  
Linear Algebra ◽  
Optimization Problems ◽  
Approximation Error ◽  
Positive Semidefinite ◽  
Numerical Linear Algebra ◽  
Maximum Volume ◽  
Deterministic Algorithms ◽  
Principal Submatrix

AbstractVarious applications in numerical linear algebra and computer science are related to selecting the $$r\times r$$ r × r submatrix of maximum volume contained in a given matrix $$A\in \mathbb R^{n\times n}$$ A ∈ R n × n . We propose a new greedy algorithm of cost $$\mathcal O(n)$$ O ( n ) , for the case A symmetric positive semidefinite (SPSD) and we discuss its extension to related optimization problems such as the maximum ratio of volumes. In the second part of the paper we prove that any SPSD matrix admits a cross approximation built on a principal submatrix whose approximation error is bounded by $$(r+1)$$ ( r + 1 ) times the error of the best rank r approximation in the nuclear norm. In the spirit of recent work by Cortinovis and Kressner we derive some deterministic algorithms, which are capable to retrieve a quasi optimal cross approximation with cost $$\mathcal O(n^3)$$ O ( n 3 ) .

scholarly journals Numerical Linear Algebra in Signal Processing Applications

2007 ◽  
Vol 2007 (1) ◽  
Author(s):  
Nicola Mastronardi ◽  
Gene H Golub ◽  
Shivkumar Chandrasekaran ◽  
Marc Moonen ◽  
Paul Van Dooren ◽  
...  
Keyword(s):  
Signal Processing ◽  
Linear Algebra ◽  
Numerical Linear Algebra
Sign in / Sign up

Export Citation Format

Share Document