scholarly journals Search What You Want: Barrier Panelty NAS for Mixed Precision Quantization

2020 ◽  
pp. 1-16
Author(s):  
Haibao Yu ◽  
Qi Han ◽  
Jianbo Li ◽  
Jianping Shi ◽  
Guangliang Cheng ◽  
...  
Keyword(s):  
Mixed Precision

Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software

2021 ◽  
Vol 47 (2) ◽  
pp. 1-28
Author(s):  
Goran Flegar ◽  
Hartwig Anzt ◽  
Terry Cojean ◽  
Enrique S. Quintana-Ortí
Keyword(s):  
Linear Algebra ◽  
Graphics Processing Units ◽  
High Performance ◽  
Numerical Algorithms ◽  
Mixed Precision ◽  
Before And After ◽  
Memory Accesses ◽  
Specialized Hardware ◽  
The Individual ◽  
Graphics Processing

The use of mixed precision in numerical algorithms is a promising strategy for accelerating scientific applications. In particular, the adoption of specialized hardware and data formats for low-precision arithmetic in high-end GPUs (graphics processing units) has motivated numerous efforts aiming at carefully reducing the working precision in order to speed up the computations. For algorithms whose performance is bound by the memory bandwidth, the idea of compressing its data before (and after) memory accesses has received considerable attention. One idea is to store an approximate operator–like a preconditioner–in lower than working precision hopefully without impacting the algorithm output. We realize the first high-performance implementation of an adaptive precision block-Jacobi preconditioner which selects the precision format used to store the preconditioner data on-the-fly, taking into account the numerical properties of the individual preconditioner blocks. We implement the adaptive block-Jacobi preconditioner as production-ready functionality in the Ginkgo linear algebra library, considering not only the precision formats that are part of the IEEE standard, but also customized formats which optimize the length of the exponent and significand to the characteristics of the preconditioner blocks. Experiments run on a state-of-the-art GPU accelerator show that our implementation offers attractive runtime savings.

Rigorous floating-point mixed-precision tuning

2017 ◽  
Author(s):  
Wei-Fan Chiang ◽  
Mark Baranowski ◽  
Ian Briggs ◽  
Alexey Solovyev ◽  
Ganesh Gopalakrishnan ◽  
...  
Keyword(s):  
Floating Point ◽  
Mixed Precision

scholarly journals GRAM

2021 ◽  
Vol 18 (2) ◽  
pp. 1-24
Author(s):  
Nhut-Minh Ho ◽  
Himeshi De silva ◽  
Weng-Fai Wong
Keyword(s):  
Performance Improvement ◽  
Trade Off ◽  
Accuracy Requirement ◽  
Output Error ◽  
Fine Grain ◽  
Mixed Precision ◽  
And Performance ◽  
Effective Use

This article presents GRAM (<underline>G</underline>PU-based <underline>R</underline>untime <underline>A</underline>daption for <underline>M</underline>ixed-precision) a framework for the effective use of mixed precision arithmetic for CUDA programs. Our method provides a fine-grain tradeoff between output error and performance. It can create many variants that satisfy different accuracy requirements by assigning different groups of threads to different precision levels adaptively at runtime . To widen the range of applications that can benefit from its approximation, GRAM comes with an optional half-precision approximate math library. Using GRAM, we can trade off precision for any performance improvement of up to 540%, depending on the application and accuracy requirement.

scholarly journals Improved Accuracy and Parallelism for MRRR-Based Eigensolvers---A Mixed Precision Approach

2014 ◽  
Vol 36 (2) ◽  
pp. C240-C263 ◽  
Author(s):  
M. Petschow ◽  
E. S. Quintana-Ortí ◽  
P. Bientinesi
Keyword(s):  
Mixed Precision ◽  
Improved Accuracy

Deep mixed precision for hyperspectral image classification

2021 ◽  
Author(s):  
M. E. Paoletti ◽  
X. Tao ◽  
J. M. Haut ◽  
S. Moreno-Álvarez ◽  
A. Plaza
Keyword(s):  
Image Classification ◽  
Hyperspectral Image ◽  
Hyperspectral Image Classification ◽  
Mixed Precision

scholarly journals Amesos2 and Belos: Direct and Iterative Solvers for Large Sparse Linear Systems

2012 ◽  
Vol 20 (3) ◽  
pp. 241-255 ◽  
Author(s):  
Eric Bavier ◽  
Mark Hoemmen ◽  
Sivasankaran Rajamanickam ◽  
Heidi Thornquist
Keyword(s):  
Linear Systems ◽  
Iterative Methods ◽  
Sparse Matrix ◽  
Sparse Matrices ◽  
Direct Methods ◽  
Iterative Solvers ◽  
Software Project ◽  
Sparse Linear Systems ◽  
Scalar Type ◽  
Mixed Precision

Solvers for large sparse linear systems come in two categories: direct and iterative. Amesos2, a package in the Trilinos software project, provides direct methods, and Belos, another Trilinos package, provides iterative methods. Amesos2 offers a common interface to many different sparse matrix factorization codes, and can handle any implementation of sparse matrices and vectors, via an easy-to-extend C++ traits interface. It can also factor matrices whose entries have arbitrary “Scalar” type, enabling extended-precision and mixed-precision algorithms. Belos includes many different iterative methods for solving large sparse linear systems and least-squares problems. Unlike competing iterative solver libraries, Belos completely decouples the algorithms from the implementations of the underlying linear algebra objects. This lets Belos exploit the latest hardware without changes to the code. Belos favors algorithms that solve higher-level problems, such as multiple simultaneous linear systems and sequences of related linear systems, faster than standard algorithms. The package also supports extended-precision and mixed-precision algorithms. Together, Amesos2 and Belos form a complete suite of sparse linear solvers.

Sign in / Sign up

Export Citation Format

Share Document