scholarly journals SURAA: A Novel Method and Tool for Loadbalanced and Coalesced SpMV Computations on GPUs

2019 ◽  
Vol 9 (5) ◽  
pp. 947 ◽  
Author(s):  
Thaha Muhammed ◽  
Rashid Mehmood ◽  
Aiiad Albeshri ◽  
Iyad Katib
Keyword(s):  
Load Balancing ◽  
Graphics Processing Units ◽  
Sparse Matrix ◽  
Memory Access ◽  
Group Matrix ◽  
The Matrix ◽  
Novel Method ◽  
Coalesced Memory ◽  
Graphics Processing ◽  
Matrix Vector

Sparse matrix-vector (SpMV) multiplication is a vital building block for numerous scientific and engineering applications. This paper proposes SURAA (translates to speed in arabic), a novel method for SpMV computations on graphics processing units (GPUs). The novelty lies in the way we group matrix rows into different segments, and adaptively schedule various segments to different types of kernels. The sparse matrix data structure is created by sorting the rows of the matrix on the basis of the nonzero elements per row ( n p r) and forming segments of equal size (containing approximately an equal number of nonzero elements per row) using the Freedman–Diaconis rule. The segments are assembled into three groups based on the mean n p r of the segments. For each group, we use multiple kernels to execute the group segments on different streams. Hence, the number of threads to execute each segment is adaptively chosen. Dynamic Parallelism available in Nvidia GPUs is utilized to execute the group containing segments with the largest mean n p r, providing improved load balancing and coalesced memory access, and hence more efficient SpMV computations on GPUs. Therefore, SURAA minimizes the adverse effects of the n p r variance by uniformly distributing the load using equal sized segments. We implement the SURAA method as a tool and compare its performance with the de facto best commercial (cuSPARSE) and open source (CUSP, MAGMA) tools using widely used benchmarks comprising 26 high n p r v a r i a n c e matrices from 13 diverse domains. SURAA outperforms the other tools by delivering 13.99x speedup on average. We believe that our approach provides a fundamental shift in addressing SpMV related challenges on GPUs including coalesced memory access, thread divergence, and load balancing, and is set to open new avenues for further improving SpMV performance in the future.

scholarly journals Parallel forming of preconditioners based on the approximation of the Sherman-Morrison inversion formula

2015 ◽  
pp. 86-93
Author(s):  
А.К. Новиков ◽  
C.П. Копысов ◽  
Н.С. Недожогин
Keyword(s):  
Parallel Algorithm ◽  
Conjugate Gradient ◽  
Graphics Processing Units ◽  
Inversion Formula ◽  
Matrix Approximation ◽  
The Matrix ◽  
New Form ◽  
Graphics Processing ◽  
Matrix Vector ◽  
Parallelization Efficiency

Исследуются возможности ускорения предобусловленных методов бисопряженных градиентов (BiCGStab, Bi-Conjugate Gradient Stabilized) с предобусловливателем на основе аппроксимации обращения матрицы по формуле Шермана-Моррисона. Рассмотрена новая форма параллельного алгоритма, использующая матрично-векторные произведения при формирования матриц предобусловливателя. Показана эффективность распараллеливания наиболее ресурсоемких операций этого предобусловливателя на графических процессорах. Acceleration of preconditioned bi-conjugate gradient stabilized (BiCGStab) methods with preconditioners based on the matrix approximation by the Sherman-Morrison inversion formula is studied. A new form of the parallel algorithm using matrix-vector products to generate preconditioning matrices is proposed. A parallelization efficiency of the most resource-intensive operations of such preconditioners on multi-core central and graphics processing units (CPUs and GPUs) is shown.

scholarly journals Performance Analysis of Sparse Matrix-Vector Multiplication (SpMV) on Graphics Processing Units (GPUs)

Electronics ◽  
2020 ◽  
Vol 9 (10) ◽  
pp. 1675
Author(s):  
Sarah AlAhmadi ◽  
Thaha Mohammed ◽  
Aiiad Albeshri ◽  
Iyad Katib ◽  
Rashid Mehmood
Keyword(s):  
Performance Analysis ◽  
Graphics Processing Units ◽  
High Performance ◽  
Heterogeneous Computing ◽  
Performance Metrics ◽  
Sparse Matrix ◽  
Extensive Literature ◽  
Compressed Sparse Row ◽  
Graphics Processing ◽  
Matrix Vector

Graphics processing units (GPUs) have delivered a remarkable performance for a variety of high performance computing (HPC) applications through massive parallelism. One such application is sparse matrix-vector (SpMV) computations, which is central to many scientific, engineering, and other applications including machine learning. No single SpMV storage or computation scheme provides consistent and sufficiently high performance for all matrices due to their varying sparsity patterns. An extensive literature review reveals that the performance of SpMV techniques on GPUs has not been studied in sufficient detail. In this paper, we provide a detailed performance analysis of SpMV performance on GPUs using four notable sparse matrix storage schemes (compressed sparse row (CSR), ELLAPCK (ELL), hybrid ELL/COO (HYB), and compressed sparse row 5 (CSR5)), five performance metrics (execution time, giga floating point operations per second (GFLOPS), achieved occupancy, instructions per warp, and warp execution efficiency), five matrix sparsity features (nnz, anpr, nprvariance, maxnpr, and distavg), and 17 sparse matrices from 10 application domains (chemical simulations, computational fluid dynamics (CFD), electromagnetics, linear programming, economics, etc.). Subsequently, based on the deeper insights gained through the detailed performance analysis, we propose a technique called the heterogeneous CPU–GPU Hybrid (HCGHYB) scheme. It utilizes both the CPU and GPU in parallel and provides better performance over the HYB format by an average speedup of 1.7x. Heterogeneous computing is an important direction for SpMV and other application areas. Moreover, to the best of our knowledge, this is the first work where the SpMV performance on GPUs has been discussed in such depth. We believe that this work on SpMV performance analysis and the heterogeneous scheme will open up many new directions and improvements for the SpMV computing field in the future.

Selecting optimal SpMV realizations for GPUs via machine learning

2021 ◽  
pp. 109434202199073
Author(s):  
Ernesto Dufrechou ◽  
Pablo Ezzatti ◽  
Enrique S Quintana-Ortí
Keyword(s):  
Machine Learning ◽  
Sparse Matrix ◽  
Machine Learning Techniques ◽  
Optimal Method ◽  
Learning Techniques ◽  
General Rules ◽  
Machine Learning Approach ◽  
The Matrix ◽  
Time And Energy ◽  
Matrix Vector

More than 10 years of research related to the development of efficient GPU routines for the sparse matrix-vector product (SpMV) have led to several realizations, each with its own strengths and weaknesses. In this work, we review some of the most relevant efforts on the subject, evaluate a few prominent routines that are publicly available using more than 3000 matrices from different applications, and apply machine learning techniques to anticipate which SpMV realization will perform best for each sparse matrix on a given parallel platform. Our numerical experiments confirm the methods offer such varied behaviors depending on the matrix structure that the identification of general rules to select the optimal method for a given matrix becomes extremely difficult, though some useful strategies (heuristics) can be defined. Using a machine learning approach, we show that it is possible to obtain unexpensive classifiers that predict the best method for a given sparse matrix with over 80% accuracy, demonstrating that this approach can deliver important reductions in both execution time and energy consumption.

scholarly journals A distributed computing approach to improve the performance of the Parallel Ocean Program (v2.1)

2014 ◽  
Vol 7 (1) ◽  
pp. 267-281 ◽  
Author(s):  
B. van Werkhoven ◽  
J. Maassen ◽  
M. Kliphuis ◽  
H. A. Dijkstra ◽  
S. E. Brunnabend ◽  
...  
Keyword(s):  
Distributed Computing ◽  
Load Balancing ◽  
Ocean Circulation ◽  
Graphics Processing Units ◽  
Block Partitioning ◽  
Model Code ◽  
Graphics Processing ◽  
Computing Approach ◽  
Parallel Ocean Program

Abstract. The Parallel Ocean Program (POP) is used in many strongly eddying ocean circulation simulations. Ideally it would be desirable to be able to do thousand-year-long simulations, but the current performance of POP prohibits these types of simulations. In this work, using a new distributed computing approach, two methods to improve the performance of POP are presented. The first is a block-partitioning scheme for the optimization of the load balancing of POP such that it can be run efficiently in a multi-platform setting. The second is the implementation of part of the POP model code on graphics processing units (GPUs). We show that the combination of both innovations also leads to a substantial performance increase when running POP simultaneously over multiple computational platforms.

scholarly journals Sparse matrix partitioning for optimizing SpMV on CPU-GPU heterogeneous platforms

2019 ◽  
Vol 34 (1) ◽  
pp. 66-80 ◽  
Author(s):  
Akrem Benatia ◽  
Weixing Ji ◽  
Yizhuo Wang ◽  
Feng Shi
Keyword(s):  
Graphics Processing Units ◽  
Sparse Matrix ◽  
Sparse Matrices ◽  
Heterogeneous Systems ◽  
Input Matrix ◽  
Heterogeneous Platforms ◽  
Mapping Algorithm ◽  
Matrix Vector Multiplication ◽  
Graphics Processing ◽  
Matrix Partitioning

Sparse matrix–vector multiplication (SpMV) kernel dominates the computing cost in numerous applications. Most of the existing studies dedicated to improving this kernel have been targeting just one type of processing units, mainly multicore CPUs or graphics processing units (GPUs), and have not explored the potential of the recent, rapidly emerging, CPU-GPU heterogeneous platforms. To take full advantage of these heterogeneous systems, the input sparse matrix has to be partitioned on different available processing units. The partitioning problem is more challenging with the existence of many sparse formats whose performances depend both on the sparsity of the input matrix and the used hardware. Thus, the best performance does not only depend on how to partition the input sparse matrix but also on which sparse format to use for each partition. To address this challenge, we propose in this article a new CPU-GPU heterogeneous method for computing the SpMV kernel that combines between different sparse formats to achieve better performance and better utilization of CPU-GPU heterogeneous platforms. The proposed solution horizontally partitions the input matrix into multiple block-rows and predicts their best sparse formats using machine learning-based performance models. A mapping algorithm is then used to assign the block-rows to the CPU and GPU(s) available in the system. Our experimental results using real-world large unstructured sparse matrices on two different machines show a noticeable performance improvement.

scholarly journals A distributed computing approach to improve the performance of the Parallel Ocean Program (v2.1)

2013 ◽  
Vol 6 (3) ◽  
pp. 4705-4744 ◽  
Author(s):  
B. van Werkhoven ◽  
J. Maassen ◽  
M. Kliphuis ◽  
H. A. Dijkstra ◽  
S. E. Brunnabend ◽  
...  
Keyword(s):  
Distributed Computing ◽  
Load Balancing ◽  
Ocean Circulation ◽  
Graphics Processing Units ◽  
Block Partitioning ◽  
Model Code ◽  
Graphics Processing ◽  
Computing Approach ◽  
Parallel Ocean Program

Abstract. The Parallel Ocean Program (POP) is used in many strongly eddying ocean circulation simulations. Ideally one would like to do thousand-year long simulations, but the current performance of POP prohibits this type of simulations. In this work, using a new distributed computing approach, two innovations to improve the performance of POP are presented. The first is a new block partitioning scheme for the optimization of the load balancing of POP such that it can be run efficiently in a multi-platform setting. The second is an implementation of part of the POP model code on Graphics Processing Units. We show that the combination of both innovations leads to a substantial performance increase also when running POP simultaneously over multiple computational platforms.

GPU Accelerated Reconstruction in Compton Scattering Tomography Using Matrix Compression

2014 ◽  
Vol 519-520 ◽  
pp. 102-107
Author(s):  
Yu Fei Yu ◽  
Bin Yan ◽  
Biao Wang ◽  
Lei Li ◽  
Yu Han ◽  
...  
Keyword(s):  
Compton Scattering ◽  
Graphics Processing Unit ◽  
Sparse Matrix ◽  
Reconstruction Algorithm ◽  
Processing Unit ◽  
Matrix Vector Multiplication ◽  
Speedup Ratio ◽  
Parallel Features ◽  
Graphics Processing ◽  
Matrix Vector

An acceleration strategy for TV-ADM reconstruction algorithm in Compton scattering tomography (CST) is proposed. By analyzing the sparse characteristic of CST projection matrixes, firstly, the sparse matrix vector CSR format and ELL format are used to store them, which greatly reduce the memory consumption. Then, a Sparse Matrix Vector multiplication (SpMV) method is utilized to accelerate the projector and back projector process. Finally, based on the parallel features, the TV-ADM is computed with Graphics Processing Unit (GPU). Numerical experiments show that the TV-ADM with the presented acceleration strategy could achieve a 96 times speedup ratio and 224 times memory compression ratio without precision loss.

Sign in / Sign up

Export Citation Format

Share Document