GPU Preconditioning for Block Linear Systems Using Block Incomplete Sparse Approximate Inverses

Solving sparse triangular systems is the building block for incomplete LU- (ILU-) based preconditioning, but parallel algorithms, such as the level-scheduling scheme, are sometimes limited by available parallelism extracted from the sparsity pattern. In this study, the block version of the incomplete sparse approximate inverses (ISAI) algorithm is studied, and the block-ISAI is considered for preconditioning by proposing an efficient algorithm and implementation on graphical processing unit (GPU) accelerators. Performance comparisons are carried out between the proposed algorithm and serial and parallel block triangular solvers from PETSc and cuSPARSE libraries. The experimental results show that GMRES (30) with the proposed block-ISAI preconditioning achieves accelerations 1.4 × –6.9 × speedups over that using the cuSPARSE library on NVIDIA Tesla V100 GPU.

A New Algorithm for Solving Large-Scale Generalized Eigenvalue Problem Based on Projection Methods

In this paper, we consider four methods for determining certain eigenvalues and corresponding eigenvectors of large-scale generalized eigenvalue problems which are located in a certain region. In these methods, a small pencil that contains only the desired eigenvalue is derived using moments that have obtained via numerical integration. Our purpose is to improve the numerical stability of the moment-based method and compare its stability with three other methods. Numerical examples show that the block version of the moment-based (SS) method with the Rayleigh–Ritz procedure has higher numerical stability than respect to other methods.

Non-Ergodic Convergence Analysis of Heavy-Ball Algorithms

In this paper, we revisit the convergence of the Heavy-ball method, and present improved convergence complexity results in the convex setting. We provide the first non-ergodic O(1/k) rate result of the Heavy-ball algorithm with constant step size for coercive objective functions. For objective functions satisfying a relaxed strongly convex condition, the linear convergence is established under weaker assumptions on the step size and inertial parameter than made in the existing literature. We extend our results to multi-block version of the algorithm with both the cyclic and stochastic update rules. In addition, our results can also be extended to decentralized optimization, where the ergodic analysis is not applicable.

Block Power Method for SVD Decomposition

We present in this paper a new method to determine the k largest singular values and their corresponding singular vectors for real rectangular matrices A ∈ Rn×m. Our approach is based on using a block version of the Power Method to compute an k-block SV D decomposition: Ak = Uk∑kVkT , where ∑k is a diagonal matrix with the k largest non-negative, monotonically decreasing diagonal σ1≥ σ2 ⋯ ≥ σk. Uk and Vk are orthogonal matrices whose columns are the left and right singular vectors of the k largest singular values. This approach is more efficient as there is no need of calculation of all singular values. The QR method is also presented to obtain the SV D decomposition.