scholarly journals On Memory Footprints of Partitioned Sparse Matrices

2017 ◽  
Author(s):  
Daniel Langr ◽  
Ivan Šimeček
Keyword(s):  
Sparse Matrices

scholarly journals Emerging issues in genomic selection

2021 ◽  
Author(s):  
I Misztal ◽  
I Aguilar ◽  
D Lourenco ◽  
L Ma ◽  
J Steibel ◽  
...  
Keyword(s):  
Genomic Selection ◽  
Sparse Matrices ◽  
Large Datasets ◽  
Single Step ◽  
Genomic Information ◽  
Long Term Effects ◽  
Additive Variance ◽  
Recent Developments ◽  
Emerging Issues ◽  
Using Data

Abstract Genomic selection is now practiced successfully across many species. However, many questions remain such as long-term effects, estimations of genomic parameters, robustness of GWAS with small and large datasets, and stability of genomic predictions. This study summarizes presentations from at the 2020 ASAS symposium. The focus of many studies until now is on linkage disequilibrium (LD) between two loci. Ignoring higher level equilibrium may lead to phantom dominance and epistasis. The Bulmer effect leads to a reduction of the additive variance; however, selection for increased recombination rate can release anew genetic variance. With genomic information, estimates of genetic parameters may be biased by genomic preselection, but costs of estimation can increase drastically due to the dense form of the genomic information. To make computation of estimates feasible, genotypes could be retained only for the most important animals, and methods of estimation should use algorithms that can recognize dense blocks in sparse matrices. GWAS studies using small genomic datasets frequently find many marker-trait associations whereas studies using much bigger datasets find only a few. Most current tools use very simple models for GWAS, possibly causing artifacts. These models are adequate for large datasets where pseudo-phenotypes such as deregressed proofs indirectly account for important effects for traits of interest. Artifacts arising in GWAS with small datasets can be minimized by using data from all animals (whether genotyped or not), realistic models, and methods that account for population structure. Recent developments permit computation of p-values from GBLUP, where models can be arbitrarily complex but restricted to genotyped animals only, and to single-step GBLUP that also uses phenotypes from ungenotyped animals. Stability was an important part of nongenomic evaluations, where genetic predictions were stable in the absence of new data even with low prediction accuracies. Unfortunately, genomic evaluations for such animals change because all animals with genotypes are connected. A top ranked animal can easily drop in the next evaluation, causing a crisis of confidence in genomic evaluations. While correlations between consecutive genomic evaluations are high, outliers can have differences as high as one SD. A solution to fluctuating genomic evaluations is to base selection decisions on groups of animals. While many issues in genomic selection have been solved, many new issues that require additional research continue to surface.

PARALLEL COMPUTING OF NUMERICAL SCHEMES AND BIG DATA ANALYTIC FOR SOLVING REAL LIFE APPLICATIONS

2016 ◽  
Vol 78 (8-2) ◽  
Author(s):  
Norma Alias ◽  
Nadia Nofri Yeni Suhari ◽  
Hafizah Farhah Saipan Saipol ◽  
Abdullah Aysh Dahawi ◽  
Masyitah Mohd Saidi ◽  
...  
Keyword(s):  
Big Data ◽  
Parallel Computing ◽  
Parallel Algorithm ◽  
Sparse Matrices ◽  
Real Life ◽  
Poor Performance ◽  
Equation System ◽  
Numerical Schemes ◽  
Linear Equation System ◽  
Data Analytic

This paper proposed the several real life applications for big data analytic using parallel computing software. Some parallel computing software under consideration are Parallel Virtual Machine, MATLAB Distributed Computing Server and Compute Unified Device Architecture to simulate the big data problems. The parallel computing is able to overcome the poor performance at the runtime, speedup and efficiency of programming in sequential computing. The mathematical models for the big data analytic are based on partial differential equations and obtained the large sparse matrices from discretization and development of the linear equation system. Iterative numerical schemes are used to solve the problems. Thus, the process of computational problems are summarized in parallel algorithm. Therefore, the parallel algorithm development is based on domain decomposition of problems and the architecture of difference parallel computing software. The parallel performance evaluations for distributed and shared memory architecture are investigated in terms of speedup, efficiency, effectiveness and temporal performance.

scholarly journals Sparse Recovery Using Sparse Matrices

2010 ◽  
Vol 98 (6) ◽  
pp. 937-947 ◽  
Author(s):  
Anna Gilbert ◽  
Piotr Indyk
Keyword(s):  
Sparse Matrices ◽  
Sparse Recovery

scholarly journals Amesos2 and Belos: Direct and Iterative Solvers for Large Sparse Linear Systems

2012 ◽  
Vol 20 (3) ◽  
pp. 241-255 ◽  
Author(s):  
Eric Bavier ◽  
Mark Hoemmen ◽  
Sivasankaran Rajamanickam ◽  
Heidi Thornquist
Keyword(s):  
Linear Systems ◽  
Iterative Methods ◽  
Sparse Matrix ◽  
Sparse Matrices ◽  
Direct Methods ◽  
Iterative Solvers ◽  
Software Project ◽  
Sparse Linear Systems ◽  
Scalar Type ◽  
Mixed Precision

Solvers for large sparse linear systems come in two categories: direct and iterative. Amesos2, a package in the Trilinos software project, provides direct methods, and Belos, another Trilinos package, provides iterative methods. Amesos2 offers a common interface to many different sparse matrix factorization codes, and can handle any implementation of sparse matrices and vectors, via an easy-to-extend C++ traits interface. It can also factor matrices whose entries have arbitrary “Scalar” type, enabling extended-precision and mixed-precision algorithms. Belos includes many different iterative methods for solving large sparse linear systems and least-squares problems. Unlike competing iterative solver libraries, Belos completely decouples the algorithms from the implementations of the underlying linear algebra objects. This lets Belos exploit the latest hardware without changes to the code. Belos favors algorithms that solve higher-level problems, such as multiple simultaneous linear systems and sequences of related linear systems, faster than standard algorithms. The package also supports extended-precision and mixed-precision algorithms. Together, Amesos2 and Belos form a complete suite of sparse linear solvers.

An Efficient Parallel Algorithm for Extreme Eigenvalues of Sparse Nonsymmetric Matrices

1992 ◽  
Vol 6 (1) ◽  
pp. 98-111 ◽  
Author(s):  
S. K. Kim ◽  
A. T. Chrortopoulos
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Sparse Matrices ◽  
Data Locality ◽  
Main Memory ◽  
Global Memory ◽  
Global Communication ◽  
Step Method ◽  
Arnoldi Algorithm ◽  
Large Sparse Matrices

Main memory accesses for shared-memory systems or global communications (synchronizations) in message passing systems decrease the computation speed. In this paper, the standard Arnoldi algorithm for approximating a small number of eigenvalues, with largest (or smallest) real parts for nonsymmetric large sparse matrices, is restructured so that only one synchronization point is required; that is, one global communication in a message passing distributed-memory machine or one global memory sweep in a shared-memory machine per each iteration is required. We also introduce an s-step Arnoldi method for finding a few eigenvalues of nonsymmetric large sparse matrices. This method generates reduction matrices that are similar to those generated by the standard method. One iteration of the s-step Arnoldi algorithm corresponds to s iterations of the standard Arnoldi algorithm. The s-step method has improved data locality, minimized global communication, and superior parallel properties. These algorithms are implemented on a 64-node NCUBE/7 Hypercube and a CRAY-2, and performance results are presented.

scholarly journals Application-based fault tolerance techniques for sparse matrix solvers

2017 ◽  
Vol 32 (5) ◽  
pp. 627-640
Author(s):  
Simon McIntosh–Smith ◽  
Rob Hunt ◽  
James Price ◽  
Alex Warwick Vesztrocy
Keyword(s):  
Fault Tolerance ◽  
High Performance Computing ◽  
High Performance ◽  
Sparse Matrix ◽  
Sparse Matrices ◽  
Error Correcting Codes ◽  
Computing Systems ◽  
Hardware Costs ◽  
Extreme Scale ◽  
Performance Computing

High-performance computing systems continue to increase in size in the quest for ever higher performance. The resulting increased electronic component count, coupled with the decrease in feature sizes of the silicon manufacturing processes used to build these components, may result in future exascale systems being more susceptible to soft errors caused by cosmic radiation than in current high-performance computing systems. Through the use of techniques such as hardware-based error-correcting codes and checkpoint-restart, many of these faults can be mitigated at the cost of increased hardware overhead, run-time, and energy consumption that can be as much as 10–20%. Some predictions expect these overheads to continue to grow over time. For extreme scale systems, these overheads will represent megawatts of power consumption and millions of dollars of additional hardware costs, which could potentially be avoided with more sophisticated fault-tolerance techniques. In this paper we present new software-based fault tolerance techniques that can be applied to one of the most important classes of software in high-performance computing: iterative sparse matrix solvers. Our new techniques enables us to exploit knowledge of the structure of sparse matrices in such a way as to improve the performance, energy efficiency, and fault tolerance of the overall solution.

Sign in / Sign up

Export Citation Format

Share Document