Accelerating NWChem Coupled Cluster through dataflow-based execution

2017 ◽  
Vol 32 (4) ◽  
pp. 540-551 ◽  
Author(s):  
Heike Jagode ◽  
Anthony Danalis ◽  
Jack Dongarra
Keyword(s):  
High Performance ◽  
Coupled Cluster ◽  
Many Body ◽  
Mass Conversion ◽  
Many Body Systems ◽  
Computationally Intensive ◽  
Scalable Computation ◽  
Cluster Methods ◽  
Performance Computing ◽  
Chemistry Community

Numerical techniques used for describing many-body systems, such as the Coupled Cluster methods (CC) of the quantum chemistry package NWChem, are of extreme interest to the computational chemistry community in fields such as catalytic reactions, solar energy, and bio-mass conversion. In spite of their importance, many of these computationally intensive algorithms have traditionally been thought of in a fairly linear fashion, or are parallelized in coarse chunks. In this paper, we present our effort of converting the NWChem’s CC code into a dataflow-based form that is capable of utilizing the task scheduling system PaRSEC (Parallel Runtime Scheduling and Execution Controller): a software package designed to enable high-performance computing at scale. We discuss the modularity of our approach and explain how the PaRSEC-enabled dataflow version of the subroutines seamlessly integrate into the NWChem codebase. Furthermore, we argue how the CC algorithms can be easily decomposed into finer-grained tasks (compared with the original version of NWChem); and how data distribution and load balancing are decoupled and can be tuned independently. We demonstrate performance acceleration by more than a factor of two in the execution of the entire CC component of NWChem, concluding that the utilization of dataflow-based execution for CC methods enables more efficient and scalable computation.

Double Precision Is Not Needed for Many-Body Calculations: New Conventional Wisdom

2018 ◽  
Author(s):  
Pavel Pokhilko ◽  
Evgeny Epifanovsky ◽  
Anna I. Krylov
Keyword(s):  
Large Scale ◽  
Computation Time ◽  
Coupled Cluster ◽  
Double Precision ◽  
Many Body ◽  
Single Precision ◽  
Parallel Performance ◽  
Point Representation ◽  
Electron Repulsion Integrals ◽  
Cluster Methods

Using single precision floating point representation reduces the size of data and computation time by a factor of two relative to double precision conventionally used in electronic structure programs. For large-scale calculations, such as those encountered in many-body theories, reduced memory footprint alleviates memory and input/output bottlenecks. Reduced size of data can lead to additional gains due to improved parallel performance on CPUs and various accelerators. However, using single precision can potentially reduce the accuracy of computed observables. Here we report an implementation of coupled-cluster and equation-of-motion coupled-cluster methods with single and double excitations in single precision. We consider both standard implementation and one using Cholesky decomposition or resolution-of-the-identity of electron-repulsion integrals. Numerical tests illustrate that when single precision is used in correlated calculations, the loss of accuracy is insignificant and pure single-precision implementation can be used for computing energies, analytic gradients, excited states, and molecular properties. In addition to pure single-precision calculations, our implementation allows one to follow a single-precision calculation by clean-up iterations, fully recovering double-precision results while retaining significant savings.

scholarly journals META-pipe cloud setup and execution

F1000Research ◽  
2017 ◽  
Vol 6 ◽  
pp. 2060
Author(s):  
Aleksandr Agafonov ◽  
Kimmo Mattila ◽  
Cuong Duong Tuan ◽  
Lars Tiede ◽  
Inge Alexander Raknes ◽  
...  
Keyword(s):  
Functional Annotation ◽  
High Performance ◽  
Sequence Data ◽  
Metagenomic Data ◽  
Taxonomic Profiling ◽  
Geographically Distributed ◽  
Computationally Intensive ◽  
High Performance Computing Cluster ◽  
And Storage ◽  
Performance Computing

META-pipe is a complete service for the analysis of marine metagenomic data. It provides assembly of high-throughput sequence data, functional annotation of predicted genes, and taxonomic profiling. The functional annotation is computationally demanding and is therefore currently run on a high-performance computing cluster in Norway. However, additional compute resources are necessary to open the service to all ELIXIR users. We describe our approach for setting up and executing the functional analysis of META-pipe on additional academic and commercial clouds. Our goal is to provide a powerful analysis service that is easy to use and to maintain. Our design therefore uses a distributed architecture where we combine central servers with multiple distributed backends that execute the computationally intensive jobs. We believe our experiences developing and operating META-pipe provides a useful model for others that plan to provide a portal based data analysis service in ELIXIR and other organizations with geographically distributed compute and storage resources.

scholarly journals High-Performance Computing of BEAST/BEAGLE in Bayesian Phylogenetics using SDumont Hybrid Resources

2020 ◽  
Author(s):  
Kary Ocaña ◽  
Micaella Coelho ◽  
Guilherme Freire ◽  
Carla Osthoff
Keyword(s):  
High Performance Computing ◽  
High Performance ◽  
Current Knowledge ◽  
Data Sets ◽  
Length Estimation ◽  
Computationally Intensive ◽  
Mcmc Chain ◽  
Performance Computing ◽  
Insight Into

Bayesian phylogenetic algorithms are computationally intensive. BEAST 1.10 inferences made use of the BEAGLE 3 high-performance library for efficient likelihood computations. The strategy allows phylogenetic inference and dating in current knowledge for SARS-CoV-2 transmission. Follow-up simulations on hybrid resources of Santos Dumont supercomputer using four phylogenomic data sets, we characterize the scaling performance behavior of BEAST 1.10. Our results provide insight into the species tree and MCMC chain length estimation, identifying preferable requirements to improve the use of high-performance computing resources. Ongoing steps involve analyzes of SARS-CoV-2 using BEAST 1.8 in multi-GPUs.

scholarly journals Implementation of Feldman-Cousins corrections and oscillation calculations in the HPC environment for the NOvA Experiment

2019 ◽  
Vol 214 ◽  
pp. 05012
Author(s):  
Alexandre Sousa ◽  
Norman Buchanan ◽  
Steven Calvez ◽  
Pengfei Ding ◽  
Derek Doyle ◽  
...  
Keyword(s):  
Data Analysis ◽  
High Performance Computing ◽  
Neutrino Oscillation ◽  
Research Program ◽  
High Performance ◽  
Sponsored Research ◽  
Statistical Correction ◽  
Fitting Procedures ◽  
Computationally Intensive ◽  
Performance Computing

Analysis of neutrino oscillation data involves a combination of complex fitting procedures and statistical correction techniques that are used to determine the full three-flavor PMNS parameters and constraint contours. These techniques rely on computationally intensive “multi-universe” stochastic modeling. The process of calculating these contours and corrections can dominate final stages of the data analysis and become a bottleneck for examining the ef-fect of systematic variations on the final results. As part of the DOE SciDAC-4 sponsored research program, we present a new implementation of a neutrino oscillation fitting and framework to carry out calculations of Feldman-Cousins corrections. This implementation is specifically designed and optimized to operate on modern High-Performance Computing (HPC) facilities. We present the performance of the system in calculating allowed regions for the NOvA experiment based on 8.9 x 1020 and 6.9 x 1020 protons-on-target (POT) neutrino and antineutrino datasets, and compare the performance of this new implementation run at the NERSC supercomputing facility with that from methods used previously by the NOvA collaboration running on grid computing facilities.

Double Precision Is Not Needed for Many-Body Calculations: New Conventional Wisdom

2018 ◽  
Author(s):  
Pavel Pokhilko ◽  
Evgeny Epifanovsky ◽  
Anna I. Krylov
Keyword(s):  
Large Scale ◽  
Computation Time ◽  
Coupled Cluster ◽  
Double Precision ◽  
Many Body ◽  
Single Precision ◽  
Parallel Performance ◽  
Point Representation ◽  
Electron Repulsion Integrals ◽  
Cluster Methods

Using single precision floating point representation reduces the size of data and computation time by a factor of two relative to double precision conventionally used in electronic structure programs. For large-scale calculations, such as those encountered in many-body theories, reduced memory footprint alleviates memory and input/output bottlenecks. Reduced size of data can lead to additional gains due to improved parallel performance on CPUs and various accelerators. However, using single precision can potentially reduce the accuracy of computed observables. Here we report an implementation of coupled-cluster and equation-of-motion coupled-cluster methods with single and double excitations in single precision. We consider both standard implementation and one using Cholesky decomposition or resolution-of-the-identity of electron-repulsion integrals. Numerical tests illustrate that when single precision is used in correlated calculations, the loss of accuracy is insignificant and pure single-precision implementation can be used for computing energies, analytic gradients, excited states, and molecular properties. In addition to pure single-precision calculations, our implementation allows one to follow a single-precision calculation by clean-up iterations, fully recovering double-precision results while retaining significant savings.

scholarly journals META-pipe cloud setup and execution

F1000Research ◽  
2019 ◽  
Vol 6 ◽  
pp. 2060
Author(s):  
Aleksandr Agafonov ◽  
Kimmo Mattila ◽  
Cuong Duong Tuan ◽  
Lars Tiede ◽  
Inge Alexander Raknes ◽  
...  
Keyword(s):  
Functional Annotation ◽  
High Performance ◽  
Sequence Data ◽  
Metagenomic Data ◽  
Taxonomic Profiling ◽  
Geographically Distributed ◽  
Computationally Intensive ◽  
High Performance Computing Cluster ◽  
And Storage ◽  
Performance Computing

META-pipe is a complete service for the analysis of marine metagenomic data. It provides assembly of high-throughput sequence data, functional annotation of predicted genes, and taxonomic profiling. The functional annotation is computationally demanding and is therefore currently run on a high-performance computing cluster in Norway. However, additional compute resources are necessary to open the service to all ELIXIR users. We describe our approach for setting up and executing the functional analysis of META-pipe on additional academic and commercial clouds. Our goal is to provide a powerful analysis service that is easy to use and to maintain. Our design therefore uses a distributed architecture where we combine central servers with multiple distributed backends that execute the computationally intensive jobs. We believe our experiences developing and operating META-pipe provides a useful model for others that plan to provide a portal based data analysis service in ELIXIR and other organizations with geographically distributed compute and storage resources.

Sign in / Sign up

Export Citation Format

Share Document