scholarly journals Dimensioning Scientific Computing Systems to Improve Performance of Map-Reduce based Applications

2012 ◽  
Vol 9 ◽  
pp. 226-235 ◽  
Author(s):  
Gabriel G. Castañè ◽  
Alberto Núneñ ◽  
Rosa Filgueira ◽  
Jesús Carretero
Keyword(s):  
Scientific Computing ◽  
Map Reduce ◽  
Improve Performance ◽  
Computing Systems

A Map-Reduce Approach for the Dijkstra Algorithm in SDN Over Osmotic Computing Systems

2021 ◽  
Author(s):  
Maria Fazio ◽  
Alina Buzachis ◽  
Antonino Galletta ◽  
Antonio Celesti ◽  
Jiafu Wan ◽  
...  
Keyword(s):  
Map Reduce ◽  
Dijkstra Algorithm ◽  
Computing Systems

scholarly journals High-performance Python for crystallographic computing

2019 ◽  
Vol 52 (4) ◽  
pp. 882-897 ◽  
Author(s):  
A. Boulle ◽  
J. Kieffer
Keyword(s):  
Programming Language ◽  
High Performance ◽  
Scientific Computing ◽  
State Of The Art ◽  
Third Party ◽  
Improve Performance ◽  
Python Language ◽  
Python Programming Language ◽  
Numerical Computing ◽  
Python Programming

The Python programming language, combined with the numerical computing library NumPy and the scientific computing library SciPy, has become the de facto standard for scientific computing in a variety of fields. This popularity is mainly due to the ease with which a Python program can be written and executed (easy syntax, dynamical typing, no compilation etc.), coupled with the existence of a large number of specialized third-party libraries that aim to lift all the limitations of the raw Python language. NumPy introduces vector programming, improving execution speeds, whereas SciPy brings a wealth of highly optimized and reliable scientific functions. There are cases, however, where vector programming alone is not sufficient to reach optimal performance. This issue is addressed with dedicated compilers that aim to translate Python code into native and statically typed code with support for the multi-core architectures of modern processors. In the present article it is shown how these approaches can be efficiently used to tackle different problems, with increasing complexity, that are relevant to crystallography: the 2D Laue function, scattering from a strained 2D crystal, scattering from 3D nanocrystals and, finally, diffraction from films and multilayers. For each case, detailed implementations and explanations of the functioning of the algorithms are provided. Different Python compilers (namely NumExpr, Numba, Pythran and Cython) are used to improve performance and are benchmarked against state-of-the-art NumPy implementations. All examples are also provided as commented and didactic Python (Jupyter) notebooks that can be used as starting points for crystallographers curious to enter the Python ecosystem or wishing to accelerate their existing codes.

Estimating the overhead and coupling of scientific computing clusters

SIMULATION ◽  
2021 ◽  
pp. 003754972110641
Author(s):  
Aurelio Vivas ◽  
Harold Castro
Keyword(s):  
High Performance ◽  
Scientific Computing ◽  
Virtual Machines ◽  
Building Blocks ◽  
Scientific Applications ◽  
Computing Systems ◽  
The Third ◽  
Cluster Of Workstations ◽  
Computer Aided ◽  
Performance Computing

Since simulation became the third pillar of scientific research, several forms of computers have become available to drive computer aided simulations, and nowadays, clusters are the most popular type of computers supporting these tasks. For instance, cluster settings, such as the so-called supercomputers, cluster of workstations (COW), cluster of desktops (COD), and cluster of virtual machines (COV) have been considered in literature to embrace a variety of scientific applications. However, those scientific applications categorized as high-performance computing (HPC) are conceptually restricted to be addressed only by supercomputers. In this aspect, we introduce the notions of cluster overhead and cluster coupling to assess the capacity of non-HPC systems to handle HPC applications. We also compare the cluster overhead with an existing measure of overhead in computing systems, the total parallel overhead, to explain the correctness of our methodology. The evaluation of capacity considers the seven dwarfs of scientific computing, which are well-known, scientific computing building blocks considered in the development of HPC applications. The evaluation of these building blocks provides insights regarding the strengths and weaknesses of non-HPC systems to deal with future HPC applications developed with one or a combination of these algorithmic building blocks.

scholarly journals Build and Evaluate a Free Virtual Cluster on Amazon Elastic Compute Cloud for Scientific Computing

2017 ◽  
Vol 13 (08) ◽  
pp. 121 ◽  
Author(s):  
Jie Xiong ◽  
Shen-Han Shi ◽  
Song Zhang
Keyword(s):  
Cloud Computing ◽  
Performance Evaluation ◽  
Scientific Computing ◽  
Prototype System ◽  
Virtual Cluster ◽  
Huge Amount ◽  
Computing Systems ◽  
Comprehensive Performance ◽  
Amazon Ec2

Scientific computing requires a huge amount of computing resources, but not all the scientific researchers have an access to sufficient high-end computing systems. Currently, Amazon provides a free tier account for cloud computing which could be used to build a virtual cluster. In order to investigate whether it is suitable for scientific computing, we first describe how to build a free virtual cluster using StarCluster on Amazon Elastic Compute Cloud (EC2). Then, we perform a comprehensive performance evaluation of the virtual cluster built before. The results show that a free virtual cluster is easily built on Amazon EC2 and it is suitable for the basic scientific computing. It is especially valuable for scientific researchers, who do not have any HPC or cluster, to develop and test their prototype system of scientific computing without paying anything, and move it to a higher performance virtual cluster when necessary by choosing more powerful instance on Amazon EC2.

scholarly journals Use cases of lossy compression for floating-point data in scientific data sets

2019 ◽  
Vol 33 (6) ◽  
pp. 1201-1220 ◽  
Author(s):  
Franck Cappello ◽  
Sheng Di ◽  
Sihuan Li ◽  
Xin Liang ◽  
Ali Murat Gok ◽  
...  
Keyword(s):  
High Performance ◽  
Scientific Computing ◽  
Lossy Compression ◽  
Scientific Data ◽  
Use Cases ◽  
Floating Point ◽  
Data Sets ◽  
Use Case ◽  
Computing Systems ◽  
Point Data

Architectural and technological trends of systems used for scientific computing call for a significant reduction of scientific data sets that are composed mainly of floating-point data. This article surveys and presents experimental results of currently identified use cases of generic lossy compression to address the different limitations of scientific computing systems. The article shows from a collection of experiments run on parallel systems of a leadership facility that lossy data compression not only can reduce the footprint of scientific data sets on storage but also can reduce I/O and checkpoint/restart times, accelerate computation, and even allow significantly larger problems to be run than without lossy compression. These results suggest that lossy compression will become an important technology in many aspects of high performance scientific computing. Because the constraints for each use case are different and often conflicting, this collection of results also indicates the need for more specialization of the compression pipelines.

scholarly journals Application of SLURM, BOINC, and GlusterFS as Software System for Sustainable Modeling and Data Analytics

2018 ◽  
Vol 173 ◽  
pp. 05010
Author(s):  
Vladislav V. Kashansky ◽  
Igor L. Kaftannikov
Keyword(s):  
Numerical Modeling ◽  
Distributed Computing ◽  
Data Analytics ◽  
Scientific Computing ◽  
Science And Technology ◽  
Systems Integration ◽  
Software System ◽  
Distributed Computing Systems ◽  
Computing Systems

Modern numerical modeling experiments and data analytics problems in various fields of science and technology reveal a wide variety of serious requirements for distributed computing systems. Many scientific computing projects sometimes exceed the available resource pool limits, requiring extra scalability and sustainability. In this paper we share the experience and findings of our own on combining the power of SLURM, BOINC and GlusterFS as software system for scientific computing. Especially, we suggest a complete architecture and highlight important aspects of systems integration.

Artefact in 20Å-Resolution Electron Images of Negatively Stained Twodimensional Membrane Protein Crystals

1982 ◽  
Vol 40 ◽  
pp. 92-93
Author(s):  
Douglas L. Dorset ◽  
Barbara Moss
Keyword(s):  
Electron Scattering ◽  
Cross Correlation ◽  
Transmission Function ◽  
Group Symmetry ◽  
Asymmetric Structure ◽  
Object Model ◽  
Phase Object ◽  
Computing Systems ◽  
Resolution Electron ◽  
Plane Group

A number of computing systems devoted to the averaging of electron images of two-dimensional macromolecular crystalline arrays have facilitated the visualization of negatively-stained biological structures. Either by simulation of optical filtering techniques or, in more refined treatments, by cross-correlation averaging, an idealized representation of the repeating asymmetric structure unit is constructed, eliminating image distortions due to radiation damage, stain irregularities and, in the latter approach, imperfections and distortions in the unit cell repeat. In these analyses it is generally assumed that the electron scattering from the thin negativelystained object is well-approximated by a phase object model. Even when absorption effects are considered (i.e. “amplitude contrast“), the expansion of the transmission function, q(x,y)=exp (iσɸ (x,y)), does not exceed the first (kinematical) term. Furthermore, in reconstruction of electron images, kinematical phases are applied to diffraction amplitudes and obey the constraints of the plane group symmetry.

Sign in / Sign up

Export Citation Format

Share Document