scholarly journals The Semi-Automatic Parallelisation of Scientific Application Codes Using a Computer Aided Parallelisation Toolkit

2001 ◽  
Vol 9 (2-3) ◽  
pp. 163-173 ◽  
Author(s):  
C.S. Ierotheou ◽  
S.P. Johnson ◽  
P.F. Leggett ◽  
M. Cross ◽  
E.W. Evans ◽  
...  
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Programming Model ◽  
Parallel Computers ◽  
Parallel Programs ◽  
Scientific Application ◽  
Real World Application ◽  
Interprocedural Analysis ◽  
Computer Aided ◽  
Application Codes

The shared-memory programming model can be an effective way to achieve parallelism on shared memory parallel computers. Historically however, the lack of a programming standard using directives and the limited scalability have affected its take-up. Recent advances in hardware and software technologies have resulted in improvements to both the performance of parallel programs with compiler directives and the issue of portability with the introduction of OpenMP. In this study, the Computer Aided Parallelisation Toolkit has been extended to automatically generate OpenMP-based parallel programs with nominal user assistance. We categorize the different loop types and show how efficient directives can be placed using the toolkit's in-depth interprocedural analysis. Examples are taken from the NAS parallel benchmarks and a number of real-world application codes. This demonstrates the great potential of using the toolkit to quickly parallelise serial programs as well as the good performance achievable on up to 300 processors for hybrid message passing-directive parallelisations.

scholarly journals Teaching tools for parallel processing

2005 ◽  
Vol 18 (2) ◽  
pp. 219-224
Author(s):  
Emina Milovanovic ◽  
Natalija Stojanovic
Keyword(s):  
Parallel Computing ◽  
Parallel Processing ◽  
Shared Memory ◽  
Message Passing ◽  
Distributed Memory ◽  
Cost Effective ◽  
Parallel Computers ◽  
Free Software ◽  
Teaching Tools ◽  
Network Of Workstations

Because many universities lack the funds to purchase expensive parallel computers, cost effective alternatives are needed to teach students about parallel processing. Free software is available to support the three major paradigms of parallel computing. Parallaxis is a sophisticated SIMD simulator which runs on a variety of platforms.jBACI shared memory simulator supports the MIMD model of computing with a common shared memory. PVM and MPI allow students to treat a network of workstations as a message passing MIMD multicomputer with distributed memory. Each of this software tools can be used in a variety of courses to give students experience with parallel algorithms.

scholarly journals Parallel Array Classes and Lightweight Sharing Mechanisms

1993 ◽  
Vol 2 (4) ◽  
pp. 203-216
Author(s):  
Steve W. Otto
Keyword(s):  
Finite Element Method ◽  
Shared Memory ◽  
Message Passing ◽  
Distributed Memory ◽  
Programming Model ◽  
Memory Usage ◽  
Particle In Cell ◽  
Parallel Array ◽  
Memory Architectures ◽  
Shared Memory Architectures

We discuss a set of parallel array classes, MetaMP, for distributed-memory architectures. The classes are implemented in C++ and interface to the PVM or Intel NX message-passing systems. An array class implements a partitioned array as a set of objects distributed across the nodes – a "collective" object. Object methods hide the low-level message-passing and implement meaningful array operations. These include transparent guard strips (or sharing regions) that support finite-difference stencils, reductions and multibroadcasts for support of pivoting and row operations, and interpolation/contraction operations for support of multigrid algorithms. The concept of guard strips is generalized to an object implementation of lightweight sharing mechanisms for finite element method (FEM) and particle-in-cell (PIC) algorithms. The sharing is accomplished through the mechanism of weak memory coherence and can be efficiently implemented. The price of the efficient implementation is memory usage and the need to explicitly specify the coherence operations. An intriguing feature of this programming model is that it maps well to both distributed-memory and shared-memory architectures.

scholarly journals Implementation and Performance of DSMPI

1997 ◽  
Vol 6 (2) ◽  
pp. 201-214 ◽  
Author(s):  
Luis M. Silva ◽  
JoÃo Gabriel Silva ◽  
Simon Chapple
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Distributed Memory ◽  
Programming Model ◽  
Distributed Shared Memory ◽  
Memory Systems ◽  
Distributed Memory Machines ◽  
Coherence Protocols ◽  
And Performance ◽  
Performance Results

Distributed shared memory has been recognized as an alternative programming model to exploit the parallelism in distributed memory systems because it provides a higher level of abstraction than simple message passing. DSM combines the simple programming model of shared memory with the scalability of distributed memory machines. This article presents DSMPI, a parallel library that runs atop of MPI and provides a DSM abstraction. It provides an easy-to-use programming interface, is fully, portable, and supports heterogeneity. For the sake of flexibility, it supports different coherence protocols and models of consistency. We present some performance results taken in a network of workstations and in a Cray T3D which show that DSMPI can be competitive with MPI for some applications.

scholarly journals Effective On-Chip Communication for Message Passing Programs on Multi-Core Processors

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2681
Author(s):  
Joonmoo Huh ◽  
Deokwoo Lee
Keyword(s):  
Parallel Programming ◽  
Shared Memory ◽  
Message Passing ◽  
Programming Model ◽  
Multicore Architectures ◽  
Worst Case ◽  
High Performing ◽  
Parallel Programming Model ◽  
On Chip ◽  
Sharing Patterns

Shared memory is the most popular parallel programming model for multi-core processors, while message passing is generally used for large distributed machines. However, as the number of cores on a chip increases, the relative merits of shared memory versus message passing change, and we argue that message passing becomes a viable, high performing, and parallel programming model. To demonstrate this hypothesis, we compare a shared memory architecture with a new message passing architecture on a suite of applications tuned for each system independently. Perhaps surprisingly, the fundamental behaviors of the applications studied in this work, when optimized for both models, are very similar to each other, and both could execute efficiently on multicore architectures despite many implementations being different from each other. Furthermore, if hardware is tuned to support message passing by supporting bulk message transfer and the elimination of unnecessary coherence overheads, and if effective support is available for global operations, then some applications would perform much better on a message passing architecture. Leveraging our insights, we design a message passing architecture that supports both memory-to-memory and cache-to-cache messaging in hardware. With the new architecture, message passing is able to outperform its shared memory counterparts on many of the applications due to the unique advantages of the message passing hardware as compared to cache coherence. In the best case, message passing achieves up to a 34% increase in speed over its shared memory counterpart, and it achieves an average 10% increase in speed. In the worst case, message passing is slowed down in two applications—CG (conjugate gradient) and FT (Fourier transform)—because it could not perform well on the unique data sharing patterns as its counterpart of shared memory. Overall, our analysis demonstrates the importance of considering message passing as a high performing and hardware-supported programming model on future multicore architectures.

MIMD, Multiple Instruction, Multiple Data

2004 ◽  
Author(s):  
Wesley Petersen ◽  
Peter Arbenz
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Distributed Memory ◽  
Programming Model ◽  
Data Access ◽  
File Server ◽  
Distributed Memory Machines ◽  
Shared Data ◽  
Multiple Data ◽  
Common Memory

The Multiple instruction, multiple data (MIMD) programming model usually refers to computing on distributed memory machines with multiple independent processors. Although processors may run independent instruction streams, we are interested in streams that are always portions of a single program. Between processors which share a coherent memory view (within a node), data access is immediate, whereas between nodes data access is effected by message passing. In this book, we use MPI for such message passing. MPI has emerged as a more/less standard message passing system used on both shared memory and distributed memory machines. It is often the case that although the system consists of multiple independent instruction streams, the programming model is not too different from SIMD. Namely, the totality of a program is logically split into many independent tasks each processed by a group (see Appendix D) of processes—but the overall program is effectively single threaded at the beginning, and likewise at the end. The MIMD model, however, is extremely flexible in that no one process is always master and the other processes slaves. A communicator group of processes performs certain tasks, usually with an arbitrary master/slave relationship. One process may be assigned to be master (or root) and coordinates the tasks of others in the group. We emphasize that the assignments of which is root is arbitrary—any processor may be chosen. Frequently, however, this choice is one of convenience—a file server node, for example. Processors and memory are connected by a network, for example, Figure 5.1. In this form, each processor has its own local memory. This is not always the case: The Cray X1, and NEC SX-6 through SX-8 series machines, have common memory within nodes. Within a node, memory coherency is maintained within local caches. Between nodes, it remains the programmer’s responsibility to assure a proper read–update relationship in the shared data. Data updated by one set of processes should not be clobbered by another set until the data are properly used.

scholarly journals Comparison of Message-Passing and Shared Memory Implementations of the GMRES Method on MIMD Computers

2001 ◽  
Vol 9 (4) ◽  
pp. 195-209 ◽  
Author(s):  
Joanna Płażek ◽  
Krzysztof Banaś ◽  
Jacek Kitowski
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Programming Model ◽  
Linear Equations ◽  
Flow Simulation ◽  
Gmres Method ◽  
Performance Measurements ◽  
Nonlinear Simulation ◽  
Implicit Model ◽  
Preconditioned Gmres

In this paper we compare different parallel implementations of the same algorithm for solving nonlinear simulation problems on unstructured meshes. In the first implementation, making use of the message-passing programming model and the PVM system, the domain decomposition of unstructured mesh is implemented, while the second implementation takes advantage of the inherent parallelism of the algorithm by adopting the shared-memory programming model. Both implementations are applied to the preconditioned GMRES method that solves iteratively the system of linear equations. A combined approach, the hybrid programming model suitable for multicomputers with SMP nodes, is introduced. For performance measurements we use compressible fluid flow simulation in which sequences of finite element solutions form time approximations to the Euler equations. The tests are performed on HP SPP1600, HP S2000 and SGI Origin2000 multiprocessors and report wall-clock execution time and speedup for different number of processing nodes and for different meshes. Experimentally, the explicit programming model proves to be more efficient than the implicit model by 20—70%, depends on the mesh and the machine.

scholarly journals The Efficiency of Linda for General Purpose Scientific Programming

1994 ◽  
Vol 3 (1) ◽  
pp. 61-71 ◽  
Author(s):  
Timothy G. Mattson
Keyword(s):  
Shared Memory ◽  
Programming Language ◽  
Programming Model ◽  
Parallel Computers ◽  
General Purpose ◽  
Registered Trademark ◽  
Multiple Data ◽  
Virtual Shared Memory ◽  
Hybrid Language ◽  
Complete Set

Linda (Linda is a registered trademark of Scientific Computing Associates, Inc.) is a programming language for coordinating the execution and interaction of processes. When combined with a language for computation (such as C or Fortran), the resulting hybrid language can be used to write portable programs for parallel and distributed multiple instruction multiple data (MIMD) computers. The Linda programming model is based on operations that read, write, and erase a virtual shared memory. It is easy to use, and lets the programmer code in a very expressive, uncoupled programming style. These benefits, however, are of little value unless Linda programs execute efficiently. The goal of this article is to demonstrate that Linda programs are efficient making Linda an effective general purpose tool for programming MIMD parallel computers. Two arguments for Linda's efficiency are given; the first is based on Linda's implementation and the second on a range of case studies spanning a complete set of parallel algorithm classes.

scholarly journals Abstract Level Parallelization of Finite Difference Methods

1997 ◽  
Vol 6 (4) ◽  
pp. 331-344 ◽  
Author(s):  
Edwin Vollebregt
Keyword(s):  
Finite Difference ◽  
Message Passing ◽  
Finite Difference Methods ◽  
Parallel Computers ◽  
Parallel Programs ◽  
Programming Paradigm ◽  
Index Sets ◽  
Process Channel ◽  
Abstract Level

A formalism is proposed for describing finite difference calculations in an abstract way. The formalism consists of index sets and stencils, for characterizing the structure of sets of data items and interactions between data items (“neighbouring relations”). The formalism provides a means for lifting programming to a more abstract level. This simplifies the tasks of performance analysis and verification of correctness, and opens the way for automaticcode generation. The notation is particularly useful in parallelization, for the systematic construction of parallel programs in a process/channel programming paradigm (e.g., message passing). This is important because message passing, unfortunately, still is the only approach that leads to acceptable performance for many more unstructured or irregular problems on parallel computers that have non-uniform memory access times. It will be shown that the use of index sets and stencils greatly simplifies the determination of which data must be exchanged between different computing processes.

Sign in / Sign up

Export Citation Format

Share Document