scholarly journals Comparison of Message-Passing and Shared Memory Implementations of the GMRES Method on MIMD Computers

2001 ◽  
Vol 9 (4) ◽  
pp. 195-209 ◽  
Author(s):  
Joanna Płażek ◽  
Krzysztof Banaś ◽  
Jacek Kitowski
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Programming Model ◽  
Linear Equations ◽  
Flow Simulation ◽  
Gmres Method ◽  
Performance Measurements ◽  
Nonlinear Simulation ◽  
Implicit Model ◽  
Preconditioned Gmres

In this paper we compare different parallel implementations of the same algorithm for solving nonlinear simulation problems on unstructured meshes. In the first implementation, making use of the message-passing programming model and the PVM system, the domain decomposition of unstructured mesh is implemented, while the second implementation takes advantage of the inherent parallelism of the algorithm by adopting the shared-memory programming model. Both implementations are applied to the preconditioned GMRES method that solves iteratively the system of linear equations. A combined approach, the hybrid programming model suitable for multicomputers with SMP nodes, is introduced. For performance measurements we use compressible fluid flow simulation in which sequences of finite element solutions form time approximations to the Euler equations. The tests are performed on HP SPP1600, HP S2000 and SGI Origin2000 multiprocessors and report wall-clock execution time and speedup for different number of processing nodes and for different meshes. Experimentally, the explicit programming model proves to be more efficient than the implicit model by 20—70%, depends on the mesh and the machine.

scholarly journals Parallel Array Classes and Lightweight Sharing Mechanisms

1993 ◽  
Vol 2 (4) ◽  
pp. 203-216
Author(s):  
Steve W. Otto
Keyword(s):  
Finite Element Method ◽  
Shared Memory ◽  
Message Passing ◽  
Distributed Memory ◽  
Programming Model ◽  
Memory Usage ◽  
Particle In Cell ◽  
Parallel Array ◽  
Memory Architectures ◽  
Shared Memory Architectures

We discuss a set of parallel array classes, MetaMP, for distributed-memory architectures. The classes are implemented in C++ and interface to the PVM or Intel NX message-passing systems. An array class implements a partitioned array as a set of objects distributed across the nodes – a "collective" object. Object methods hide the low-level message-passing and implement meaningful array operations. These include transparent guard strips (or sharing regions) that support finite-difference stencils, reductions and multibroadcasts for support of pivoting and row operations, and interpolation/contraction operations for support of multigrid algorithms. The concept of guard strips is generalized to an object implementation of lightweight sharing mechanisms for finite element method (FEM) and particle-in-cell (PIC) algorithms. The sharing is accomplished through the mechanism of weak memory coherence and can be efficiently implemented. The price of the efficient implementation is memory usage and the need to explicitly specify the coherence operations. An intriguing feature of this programming model is that it maps well to both distributed-memory and shared-memory architectures.

scholarly journals The Semi-Automatic Parallelisation of Scientific Application Codes Using a Computer Aided Parallelisation Toolkit

2001 ◽  
Vol 9 (2-3) ◽  
pp. 163-173 ◽  
Author(s):  
C.S. Ierotheou ◽  
S.P. Johnson ◽  
P.F. Leggett ◽  
M. Cross ◽  
E.W. Evans ◽  
...  
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Programming Model ◽  
Parallel Computers ◽  
Parallel Programs ◽  
Scientific Application ◽  
Real World Application ◽  
Interprocedural Analysis ◽  
Computer Aided ◽  
Application Codes

The shared-memory programming model can be an effective way to achieve parallelism on shared memory parallel computers. Historically however, the lack of a programming standard using directives and the limited scalability have affected its take-up. Recent advances in hardware and software technologies have resulted in improvements to both the performance of parallel programs with compiler directives and the issue of portability with the introduction of OpenMP. In this study, the Computer Aided Parallelisation Toolkit has been extended to automatically generate OpenMP-based parallel programs with nominal user assistance. We categorize the different loop types and show how efficient directives can be placed using the toolkit's in-depth interprocedural analysis. Examples are taken from the NAS parallel benchmarks and a number of real-world application codes. This demonstrates the great potential of using the toolkit to quickly parallelise serial programs as well as the good performance achievable on up to 300 processors for hybrid message passing-directive parallelisations.

scholarly journals Implementation and Performance of DSMPI

1997 ◽  
Vol 6 (2) ◽  
pp. 201-214 ◽  
Author(s):  
Luis M. Silva ◽  
JoÃo Gabriel Silva ◽  
Simon Chapple
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Distributed Memory ◽  
Programming Model ◽  
Distributed Shared Memory ◽  
Memory Systems ◽  
Distributed Memory Machines ◽  
Coherence Protocols ◽  
And Performance ◽  
Performance Results

Distributed shared memory has been recognized as an alternative programming model to exploit the parallelism in distributed memory systems because it provides a higher level of abstraction than simple message passing. DSM combines the simple programming model of shared memory with the scalability of distributed memory machines. This article presents DSMPI, a parallel library that runs atop of MPI and provides a DSM abstraction. It provides an easy-to-use programming interface, is fully, portable, and supports heterogeneity. For the sake of flexibility, it supports different coherence protocols and models of consistency. We present some performance results taken in a network of workstations and in a Cray T3D which show that DSMPI can be competitive with MPI for some applications.

scholarly journals Effective On-Chip Communication for Message Passing Programs on Multi-Core Processors

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2681
Author(s):  
Joonmoo Huh ◽  
Deokwoo Lee
Keyword(s):  
Parallel Programming ◽  
Shared Memory ◽  
Message Passing ◽  
Programming Model ◽  
Multicore Architectures ◽  
Worst Case ◽  
High Performing ◽  
Parallel Programming Model ◽  
On Chip ◽  
Sharing Patterns

Shared memory is the most popular parallel programming model for multi-core processors, while message passing is generally used for large distributed machines. However, as the number of cores on a chip increases, the relative merits of shared memory versus message passing change, and we argue that message passing becomes a viable, high performing, and parallel programming model. To demonstrate this hypothesis, we compare a shared memory architecture with a new message passing architecture on a suite of applications tuned for each system independently. Perhaps surprisingly, the fundamental behaviors of the applications studied in this work, when optimized for both models, are very similar to each other, and both could execute efficiently on multicore architectures despite many implementations being different from each other. Furthermore, if hardware is tuned to support message passing by supporting bulk message transfer and the elimination of unnecessary coherence overheads, and if effective support is available for global operations, then some applications would perform much better on a message passing architecture. Leveraging our insights, we design a message passing architecture that supports both memory-to-memory and cache-to-cache messaging in hardware. With the new architecture, message passing is able to outperform its shared memory counterparts on many of the applications due to the unique advantages of the message passing hardware as compared to cache coherence. In the best case, message passing achieves up to a 34% increase in speed over its shared memory counterpart, and it achieves an average 10% increase in speed. In the worst case, message passing is slowed down in two applications—CG (conjugate gradient) and FT (Fourier transform)—because it could not perform well on the unique data sharing patterns as its counterpart of shared memory. Overall, our analysis demonstrates the importance of considering message passing as a high performing and hardware-supported programming model on future multicore architectures.

MIMD, Multiple Instruction, Multiple Data

2004 ◽  
Author(s):  
Wesley Petersen ◽  
Peter Arbenz
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Distributed Memory ◽  
Programming Model ◽  
Data Access ◽  
File Server ◽  
Distributed Memory Machines ◽  
Shared Data ◽  
Multiple Data ◽  
Common Memory

The Multiple instruction, multiple data (MIMD) programming model usually refers to computing on distributed memory machines with multiple independent processors. Although processors may run independent instruction streams, we are interested in streams that are always portions of a single program. Between processors which share a coherent memory view (within a node), data access is immediate, whereas between nodes data access is effected by message passing. In this book, we use MPI for such message passing. MPI has emerged as a more/less standard message passing system used on both shared memory and distributed memory machines. It is often the case that although the system consists of multiple independent instruction streams, the programming model is not too different from SIMD. Namely, the totality of a program is logically split into many independent tasks each processed by a group (see Appendix D) of processes—but the overall program is effectively single threaded at the beginning, and likewise at the end. The MIMD model, however, is extremely flexible in that no one process is always master and the other processes slaves. A communicator group of processes performs certain tasks, usually with an arbitrary master/slave relationship. One process may be assigned to be master (or root) and coordinates the tasks of others in the group. We emphasize that the assignments of which is root is arbitrary—any processor may be chosen. Frequently, however, this choice is one of convenience—a file server node, for example. Processors and memory are connected by a network, for example, Figure 5.1. In this form, each processor has its own local memory. This is not always the case: The Cray X1, and NEC SX-6 through SX-8 series machines, have common memory within nodes. Within a node, memory coherency is maintained within local caches. Between nodes, it remains the programmer’s responsibility to assure a proper read–update relationship in the shared data. Data updated by one set of processes should not be clobbered by another set until the data are properly used.

GPU-accelerated preconditioned GMRES method for two-dimensional Maxwell's equations

2017 ◽  
Vol 94 (10) ◽  
pp. 2122-2144 ◽  
Author(s):  
Jiaquan Gao ◽  
Kesong Wu ◽  
Yushun Wang ◽  
Panpan Qi ◽  
Guixia He
Keyword(s):  
Maxwell’S Equations ◽  
Maxwell's Equations ◽  
Gmres Method ◽  
Two Dimensional ◽  
Preconditioned Gmres

Parallel Nonnegative Matrix Factorization via Newton Iteration

2016 ◽  
Vol 26 (03) ◽  
pp. 1650014 ◽  
Author(s):  
Markus Flatz ◽  
Marián Vajteršic
Keyword(s):  
Shared Memory ◽  
Matrix Factorization ◽  
Message Passing ◽  
Nonnegative Matrix Factorization ◽  
Nonnegative Matrix ◽  
Newton Iteration ◽  
Parallel Execution ◽  
Kkt Conditions ◽  
Nonnegative Matrices ◽  
First Order

The goal of Nonnegative Matrix Factorization (NMF) is to represent a large nonnegative matrix in an approximate way as a product of two significantly smaller nonnegative matrices. This paper shows in detail how an NMF algorithm based on Newton iteration can be derived using the general Karush-Kuhn-Tucker (KKT) conditions for first-order optimality. This algorithm is suited for parallel execution on systems with shared memory and also with message passing. Both versions were implemented and tested, delivering satisfactory speedup results.

An Efficient Parallel Algorithm for Extreme Eigenvalues of Sparse Nonsymmetric Matrices

1992 ◽  
Vol 6 (1) ◽  
pp. 98-111 ◽  
Author(s):  
S. K. Kim ◽  
A. T. Chrortopoulos
Keyword(s):  
Shared Memory ◽  
Message Passing ◽  
Sparse Matrices ◽  
Data Locality ◽  
Main Memory ◽  
Global Memory ◽  
Global Communication ◽  
Step Method ◽  
Arnoldi Algorithm ◽  
Large Sparse Matrices

Main memory accesses for shared-memory systems or global communications (synchronizations) in message passing systems decrease the computation speed. In this paper, the standard Arnoldi algorithm for approximating a small number of eigenvalues, with largest (or smallest) real parts for nonsymmetric large sparse matrices, is restructured so that only one synchronization point is required; that is, one global communication in a message passing distributed-memory machine or one global memory sweep in a shared-memory machine per each iteration is required. We also introduce an s-step Arnoldi method for finding a few eigenvalues of nonsymmetric large sparse matrices. This method generates reduction matrices that are similar to those generated by the standard method. One iteration of the s-step Arnoldi algorithm corresponds to s iterations of the standard Arnoldi algorithm. The s-step method has improved data locality, minimized global communication, and superior parallel properties. These algorithms are implemented on a 64-node NCUBE/7 Hypercube and a CRAY-2, and performance results are presented.

Sign in / Sign up

Export Citation Format

Share Document