Parallel programming with message passing and directives

Shared memory is the most popular parallel programming model for multi-core processors, while message passing is generally used for large distributed machines. However, as the number of cores on a chip increases, the relative merits of shared memory versus message passing change, and we argue that message passing becomes a viable, high performing, and parallel programming model. To demonstrate this hypothesis, we compare a shared memory architecture with a new message passing architecture on a suite of applications tuned for each system independently. Perhaps surprisingly, the fundamental behaviors of the applications studied in this work, when optimized for both models, are very similar to each other, and both could execute efficiently on multicore architectures despite many implementations being different from each other. Furthermore, if hardware is tuned to support message passing by supporting bulk message transfer and the elimination of unnecessary coherence overheads, and if effective support is available for global operations, then some applications would perform much better on a message passing architecture. Leveraging our insights, we design a message passing architecture that supports both memory-to-memory and cache-to-cache messaging in hardware. With the new architecture, message passing is able to outperform its shared memory counterparts on many of the applications due to the unique advantages of the message passing hardware as compared to cache coherence. In the best case, message passing achieves up to a 34% increase in speed over its shared memory counterpart, and it achieves an average 10% increase in speed. In the worst case, message passing is slowed down in two applications—CG (conjugate gradient) and FT (Fourier transform)—because it could not perform well on the unique data sharing patterns as its counterpart of shared memory. Overall, our analysis demonstrates the importance of considering message passing as a high performing and hardware-supported programming model on future multicore architectures.

Download Full-text

Using MPI: Portable parallel programming with the message-passing interface

Computers & Mathematics with Applications ◽

10.1016/s0898-1221(00)90207-4 ◽

2000 ◽

Vol 40 (2-3) ◽

pp. 419 ◽

Cited By ~ 1

Keyword(s):

Parallel Programming ◽

Message Passing ◽

Message Passing Interface ◽

Portable Parallel Programming

Download Full-text

EFFICIENT SUPPORT FOR SKELETONS ON WORKSTATION CLUSTERS

Parallel Processing Letters ◽

10.1142/s0129626401000415 ◽

2001 ◽

Vol 11 (01) ◽

pp. 41-56 ◽

Cited By ~ 21

Author(s):

MARCO DANELUTTO

Keyword(s):

Parallel Programming ◽

Message Passing ◽

Low Cost ◽

Implementation Strategies ◽

Parallel Applications ◽

Programming Models ◽

Workstation Clusters ◽

Programming Environments ◽

Implementation Techniques ◽

Reasonable Cost

Beowulf class clusters are gaining more and more interest as low cost parallel architectures. They deliver reasonable performance at a very reasonable cost, compared to classical MPP machines. Parallel applications are usually developed on clusters using MPI/PVM message passing or HPF programming environments. Here we discuss new implementation strategies to support structured parallel programming environments for clusters based on skeletons. The adoption of structured parallel programming models greatly reduces the time spent in developing new parallel applications on clusters. The adoption of our implementation techniques based on macro data flow allows very efficient parallel applications to be developed on clusters. We discuss experiments that demonstrate the full feasibility of the approach.

Download Full-text