Parallel Programming for Modern High Performance Computing Systems

2018 ◽  
Author(s):  
Paweł Czarnul
Keyword(s):  
High Performance Computing ◽  
Parallel Programming ◽  
High Performance ◽  
Computing Systems ◽  
Performance Computing

scholarly journals Survey of Methodologies, Approaches, and Challenges in Parallel Programming Using High-Performance Computing Systems

2020 ◽  
Vol 2020 ◽  
pp. 1-19 ◽  
Author(s):  
Paweł Czarnul ◽  
Jerzy Proficz ◽  
Krzysztof Drypczewski
Keyword(s):  
High Performance Computing ◽  
Parallel Programming ◽  
High Performance ◽  
Programming Model ◽  
Parallel Execution ◽  
Target System ◽  
Computing Systems ◽  
Programming Abstraction ◽  
Near Future ◽  
Performance Computing

This paper provides a review of contemporary methodologies and APIs for parallel programming, with representative technologies selected in terms of target system type (shared memory, distributed, and hybrid), communication patterns (one-sided and two-sided), and programming abstraction level. We analyze representatives in terms of many aspects including programming model, languages, supported platforms, license, optimization goals, ease of programming, debugging, deployment, portability, level of parallelism, constructs enabling parallelism and synchronization, features introduced in recent versions indicating trends, support for hybridity in parallel execution, and disadvantages. Such detailed analysis has led us to the identification of trends in high-performance computing and of the challenges to be addressed in the near future. It can help to shape future versions of programming standards, select technologies best matching programmers’ needs, and avoid potential difficulties while using high-performance computing systems.

scholarly journals Application-based fault tolerance techniques for sparse matrix solvers

2017 ◽  
Vol 32 (5) ◽  
pp. 627-640
Author(s):  
Simon McIntosh–Smith ◽  
Rob Hunt ◽  
James Price ◽  
Alex Warwick Vesztrocy
Keyword(s):  
Fault Tolerance ◽  
High Performance Computing ◽  
High Performance ◽  
Sparse Matrix ◽  
Sparse Matrices ◽  
Error Correcting Codes ◽  
Computing Systems ◽  
Hardware Costs ◽  
Extreme Scale ◽  
Performance Computing

High-performance computing systems continue to increase in size in the quest for ever higher performance. The resulting increased electronic component count, coupled with the decrease in feature sizes of the silicon manufacturing processes used to build these components, may result in future exascale systems being more susceptible to soft errors caused by cosmic radiation than in current high-performance computing systems. Through the use of techniques such as hardware-based error-correcting codes and checkpoint-restart, many of these faults can be mitigated at the cost of increased hardware overhead, run-time, and energy consumption that can be as much as 10–20%. Some predictions expect these overheads to continue to grow over time. For extreme scale systems, these overheads will represent megawatts of power consumption and millions of dollars of additional hardware costs, which could potentially be avoided with more sophisticated fault-tolerance techniques. In this paper we present new software-based fault tolerance techniques that can be applied to one of the most important classes of software in high-performance computing: iterative sparse matrix solvers. Our new techniques enables us to exploit knowledge of the structure of sparse matrices in such a way as to improve the performance, energy efficiency, and fault tolerance of the overall solution.

Sign in / Sign up

Export Citation Format

Share Document