On a Pipeline-Based Architecture for Parallel Visualization of Large-Scale Scientific Data

Large scientific experimental facilities currently are generating a tremendous amount of data. In recent years, the significant growth of scientific data analysis has been observed across scientific research centers. Scientific experimental facilities are producing an unprecedented amount of data and facing new challenges to transfer the large data sets across multi continents. In particular, these days the data transfer is playing an important role in new scientific discoveries. The performance of distributed scientific environment is highly dependent on high-performance, adaptive, and robust network service infrastructures. To support large scale data transfer for extreme-scale distributed science, there is the need of high performance, scalable, end-to-end, and programmable networks that enable scientific applications to use the networks efficiently. We worked on the AmoebaNet solution to address the problems of a dynamic programmable network for bulk data transfer in extreme-scale distributed science environments. A major goal of the AmoebaNet project is to apply software-defined networking (SDN) technology to provide “Application-aware” network to facilitate bulk data transfer. We have prototyped AmoebaNet’s SDN-enabled network service that allows application to dynamically program the networks at run-time for bulk data transfers. In this paper, we evaluated AmoebaNet solution with real world test cases and shown that how it efficiently and dynamically can use the networks for bulk data transfer in large-scale scientific environments.

Download Full-text

HIGH-PERFORMANCE PARALLEL VISUALIZATION LIBRARY FOR VARIOUS SIMULATION ENVIRONMENTS

International Journal of Computational Methods ◽

10.1142/s0219876208001467 ◽

2008 ◽

Vol 05 (02) ◽

pp. 273-287

Author(s):

LI CHEN ◽

HIROSHI OKUDA

Keyword(s):

High Performance ◽

Large Scale ◽

Unstructured Grids ◽

Structured Grids ◽

The Earth ◽

Parallel Frameworks ◽

Parallel Visualization ◽

Simulation Environments

This paper describes a parallel visualization library for large-scale datasets developed in the HPC-MW project. Three parallel frameworks are provided in the library to satisfy different requirements of applications. Meanwhile, it is applicable for a variety of mesh types covering particles, structured grids and unstructured grids. Many techniques have been employed to improve the quality of the visualization. High speedup performance has been achieved by some hardware-oriented optimization strategies on different platforms, from PC clusters to the Earth Simulator. Good results have been obtained on some typical parallel platforms, thus demonstrating the feasibility and effectiveness of our library.

Download Full-text