High performance wide area data transfers over high performance networks

Data integration and fast eﬀective data processing are the primary challenges in today’s high-performance computing systems used for Big Data processing and analysis in practical scenarios. Blockchain (BC) is a hot, modern technology that ensures high security of data processes stored in highly distributed networks and ICT infrastructures. BC enables secure data transfers in distributed systems without the need for all operations and processes in the network to be initiated and monitored by any central authority (system manager). This paper presents the background of a generic architectural model of a BC system and explains the concept behind the consensus models used in BC transactions. Security is the main aspect of all deﬁned operations and BC nodes. The paper presents also speciﬁc BC use cases to illustrate the performance of the system in practical scenarios..

Download Full-text

Dynamic Protocol Tuning Algorithms for High Performance Data Transfers

Euro-Par 2013 Parallel Processing - Lecture Notes in Computer Science ◽

10.1007/978-3-642-40047-6_72 ◽

2013 ◽

pp. 725-736 ◽

Cited By ~ 12

Author(s):

Engin Arslan ◽

Brandon Ross ◽

Tevfik Kosar

Keyword(s):

High Performance ◽

Performance Data ◽

Data Transfers ◽

Protocol Tuning

Download Full-text

Data Storage, Retrieval and Management

Advances in Systems Analysis, Software Engineering, and High Performance Computing - Large-Scale Distributed Computing and Applications ◽

10.4018/978-1-61520-703-9.ch006 ◽

2010 ◽

pp. 111-140

Author(s):

Valentin Cristea ◽

Ciprian Dobre ◽

Corina Stratan ◽

Florin Pop

Keyword(s):

Data Storage ◽

Resource Sharing ◽

High Performance ◽

Large Scale ◽

Workflow Management ◽

Large Data ◽

Data Retrieval ◽

Distributed Data Storage ◽

Processing Power ◽

Data Transfers

The latest advances in network and distributedsystem technologies now allow integration of a vast variety of services with almost unlimited processing power, using large amounts of data. Sharing of resources is often viewed as the key goal for distributed systems, and in this context the sharing of stored data appears as the most important aspect of distributed resource sharing. Scientific applications are the first to take advantage of such environments as the requirements of current and future high performance computing experiments are pressing, in terms of even higher volumes of issued data to be stored and managed. While these new environments reveal huge opportunities for large-scale distributed data storage and management, they also raise important technical challenges, which need to be addressed. The ability to support persistent storage of data on behalf of users, the consistent distribution of up-to-date data, the reliable replication of fast changing datasets or the efficient management of large data transfers are just some of these new challenges. In this chapter we discuss how the existing distributed computing infrastructure is adequate for supporting the required data storage and management functionalities. We highlight the issues raised from storing data over large distributed environments and discuss the recent research efforts dealing with challenges of data retrieval, replication and fast data transfers. Interaction of data management with other data sensitive, emerging technologies as the workflow management is also addressed.

Download Full-text