scholarly journals A New Paradigm in Data Intensive Computing: Stork and the Data-Aware Schedulers

2006 ◽  
Author(s):  
T. Kosar
Keyword(s):  
New Paradigm ◽  
Data Intensive Computing ◽  
Data Intensive

A Throughput-Optimized Optical Network for Data-Intensive Computing

IEEE Micro ◽  
2014 ◽  
Vol 34 (5) ◽  
pp. 52-63 ◽  
Author(s):  
Laurent Schares ◽  
Benjamin G. Lee ◽  
Fabio Checconi ◽  
Russell Budd ◽  
Alexander Rylyakov ◽  
...  
Keyword(s):  
Optical Network ◽  
Data Intensive Computing ◽  
Data Intensive

scholarly journals Performance Evaluations of Distributed File Systems for Scientific Big Data in FUSE Environment

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1471
Author(s):  
Jun-Yeong Lee ◽  
Moon-Hyun Kim ◽  
Syed Asif Raza Raza Shah ◽  
Sang-Un Ahn ◽  
Heejun Yoon ◽  
...  
Keyword(s):  
Data Storage ◽  
Scale Up ◽  
File Systems ◽  
Performance Evaluations ◽  
Distributed File Systems ◽  
Data Intensive Computing ◽  
Data Intensive ◽  
Tremendous Amount ◽  
Computing Environments ◽  
And Performance

Data are important and ever growing in data-intensive scientific environments. Such research data growth requires data storage systems that play pivotal roles in data management and analysis for scientific discoveries. Redundant Array of Independent Disks (RAID), a well-known storage technology combining multiple disks into a single large logical volume, has been widely used for the purpose of data redundancy and performance improvement. However, this requires RAID-capable hardware or software to build up a RAID-enabled disk array. In addition, it is difficult to scale up the RAID-based storage. In order to mitigate such a problem, many distributed file systems have been developed and are being actively used in various environments, especially in data-intensive computing facilities, where a tremendous amount of data have to be handled. In this study, we investigated and benchmarked various distributed file systems, such as Ceph, GlusterFS, Lustre and EOS for data-intensive environments. In our experiment, we configured the distributed file systems under a Reliable Array of Independent Nodes (RAIN) structure and a Filesystem in Userspace (FUSE) environment. Our results identify the characteristics of each file system that affect the read and write performance depending on the features of data, which have to be considered in data-intensive computing environments.

Sign in / Sign up

Export Citation Format

Share Document