Optimizing Large File Transfer on Data Grid

2005 ◽  
pp. 455-460
Author(s):  
Teng Ma ◽  
Junzhou Luo
Keyword(s):  
Data Grid ◽  
File Transfer ◽  
Large File

Replica Placement Strategy for Data Grid Environment

2013 ◽  
Vol 5 (1) ◽  
pp. 70-81 ◽  
Author(s):  
Mohammed K. Madi ◽  
Yuhanis Yusof ◽  
Suhaidi Hassan
Keyword(s):  
Resource Sharing ◽  
Data Access ◽  
Data Grid ◽  
Computational Time ◽  
Replica Placement ◽  
File Transfer ◽  
Large Bandwidth ◽  
Multiple Copies ◽  
Data Files ◽  
Computational Resources

Data Grid is an infrastructure that manages huge amount of data files, and provides intensive computational resources across geographically distributed collaboration. To increase resource availability and to ease resource sharing in such environment, there is a need for replication services. Data replication is one of the methods used to improve the performance of data access in distributed systems by replicating multiple copies of data files in the distributed sites. Replica placement mechanism is the process of identifying where to place copies of replicated data files in a Grid system. Existing work identifies the suitable sites based on number of requests and read cost of the required file. Such approaches consume large bandwidth and increases the computational time. The authors propose a replica placement strategy (RPS) that finds the best locations to store replicas based on four criteria, namely, 1) Read Cost, 2) File Transfer Time, 3) Sites’ Workload, and 4) Replication Sites. OptorSim is used to evaluate the performance of this replica placement strategy. The simulation results show that RPS requires less execution time and consumes less network usage compared to existing approaches of Simple Optimizer and LFU (Least Frequently Used).

scholarly journals Performance impact of large file transfer on web proxy caching: A case study in a high bandwidth campus network environment

2010 ◽  
Vol 12 (1) ◽  
pp. 52-66 ◽  
Author(s):  
Hyun-Chul Kim ◽  
Dongman Lee ◽  
Kilnam Chon ◽  
Beakcheol Jang ◽  
Taekyoung Kwon ◽  
...  
Keyword(s):  
Network Environment ◽  
Campus Network ◽  
File Transfer ◽  
Performance Impact ◽  
Proxy Caching ◽  
High Bandwidth ◽  
Large File

scholarly journals A four-phase data replication algorithm for data grid

2015 ◽  
Vol 4 (1) ◽  
pp. 163 ◽  
Author(s):  
Alireza Saleh ◽  
Reza Javidan ◽  
Mohammad Taghi FatehiKhajeh
Keyword(s):  
Large Scale ◽  
Data Replication ◽  
Data Grid ◽  
Data Availability ◽  
Access Time ◽  
File Transfer ◽  
System Availability ◽  
Large Scale Data ◽  
Effective Network ◽  
Average Access Time

<p>Nowadays, scientific applications generate a huge amount of data in terabytes or petabytes. Data grids currently proposed solutions to large scale data management problems including efficient file transfer and replication. Data is typically replicated in a Data Grid to improve the job response time and data availability. A reasonable number and right locations for replicas has become a challenge in the Data Grid. In this paper, a four-phase dynamic data replication algorithm based on Temporal and Geographical locality is proposed. It includes: 1) evaluating and identifying the popular data and triggering a replication operation when the popularity data passes a dynamic threshold; 2) analyzing and modeling the relationship between system availability and the number of replicas, and calculating a suitable number of new replicas; 3) evaluating and identifying the popular data in each site, and placing replicas among them; 4) removing files with least cost of average access time when encountering insufficient space for replication. The algorithm was tested using a grid simulator, OptorSim developed by European Data Grid Projects. The simulation results show that the proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, effective network usage and percentage of storage filled.</p>

Joint Bandwidth Assignment and Routing for Power Saving on Large File Transfer with Time Constraints

2020 ◽  
Vol E103.B (4) ◽  
pp. 431-439
Author(s):  
Kazuhiko KINOSHITA ◽  
Masahiko AIHARA ◽  
Nariyoshi YAMAI ◽  
Takashi WATANABE
Keyword(s):  
Power Saving ◽  
Time Constraints ◽  
File Transfer ◽  
Large File

Joint Bandwidth Scheduling and Routing Method for Large File Transfer with Time Constraint and Its Implementation

2018 ◽  
Vol E101.B (3) ◽  
pp. 763-771 ◽  
Author(s):  
Kazuhiko KINOSHITA ◽  
Masahiko AIHARA ◽  
Shiori KONO ◽  
Nariyoshi YAMAI ◽  
Takashi WATANABE
Keyword(s):  
Time Constraint ◽  
File Transfer ◽  
Bandwidth Scheduling ◽  
Routing Method ◽  
Large File
Sign in / Sign up

Export Citation Format

Share Document