Task Scheduling and File Replication for Data-Intensive Jobs with Batch-shared I/O

2006 ◽  
Author(s):  
G. Khanna ◽  
N. Vydyanathan ◽  
U. Catalyurek ◽  
T. Kurc ◽  
S. Krishnamoorthy ◽  
...  
Keyword(s):  
Task Scheduling ◽  
Data Intensive ◽  
File Replication

Cost- and Time-Based Data Deployment for Improving Scheduling Efficiency in Distributed Clouds

2020 ◽  
Author(s):  
Chunlin Li ◽  
Yihan Zhang ◽  
Xiaomei Qu ◽  
Youlong Luo
Keyword(s):  
Task Scheduling ◽  
Scheduling Algorithm ◽  
Service Level Agreement ◽  
Data Access ◽  
Service Level ◽  
Speculative Execution ◽  
Improved Genetic Algorithm ◽  
Real Time Processing ◽  
Data Intensive ◽  
Access Cost

Abstract In recent years, with the continuous development of internet of things and cloud computing technologies, data intensive applications have gotten more and more attention. In the distributed cloud environment, the access of massive data is often the bottleneck of its performance. It is very significant to propose a suitable data deployment algorithm for improving the utilization of cloud server and the efficiency of task scheduling. In order to reduce data access cost and data deployment time, an optimal data deployment algorithm is proposed in this paper. By modeling and analyzing the data deployment problem, the problem is solved by using the improved genetic algorithm. After the data are well deployed, aiming at improving the efficiency of task scheduling, a task progress aware scheduling algorithm is proposed in this paper in order to make the speculative execution mechanism more accurate. Firstly, the threshold to detect the slow tasks and fast nodes are set. Then, the slow tasks and fast nodes are detected by calculating the remaining time of the tasks and the real-time processing ability of the nodes, respectively. Finally, the backup execution of the slow tasks is performed on the fast nodes. While satisfying the load balancing of the system, the experimental results show that the proposed algorithms can obviously reduce data access cost, service-level agreement (SLA) default rate and the execution time of the system and optimize data deployment for improving scheduling efficiency in distributed clouds.

Sign in / Sign up

Export Citation Format

Share Document