Shared data-aware dynamic resource provisioning and task scheduling for data intensive applications on hybrid clouds using Aneka

2020 ◽  
Vol 106 ◽  
pp. 595-606
Author(s):  
Shreshth Tuli ◽  
Rajinder Sandhu ◽  
Rajkumar Buyya
Keyword(s):  
Task Scheduling ◽  
Resource Provisioning ◽  
Hybrid Clouds ◽  
Data Intensive ◽  
Shared Data ◽  
Dynamic Resource Provisioning ◽  
Dynamic Resource ◽  
Data Intensive Applications ◽  
And Task

Towards Data Intensive Many-Task Computing

2012 ◽  
pp. 28-73 ◽  
Author(s):  
Ioan Raicu ◽  
Ian Foster ◽  
Yong Zhao ◽  
Alex Szalay ◽  
Philip Little ◽  
...  
Keyword(s):  
High Performance ◽  
File Systems ◽  
Data Locality ◽  
Resource Provisioning ◽  
Parallel File Systems ◽  
Data Intensive ◽  
Dynamic Resource Provisioning ◽  
Rate Of Increase ◽  
Parallel File ◽  
Data Intensive Applications

Many-task computing aims to bridge the gap between two computing paradigms, high throughput computing and high performance computing. Traditional techniques to support many-task computing commonly found in scientific computing (i.e. the reliance on parallel file systems with static configurations) do not scale to today’s largest systems for data intensive application, as the rate of increase in the number of processors per system is outgrowing the rate of performance increase of parallel file systems. In this chapter, the authors argue that in such circumstances, data locality is critical to the successful and efficient use of large distributed systems for data-intensive applications. They propose a “data diffusion” approach to enable data-intensive many-task computing. They define an abstract model for data diffusion, define and implement scheduling policies with heuristics that optimize real world performance, and develop a competitive online caching eviction policy. They also offer many empirical experiments to explore the benefits of data diffusion, both under static and dynamic resource provisioning, demonstrating approaches that improve both performance and scalability.

Dynamic Resource Provisioning With Fault Tolerance for Data-Intensive Meteorological Workflows in Cloud

2020 ◽  
Vol 16 (9) ◽  
pp. 6172-6181 ◽  
Author(s):  
Xiaolong Xu ◽  
Ruichao Mo ◽  
Fei Dai ◽  
Wenmin Lin ◽  
Shaohua Wan ◽  
...  
Keyword(s):  
Fault Tolerance ◽  
Resource Provisioning ◽  
Data Intensive ◽  
Dynamic Resource Provisioning ◽  
Dynamic Resource
Sign in / Sign up

Export Citation Format

Share Document