A Data Placement Strategy for Scientific Workflow in Hybrid Cloud

Author(s):  
Zhanghui Liu ◽  
Tao Xiang ◽  
Bing Lin ◽  
Xinshu Ye ◽  
Haijiang Wang ◽  
...  
2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Zheyi Chen ◽  
Xu Zhao ◽  
Bing Lin

In hybrid cloud environments, reasonable data placement strategies are critical to the efficient execution of scientific workflows. Due to various loads, bandwidth fluctuations, and network congestions between different data centers as well as the dynamics of hybrid cloud environments, the data transmission time is uncertain. Thus, it poses huge challenges to the efficient data placement for scientific workflows. However, most of the traditional solutions for data placement focus on deterministic cloud environments, which lead to the excessive data transmission time of scientific workflows. To address this problem, we propose an adaptive discrete particle swarm optimization algorithm based on the fuzzy theory and genetic algorithm operators (DPSO-FGA) to minimize the fuzzy data transmission time of scientific workflows. The DPSO-FGA can rationally place the scientific workflow data while meeting the requirements of data privacy and the capacity limitations of data centers. Simulation results show that the DPSO-FGA can effectively reduce the fuzzy data transmission time of scientific workflows in hybrid cloud environments.


2020 ◽  
Vol 13 (5) ◽  
pp. 871-883
Author(s):  
Avinash Kaur ◽  
Pooja Gupta ◽  
Parminder Singh ◽  
Manpreet Singh

Background: A large number of communities and enterprises deploy numerous scientific workflow applications on cloud service. Aims: The main aim of the cloud service provider is to execute the workflows with a minimal budget and makespan. Most of the existing techniques for budget and makespan are employed for the traditional platform of computing and are not applicable to cloud computing platforms with unique resource management methods and pricing strategies based on service. Methods: In this paper, we studied the joint optimization of cost and makespan of scheduling workflows in IaaS clouds, and proposed a novel workflow scheduling scheme. Also, data placement is included in the proposed algorithm. Results: In this scheme, DPO-HEFT (Data Placement Oriented HEFT) algorithm is developed which closely integrates the data placement mechanism with the list scheduling heuristic HEFT. Extensive experiments using the real-world and synthetic workflow demonstrate the efficacy of our scheme. Conclusion: Our scheme can achieve significantly better cost and makespan trade-off fronts with remarkably higher hypervolume and can run up to hundreds times faster than the state-of-the-art algorithms.


Author(s):  
Hindol Bhattacharya ◽  
Matangini Chattopadhyay ◽  
Samiran Chattopadhay

Author(s):  
Mirsaeid Hosseini Shirvani ◽  
Reza Noorian Talouki

AbstractScheduling of scientific workflows on hybrid cloud architecture, which contains private and public clouds, is a challenging task because schedulers should be aware of task inter-dependencies, underlying heterogeneity, cost diversity, and virtual machine (VM) variable configurations during the scheduling process. On the one side, reaching a minimum total execution time or makespan is a favorable issue for users whereas the cost of utilizing quicker VMs may lead to conflict with their budget on the other side. Existing works in the literature scarcely consider VM’s monetary cost in the scheduling process but mainly focus on makespan. Therefore, in this paper, the problem of scientific workflow scheduling running on hybrid cloud architecture is formulated to a bi-objective optimization problem with makespan and monetary cost minimization viewpoint. To address this combinatorial discrete problem, this paper presents a hybrid bi-objective optimization based on simulated annealing and task duplication algorithms (BOSA-TDA) that exploits two important heuristics heterogeneous earliest finish time (HEFT) and duplication techniques to improve canonical SA. The extensive simulation results reported of running different well-known scientific workflows such as LIGO, SIPHT, Cybershake, Montage, and Epigenomics demonstrate that proposed BOSA-TDA has the amount of 12.5%, 14.5%, 17%, 13.5%, and 18.5% average improvement against other existing approaches in terms of makespan, monetary cost, speed up, SLR, and efficiency metrics, respectively.


2017 ◽  
Vol 2017 ◽  
pp. 1-15 ◽  
Author(s):  
Xiaolong Xu ◽  
Xuan Zhao ◽  
Feng Ruan ◽  
Jie Zhang ◽  
Wei Tian ◽  
...  

Nowadays, a large number of groups choose to deploy their applications to cloud platforms, especially for the big data era. Currently, the hybrid cloud is one of the most popular computing paradigms for holding the privacy-aware applications driven by the requirements of privacy protection and cost saving. However, it is still a challenge to realize data placement considering both the energy consumption in private cloud and the cost for renting the public cloud services. In view of this challenge, a cost and energy aware data placement method, named CEDP, for privacy-aware applications over big data in hybrid cloud is proposed. Technically, formalized analysis of cost, access time, and energy consumption is conducted in the hybrid cloud environment. Then a corresponding data placement method is designed to accomplish the cost saving for renting the public cloud services and energy savings for task execution within the private cloud platforms. Experimental evaluations validate the efficiency and effectiveness of our proposed method.


2019 ◽  
Vol 15 (7) ◽  
pp. 4254-4265 ◽  
Author(s):  
Bing Lin ◽  
Fangning Zhu ◽  
Jianshan Zhang ◽  
Jiaqing Chen ◽  
Xing Chen ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document