A Data Placement Strategy for Scientific Workflow in Hybrid Cloud

2018 ◽  
Author(s):  
Zhanghui Liu ◽  
Tao Xiang ◽  
Bing Lin ◽  
Xinshu Ye ◽  
Haijiang Wang ◽  
...  
Keyword(s):  
Data Placement ◽  
Scientific Workflow ◽  
Hybrid Cloud

scholarly journals Fuzzy Theory-Based Data Placement for Scientific Workflows in Hybrid Cloud Environments

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Zheyi Chen ◽  
Xu Zhao ◽  
Bing Lin
Keyword(s):  
Data Transmission ◽  
Data Centers ◽  
Fuzzy Theory ◽  
Data Placement ◽  
Scientific Workflow ◽  
Scientific Workflows ◽  
Transmission Time ◽  
Fuzzy Data ◽  
Hybrid Cloud ◽  
Cloud Environments

In hybrid cloud environments, reasonable data placement strategies are critical to the efficient execution of scientific workflows. Due to various loads, bandwidth fluctuations, and network congestions between different data centers as well as the dynamics of hybrid cloud environments, the data transmission time is uncertain. Thus, it poses huge challenges to the efficient data placement for scientific workflows. However, most of the traditional solutions for data placement focus on deterministic cloud environments, which lead to the excessive data transmission time of scientific workflows. To address this problem, we propose an adaptive discrete particle swarm optimization algorithm based on the fuzzy theory and genetic algorithm operators (DPSO-FGA) to minimize the fuzzy data transmission time of scientific workflows. The DPSO-FGA can rationally place the scientific workflow data while meeting the requirements of data privacy and the capacity limitations of data centers. Simulation results show that the DPSO-FGA can effectively reduce the fuzzy data transmission time of scientific workflows in hybrid cloud environments.

Data Placement Oriented Scheduling Algorithm for Scheduling Scientific Workflow in Cloud: A Budget-Aware Approach

2020 ◽  
Vol 13 (5) ◽  
pp. 871-883
Author(s):  
Avinash Kaur ◽  
Pooja Gupta ◽  
Parminder Singh ◽  
Manpreet Singh
Keyword(s):  
State Of The Art ◽  
Scheduling Algorithm ◽  
Cloud Service ◽  
Data Placement ◽  
Scientific Workflow ◽  
Pricing Strategies ◽  
Cloud Service Provider ◽  
Computing Platforms ◽  
Run Up ◽  
Scheduling Heuristic

Background: A large number of communities and enterprises deploy numerous scientific workflow applications on cloud service. Aims: The main aim of the cloud service provider is to execute the workflows with a minimal budget and makespan. Most of the existing techniques for budget and makespan are employed for the traditional platform of computing and are not applicable to cloud computing platforms with unique resource management methods and pricing strategies based on service. Methods: In this paper, we studied the joint optimization of cost and makespan of scheduling workflows in IaaS clouds, and proposed a novel workflow scheduling scheme. Also, data placement is included in the proposed algorithm. Results: In this scheme, DPO-HEFT (Data Placement Oriented HEFT) algorithm is developed which closely integrates the data placement mechanism with the list scheduling heuristic HEFT. Extensive experiments using the real-world and synthetic workflow demonstrate the efficacy of our scheme. Conclusion: Our scheme can achieve significantly better cost and makespan trade-off fronts with remarkably higher hypervolume and can run up to hundreds times faster than the state-of-the-art algorithms.

A Case for Splitting a File for Data Placement in a Distributed Scientific Workflow

2021 ◽  
Author(s):  
Hindol Bhattacharya ◽  
Matangini Chattopadhyay ◽  
Samiran Chattopadhay
Keyword(s):  
Data Placement ◽  
Scientific Workflow

scholarly journals Bi-objective scheduling algorithm for scientific workflows on cloud computing platform with makespan and monetary cost minimization approach

2021 ◽  
Author(s):  
Mirsaeid Hosseini Shirvani ◽  
Reza Noorian Talouki
Keyword(s):  
Cost Minimization ◽  
Scheduling Algorithm ◽  
Scientific Workflow ◽  
Scientific Workflows ◽  
Discrete Problem ◽  
Hybrid Cloud ◽  
Monetary Cost ◽  
Computing Platform ◽  
Cloud Architecture ◽  
Private And Public

AbstractScheduling of scientific workflows on hybrid cloud architecture, which contains private and public clouds, is a challenging task because schedulers should be aware of task inter-dependencies, underlying heterogeneity, cost diversity, and virtual machine (VM) variable configurations during the scheduling process. On the one side, reaching a minimum total execution time or makespan is a favorable issue for users whereas the cost of utilizing quicker VMs may lead to conflict with their budget on the other side. Existing works in the literature scarcely consider VM’s monetary cost in the scheduling process but mainly focus on makespan. Therefore, in this paper, the problem of scientific workflow scheduling running on hybrid cloud architecture is formulated to a bi-objective optimization problem with makespan and monetary cost minimization viewpoint. To address this combinatorial discrete problem, this paper presents a hybrid bi-objective optimization based on simulated annealing and task duplication algorithms (BOSA-TDA) that exploits two important heuristics heterogeneous earliest finish time (HEFT) and duplication techniques to improve canonical SA. The extensive simulation results reported of running different well-known scientific workflows such as LIGO, SIPHT, Cybershake, Montage, and Epigenomics demonstrate that proposed BOSA-TDA has the amount of 12.5%, 14.5%, 17%, 13.5%, and 18.5% average improvement against other existing approaches in terms of makespan, monetary cost, speed up, SLR, and efficiency metrics, respectively.

scholarly journals Data Placement for Privacy-Aware Applications over Big Data in Hybrid Clouds

2017 ◽  
Vol 2017 ◽  
pp. 1-15 ◽  
Author(s):  
Xiaolong Xu ◽  
Xuan Zhao ◽  
Feng Ruan ◽  
Jie Zhang ◽  
Wei Tian ◽  
...  
Keyword(s):  
Big Data ◽  
Energy Consumption ◽  
Data Placement ◽  
Cloud Services ◽  
Hybrid Cloud ◽  
Public Cloud ◽  
Private Cloud ◽  
The Public ◽  
Placement Method ◽  
The Cost

Nowadays, a large number of groups choose to deploy their applications to cloud platforms, especially for the big data era. Currently, the hybrid cloud is one of the most popular computing paradigms for holding the privacy-aware applications driven by the requirements of privacy protection and cost saving. However, it is still a challenge to realize data placement considering both the energy consumption in private cloud and the cost for renting the public cloud services. In view of this challenge, a cost and energy aware data placement method, named CEDP, for privacy-aware applications over big data in hybrid cloud is proposed. Technically, formalized analysis of cost, access time, and energy consumption is conducted in the hybrid cloud environment. Then a corresponding data placement method is designed to accomplish the cost saving for renting the public cloud services and energy savings for task execution within the private cloud platforms. Experimental evaluations validate the efficiency and effectiveness of our proposed method.

scholarly journals A Time-Driven Data Placement Strategy for a Scientific Workflow Combining Edge Computing and Cloud Computing

2019 ◽  
Vol 15 (7) ◽  
pp. 4254-4265 ◽  
Author(s):  
Bing Lin ◽  
Fangning Zhu ◽  
Jianshan Zhang ◽  
Jiaqing Chen ◽  
Xing Chen ◽  
...  
Keyword(s):  
Cloud Computing ◽  
Data Placement ◽  
Scientific Workflow ◽  
Edge Computing
Sign in / Sign up

Export Citation Format

Share Document