scholarly journals Managing Large-Scale Scientific Workflows in Distributed Environments: Experiences and Challenges

2006 ◽  
Author(s):  
Ewa Deelman ◽  
Yolanda Gil
Keyword(s):  
Large Scale ◽  
Scientific Workflows ◽  
Distributed Environments

Data Management in Scientific Workflows

2012 ◽  
pp. 177-187
Author(s):  
Ewa Deelman ◽  
Ann Chervenak
Keyword(s):  
Data Management ◽  
Gravitational Wave ◽  
Large Scale ◽  
Real Life ◽  
Scientific Workflows ◽  
Workflow Systems ◽  
Automated Generation ◽  
Workflow Execution ◽  
Data Products ◽  
Workflow Planning

Scientific applications such as those in astronomy, earthquake science, gravitational-wave physics, and others have embraced workflow technologies to do large-scale science. Workflows enable researchers to collaboratively design, manage, and obtain results that involve hundreds of thousands of steps, access terabytes of data, and generate similar amounts of intermediate and final data products. Although workflow systems are able to facilitate the automated generation of data products, many issues still remain to be addressed. These issues exist in different forms in the workflow lifecycle. This chapter describes a workflow lifecycle as consisting of a workflow generation phase where the analysis is defined, the workflow planning phase where resources needed for execution are selected, the workflow execution part, where the actual computations take place, and the result, metadata, and provenance storing phase. The authors discuss the issues related to data management at each step of the workflow cycle. They describe challenge problems and illustrate them in the context of real-life applications. They discuss the challenges, possible solutions, and open issues faced when mapping and executing large-scale workflows on current cyberinfrastructure. They particularly emphasize the issues related to the management of data throughout the workflow lifecycle.

scholarly journals An Exotic IWD - SVR Based Approach for Failure Prognostication in Cloud-Based Scientific Workflows

2021 ◽  
Author(s):  
Sridevi S ◽  
Jeevaa Katiravan Jeevaa Katiravan
Keyword(s):  
Large Scale ◽  
Performance Metrics ◽  
Prediction Models ◽  
Fault Tolerant ◽  
Scientific Workflow ◽  
Scientific Workflows ◽  
Support Vector ◽  
Learning Approaches ◽  
Task Failure ◽  
Proactive Measures

Abstract Scientific workflows deserve the emerging attention in sophisticated large-scale scientific problem-solving environments. Though a single task failure occurs in workflow based applications, due to its task dependency nature the reliability of the overall system will be affected drastically. Hence rather than reactive fault tolerant approaches, proactive measures are vital in scientific workflows. This work puts forth an attempt to concentrate on the exploration issue of structuring an Exotic Intelligent Water Drops - Support Vector Regression-based approach for task failure prognostication which facilitates proactive fault tolerance in scientific workflow applications. The failure prediction models in this study have been implemented through SVR-based machine learning approaches and its precision accuracy is optimized by IWDA and various performance metrics were evaluated. The experimental results prove that the proposed approach performs better compared with the other existing techniques.

Sign in / Sign up

Export Citation Format

Share Document