A Case for the Optimization of Database Workflow Orchestration in Heterogenous Environments

2010 ◽  
Author(s):  
Werner Mach ◽  
Erich Schikuta
Keyword(s):  
Workflow Orchestration

A Step Change in the Digital Oilfield Arena: Cloud Computing and Workflow Integration for Production Operations Solutions

2021 ◽  
Author(s):  
Aditya Kotiyal ◽  
Guru Prasad Nagaraj ◽  
Lester Tugung Michael
Keyword(s):  
Oil And Gas ◽  
Dimensional Change ◽  
Oil And Gas Industry ◽  
Gas Production ◽  
System Stability ◽  
Production Data ◽  
Oil And Gas Production ◽  
Common Domain ◽  
Digital Oilfield ◽  
Workflow Orchestration

Abstract Digital oilfield applications have been implemented in numerous operating companies to streamline processes and automate workflows to optimize oil and gas production in real-time. These applications are mostly deployed using traditional on-premises systems; where maintenance, accessibility and scalability serves as a major bottleneck for an efficient outcome. In addition to this challenge, the sector still faces limitations in data integration from disparate data sources, liberation of consolidated data for consumption and cross domain workflow orchestration of that data. The dimensional change brought by digital transformation strategies has paved a path for the Cloud- based solutions, which have recently gained momentum in the oil and gas industry pertaining to their wider accessibility, simpler customization, greater system stability and scalability to support larger amount of data in a performant way. To address the challenges mentioned earlier, we have embarked on a journey with Production Data Foundation which brings together production and equipment data from across an organization. In this paper, we will highlight how Production Data Foundation, hosted on the cloud, provides the underlying infrastructure, services, interfaces required to support and unify production data ingestion, workflow orchestration, and through the alignment of the common domain and digital concepts, improve collaboration between people in distinct roles, such as production engineers, reservoir engineers, drilling engineers, deployment engineers, software developers, data scientists, architects, and subject matter experts (SME) working with production operations products and solutions.

Polyphony: A Workflow Orchestration Framework for Cloud Computing

2010 ◽  
Author(s):  
Khawaja S. Shams ◽  
Mark W. Powell ◽  
Tom M. Crockett ◽  
Jeffrey S. Norris ◽  
Ryan Rossi ◽  
...  
Keyword(s):  
Cloud Computing ◽  
Workflow Orchestration

scholarly journals Trust enforcement through self-adapting cloud workflow orchestration

2019 ◽  
Vol 97 ◽  
pp. 462-481 ◽  
Author(s):  
Hadeel T. El-Kassabi ◽  
M. Adel Serhani ◽  
Rachida Dssouli ◽  
Alramzana N. Navaz
Keyword(s):  
Workflow Orchestration

scholarly journals Two-level Dynamic Workflow Orchestration in the INDIGO DataCloud for Large-scale, Climate Change Data Analytics Experiments

2016 ◽  
Vol 80 ◽  
pp. 722-733 ◽  
Author(s):  
Marcin Płóciennik ◽  
Sandro Fiore ◽  
Giacinto Donvito ◽  
Michał Owsiak ◽  
Marco Fargetta ◽  
...  
Keyword(s):  
Climate Change ◽  
Data Analytics ◽  
Large Scale ◽  
Dynamic Workflow ◽  
Workflow Orchestration

Scientific workflow orchestration interoperating HTC and HPC resources

2011 ◽  
Vol 182 (4) ◽  
pp. 890-897 ◽  
Author(s):  
Luis Cabellos ◽  
Isabel Campos ◽  
Enol Fernández-del-Castillo ◽  
Michał Owsiak ◽  
Bartek Palak ◽  
...  
Keyword(s):  
Scientific Workflow ◽  
Workflow Orchestration

scholarly journals BioExcel Building Blocks, a software library for interoperable biomolecular simulation workflows

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Pau Andrio ◽  
Adam Hospital ◽  
Javier Conejero ◽  
Luis Jordá ◽  
Marc Del Pino ◽  
...  
Keyword(s):  
Building Blocks ◽  
Biological Processes ◽  
Software Management ◽  
Software Library ◽  
Management Tools ◽  
Software Packages ◽  
Biomolecular Simulations ◽  
Software And Hardware ◽  
A Chain ◽  
Workflow Orchestration

Abstract In the recent years, the improvement of software and hardware performance has made biomolecular simulations a mature tool for the study of biological processes. Simulation length and the size and complexity of the analyzed systems make simulations both complementary and compatible with other bioinformatics disciplines. However, the characteristics of the software packages used for simulation have prevented the adoption of the technologies accepted in other bioinformatics fields like automated deployment systems, workflow orchestration, or the use of software containers. We present here a comprehensive exercise to bring biomolecular simulations to the “bioinformatics way of working”. The exercise has led to the development of the BioExcel Building Blocks (BioBB) library. BioBB’s are built as Python wrappers to provide an interoperable architecture. BioBB’s have been integrated in a chain of usual software management tools to generate data ontologies, documentation, installation packages, software containers and ways of integration with workflow managers, that make them usable in most computational environments.

scholarly journals Buttressing volatile desktop grids with cloud resources within a reconfigurable environment service for workflow orchestration

2014 ◽  
Vol 3 (1) ◽  
pp. 1 ◽  
Author(s):  
Stephen C Winter ◽  
Christopher J Reynolds ◽  
Tamas Kiss ◽  
Gabor Z Terstyanszky ◽  
Pamela Greenwell ◽  
...  
Keyword(s):  
Desktop Grids ◽  
Environment Service ◽  
Workflow Orchestration ◽  
Cloud Resources

scholarly journals Big Data Workflows: Locality-Aware Orchestration Using Software Containers

Sensors ◽  
2021 ◽  
Vol 21 (24) ◽  
pp. 8212
Author(s):  
Andrei-Alin Corodescu ◽  
Nikolay Nikolov ◽  
Akif Quddus Khan ◽  
Ahmet Soylu ◽  
Mihhail Matskin ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Locality ◽  
Computing Paradigm ◽  
Limited Support ◽  
Execution Speed ◽  
Significant Performance ◽  
Geographically Distributed ◽  
Workflow Orchestration ◽  
Data Centres

The emergence of the edge computing paradigm has shifted data processing from centralised infrastructures to heterogeneous and geographically distributed infrastructures. Therefore, data processing solutions must consider data locality to reduce the performance penalties from data transfers among remote data centres. Existing big data processing solutions provide limited support for handling data locality and are inefficient in processing small and frequent events specific to the edge environments. This article proposes a novel architecture and a proof-of-concept implementation for software container-centric big data workflow orchestration that puts data locality at the forefront. The proposed solution considers the available data locality information, leverages long-lived containers to execute workflow steps, and handles the interaction with different data sources through containers. We compare the proposed solution with Argo workflows and demonstrate a significant performance improvement in the execution speed for processing the same data units. Finally, we carry out experiments with the proposed solution under different configurations and analyze individual aspects affecting the performance of the overall solution.

scholarly journals Locality-Aware Workflow Orchestration for Big Data

2021 ◽  
Author(s):  
Andrei-Alin Corodescu ◽  
Nikolay Nikolov ◽  
Akif Quddus Khan ◽  
Ahmet Soylu ◽  
Mihhail Matskin ◽  
...  
Keyword(s):  
Big Data ◽  
Workflow Orchestration
Sign in / Sign up

Export Citation Format

Share Document