scholarly journals A Flexible Framework for Asynchronous in Situ and in Transit Analytics for Scientific Simulations

2014 ◽  
Author(s):  
Matthieu Dreher ◽  
Bruno Raffin
Keyword(s):  
Scientific Simulations ◽  
Flexible Framework ◽  
In Transit

Summary of the SciTech 2020 Technical Panel on In Situ/In Transit Computational Environments for Visualization and Data Analytics

2021 ◽  
Author(s):  
Earl P. Duque ◽  
Steve M. Legensky ◽  
Brad J. Whitlock ◽  
David H. Rogers ◽  
Andrew C. Bauer ◽  
...  
Keyword(s):  
Data Analytics ◽  
In Transit

scholarly journals Accelerating In-Transit Co-Processing for Scientific Simulations Using Region-Based Data-Driven Analysis

Algorithms ◽  
2021 ◽  
Vol 14 (5) ◽  
pp. 154
Author(s):  
Marcus Walldén ◽  
Masao Okita ◽  
Fumihiko Ino ◽  
Dimitris Drikakis ◽  
Ioannis Kokkinakis
Keyword(s):  
Large Scale ◽  
Data Driven ◽  
Data Sets ◽  
Output Constraints ◽  
Data Driven Approach ◽  
Scientific Simulations ◽  
Multiple Metrics ◽  
In Transit ◽  
Multiple Compression ◽  
Large Scale Simulations

Increasing processing capabilities and input/output constraints of supercomputers have increased the use of co-processing approaches, i.e., visualizing and analyzing data sets of simulations on the fly. We present a method that evaluates the importance of different regions of simulation data and a data-driven approach that uses the proposed method to accelerate in-transit co-processing of large-scale simulations. We use the importance metrics to simultaneously employ multiple compression methods on different data regions to accelerate the in-transit co-processing. Our approach strives to adaptively compress data on the fly and uses load balancing to counteract memory imbalances. We demonstrate the method’s efficiency through a fluid mechanics application, a Richtmyer–Meshkov instability simulation, showing how to accelerate the in-transit co-processing of simulations. The results show that the proposed method expeditiously can identify regions of interest, even when using multiple metrics. Our approach achieved a speedup of 1.29× in a lossless scenario. The data decompression time was sped up by 2× compared to using a single compression method uniformly.

In Situ and In Transit Computing for Large Scale Geoscientific Simulation

2020 ◽  
Author(s):  
Sebastian Friedemann ◽  
Bruno Raffin ◽  
Basile Hector ◽  
Jean-Martial Cohard
Keyword(s):  
Data Assimilation ◽  
High Performance ◽  
Large Scale ◽  
Transfer Model ◽  
Simulation Data ◽  
Computing Platform ◽  
Continental Scale ◽  
Physically Based ◽  
In Transit

<p>In situ and in transit computing is an effective way to place postprocessing and preprocessing tasks for large scale simulations on the high performance computing platform. The resulting proximity between the execution of preprocessing, simulation and postprocessing permits to lower I/O by bypassing slow and energy inefficient persistent storages. This permits to scale workflows consisting of heterogeneous components such as simulation, data analysis and visualization, to modern massively parallel high performance platforms. Reordering the workflow components gives a manifold of new advanced data processing possibilities for research. Thus in situ and in transit computing are vital for advances in the domain of geoscientific simulation which relies on the increasing amount of sensor and simulation data available.</p><p>In this talk, different in situ and in transit workflows, especially those that are useful in the field of geoscientific simulation, are discussed. Furthermore our experiences augmenting ParFlow-CLM, a physically based, state-of-the-art, fully coupled water transfer model for the critical zone, with FlowVR, an in situ framework with a strict component paradigm, are presented.<br>This allows shadowed in situ file writing, in situ online steering and in situ visualization.</p><p>In situ frameworks further can be coupled to data assimilation tools.<br>In the on going EoCoE-II we propose to embed data assimilation codes into an in transit computing environment. This is expected to enable ensemble based data assimilation on continental scale hydrological simulations with multiple thousands of ensemble members.</p>

scholarly journals A flexible framework for multi-particle refinement in cryo-electron tomography

PLoS Biology ◽  
2021 ◽  
Vol 19 (8) ◽  
pp. e3001319
Author(s):  
Alister Burt ◽  
Lorenzo Gaifas ◽  
Tom Dendooven ◽  
Irina Gutsche
Keyword(s):  
Software Package ◽  
Electron Tomography ◽  
Macromolecular Structure ◽  
Computational Tools ◽  
Cryo Electron Tomography ◽  
Flexible Framework ◽  
Subtomogram Averaging ◽  
Particle Refinement ◽  
Hiv 1

Cryo-electron tomography (cryo-ET) and subtomogram averaging (STA) are increasingly used for macromolecular structure determination in situ. Here, we introduce a set of computational tools and resources designed to enable flexible approaches to STA through increased automation and simplified metadata handling. We create a bidirectional interface between the Dynamo software package and the Warp-Relion-M pipeline, providing a framework for ab initio and geometrical approaches to multiparticle refinement in M. We illustrate the power of working within this framework by applying it to EMPIAR-10164, a publicly available dataset containing immature HIV-1 virus-like particles (VLPs), and a challenging in situ dataset containing chemosensory arrays in bacterial minicells. Additionally, we provide a comprehensive, step-by-step guide to obtaining a 3.4-Å reconstruction from EMPIAR-10164. The guide is hosted on https://teamtomo.org/, a collaborative online platform we establish for sharing knowledge about cryo-ET.

scholarly journals Melanoma in situ with in-transit metastases

JPRAS Open ◽  
2017 ◽  
Vol 11 ◽  
pp. 37-42
Author(s):  
Joachim Mikkelsen ◽  
Anne Lene Hagen Wagenblast ◽  
Nille Behrendt ◽  
Jørgen Lock-Andersen
Keyword(s):  
Melanoma In Situ ◽  
In Transit

scholarly journals In Situ Monitoring of Unique Switching Transitions in the Pressure-Amplifying Flexible Framework Material DUT-49 by High-Pressure 129Xe NMR Spectroscopy

2017 ◽  
Vol 121 (9) ◽  
pp. 5195-5200 ◽  
Author(s):  
Jana Schaber ◽  
Simon Krause ◽  
Silvia Paasch ◽  
Irena Senkovska ◽  
Volodymyr Bon ◽  
...  
Keyword(s):  
High Pressure ◽  
Nmr Spectroscopy ◽  
In Situ Monitoring ◽  
129Xe Nmr ◽  
Flexible Framework

scholarly journals In situ and in-transit analysis of cosmological simulations

2016 ◽  
Vol 3 (1) ◽  
Author(s):  
Brian Friesen ◽  
Ann Almgren ◽  
Zarija Lukić ◽  
Gunther Weber ◽  
Dmitriy Morozov ◽  
...  
Keyword(s):  
Cosmological Simulations ◽  
In Transit
Sign in / Sign up

Export Citation Format

Share Document