scholarly journals Progress on H5Part: a portable high performance parallel data interface for electromagnetics simulations

2007 ◽  
Author(s):  
A. Adelmann ◽  
A. Gsell ◽  
B. Oswald ◽  
T. Schietinger ◽  
W. Bethel ◽  
...  
Keyword(s):  
High Performance ◽  
Parallel Data

scholarly journals Bonsai: An event-based framework for processing and controlling data streams

2014 ◽  
Author(s):  
Gonçalo Lopes ◽  
Niccolò Bonacchi ◽  
João Frazão ◽  
Joana P. Neto ◽  
Bassam V. Atallah ◽  
...  
Keyword(s):  
Data Streams ◽  
High Performance ◽  
Visual Programming ◽  
Scientific Data ◽  
Video Tracking ◽  
Loop Control ◽  
Online Processing ◽  
Parallel Data ◽  
Develop Software ◽  
Event Based

The design of modern scientific experiments requires the control and monitoring of many parallel data streams. However, the serial execution of programming instructions in a computer makes it a challenge to develop software that can deal with the asynchronous, parallel nature of scientific data. Here we present Bonsai, a modular, high-performance, open-source visual programming framework for the acquisition and online processing of data streams. We describe Bonsai's core principles and architecture and demonstrate how it allows for flexible and rapid prototyping of integrated experimental designs in neuroscience. We specifically highlight different possible applications which require the combination of many different hardware and software components, including behaviour video tracking, electrophysiology and closed-loop control of stimulation parameters.

scholarly journals Memory mapped parallel BAM file access API for high throughput sequence analysis informatics

2021 ◽  
Author(s):  
Timothy A. Pitman ◽  
Xiaomeng Huang ◽  
Gabor T Marth ◽  
Yi Qiao
Keyword(s):  
Sequence Analysis ◽  
Precision Medicine ◽  
High Performance ◽  
Sequencing Data ◽  
Analysis Software ◽  
File Access ◽  
Parallel Data ◽  
Timely Fashion ◽  
New Generation ◽  
Informatics Tools

In precision medicine, genomic data needs to be processed as fast as possible to arrive at treatment decisions in a timely fashion. We developed mmbam, a library to allow sequence analysis informatics software to access raw sequencing data stored in BAM files extremely fast. Taking advantage of memory mapped file access and parallel data processing, we demonstrate that analysis software ported to mmbam consistently outperforms their stock versions. Open source and freely available, we envision that mmbam will enable a new generation of high performance informatics tools for precision medicine.

Abstractions and Middleware for Petascale Computing and Beyond

2012 ◽  
pp. 161-178
Author(s):  
Ivo F. Sbalzarini
Keyword(s):  
Numerical Simulations ◽  
High Performance Computing ◽  
Data Structures ◽  
High Performance ◽  
Source Code ◽  
Limiting Factors ◽  
Communication Overhead ◽  
Parallel Data ◽  
Petascale Computing ◽  
Performance Computing

As high-performance computing moves to the petascale and beyond, a number of algorithmic and software challenges need to be addressed. This paper reviews the main performance-limiting factors in today’s high-performance computing software and outlines a possible new programming paradigm to address them. The proposed paradigm is based on abstract parallel data structures and operations that encapsulate much of the complexity of an application, but still make communication overhead explicit. The authors argue that all numerical simulations can be formulated in terms of the presented abstractions, which thus define an abstract semantic specification language for parallel numerical simulations. Simulations defined in this language can automatically be translated to source code containing the appropriate calls to a middleware that implements the underlying abstractions. Finally, the structure and functionality of such a middleware are outlined while demonstrating its feasibility on the example of the parallel particle-mesh library (PPM).

Abstractions and Middleware for Petascale Computing and Beyond

2010 ◽  
Vol 1 (2) ◽  
pp. 40-56 ◽  
Author(s):  
Ivo F. Sbalzarini
Keyword(s):  
Numerical Simulations ◽  
High Performance Computing ◽  
High Performance ◽  
Source Code ◽  
Limiting Factors ◽  
Communication Overhead ◽  
Programming Paradigm ◽  
Parallel Data ◽  
Petascale Computing ◽  
Performance Computing

As high-performance computing moves to the petascale and beyond, a number of algorithmic and software challenges need to be addressed. This paper reviews the main performance-limiting factors in today’s high-performance computing software and outlines a possible new programming paradigm to address them. The proposed paradigm is based on abstract parallel data structures and operations that encapsulate much of the complexity of an application, but still make communication overhead explicit. The authors argue that all numerical simulations can be formulated in terms of the presented abstractions, which thus define an abstract semantic specification language for parallel numerical simulations. Simulations defined in this language can automatically be translated to source code containing the appropriate calls to a middleware that implements the underlying abstractions. Finally, the structure and functionality of such a middleware are outlined while demonstrating its feasibility on the example of the parallel particle-mesh library (PPM).

scholarly journals Unsupervised Interlingual Semantic Representations from Sentence Embeddings for Zero-Shot Cross-Lingual Transfer

2020 ◽  
Vol 34 (05) ◽  
pp. 7944-7951
Author(s):  
Channy Hong ◽  
Jaeyeon Lee ◽  
Jungkwon Lee
Keyword(s):  
Natural Language ◽  
High Performance ◽  
Data Sets ◽  
Semantic Representations ◽  
Parallel Corpora ◽  
Low Resource ◽  
Parallel Data ◽  
Language Data ◽  
Cross Lingual ◽  
Inference Task

As numerous modern NLP models demonstrate high-performance in various tasks when trained with resource-rich language data sets such as those of English, there has been a shift in attention to the idea of applying such learning to low-resource languages via zero-shot or few-shot cross-lingual transfer. While the most prominent efforts made previously on achieving this feat entails the use of parallel corpora for sentence alignment training, we seek to generalize further by assuming plausible scenarios in which such parallel data sets are unavailable. In this work, we present a novel architecture for training interlingual semantic representations on top of sentence embeddings in a completely unsupervised manner, and demonstrate its effectiveness in zero-shot cross-lingual transfer in natural language inference task. Furthermore, we showcase a method of leveraging this framework in a few-shot scenario, and finally analyze the distributional and permutational alignment across languages of these interlingual semantic representations.

Parallel Processing in Network Processor Architectures (Parallelverarbeitung in Netzwerkprozessorarchitekturen)

2005 ◽  
Vol 47 (3) ◽  
Author(s):  
Rainer Hagenau ◽  
Carsten Albrecht ◽  
Erik Maehle ◽  
Andreas C. Döring
Keyword(s):  
Parallel Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Network Processor ◽  
Processor Architectures ◽  
Parallel Data ◽  
Data Plane ◽  
Parallel Network ◽  
Basic Approaches ◽  
Performance Computing

SummuryParallel processing is well established in high-performance computing. Currently, network processors as new emerging, special-purpose processors are targeted at the exploitation of parallelism to meet the requirements in data-plane processing with wire-speed. The achievable level of parallelism is determined by decisions in the architecture design and by the characteristics of the data-plane applications executed. We discuss two basic approaches in parallel processing, namely pipelining and concurrency, which establish basic models for parallel network processor organization. The features and constraints of these models are studied. Using this background some existing network processor architectures are reviewed and characterized regarding their potential in parallel data-plane processing.

Abstractions and Middleware for Petascale Computing and Beyond

2012 ◽  
pp. 1998-2015
Author(s):  
Ivo F. Sbalzarini
Keyword(s):  
Numerical Simulations ◽  
High Performance Computing ◽  
Data Structures ◽  
High Performance ◽  
Source Code ◽  
Limiting Factors ◽  
Communication Overhead ◽  
Parallel Data ◽  
Petascale Computing ◽  
Performance Computing

As high-performance computing moves to the petascale and beyond, a number of algorithmic and software challenges need to be addressed. This paper reviews the main performance-limiting factors in today’s high-performance computing software and outlines a possible new programming paradigm to address them. The proposed paradigm is based on abstract parallel data structures and operations that encapsulate much of the complexity of an application, but still make communication overhead explicit. The authors argue that all numerical simulations can be formulated in terms of the presented abstractions, which thus define an abstract semantic specification language for parallel numerical simulations. Simulations defined in this language can automatically be translated to source code containing the appropriate calls to a middleware that implements the underlying abstractions. Finally, the structure and functionality of such a middleware are outlined while demonstrating its feasibility on the example of the parallel particle-mesh library (PPM).

Sign in / Sign up

Export Citation Format

Share Document