Coherent Application Delivery on Hybrid Distributed Computing Infrastructures of Virtual Machines and Docker Containers

2017 ◽  
Author(s):  
German Molto ◽  
Miguel Caballer ◽  
Alfonso Perez ◽  
Carlos De Alfonso ◽  
Ignacio Blanquer
Keyword(s):  
Distributed Computing ◽  
Virtual Machines ◽  
Distributed Computing Infrastructures

Clustering error messages produced by distributed computing infrastructure during the processing of high energy physics data

2021 ◽  
Vol 36 (10) ◽  
pp. 2150070
Author(s):  
Maria Grigorieva ◽  
Dmitry Grin
Keyword(s):  
Distributed Computing ◽  
Large Scale ◽  
High Energy Physics ◽  
High Energy ◽  
Machine Learning Algorithms ◽  
Service Failures ◽  
Fault Handling ◽  
Scientific Experiments ◽  
Computing Centers ◽  
Distributed Computing Infrastructures

Large-scale distributed computing infrastructures ensure the operation and maintenance of scientific experiments at the LHC: more than 160 computing centers all over the world execute tens of millions of computing jobs per day. ATLAS — the largest experiment at the LHC — creates an enormous flow of data which has to be recorded and analyzed by a complex heterogeneous and distributed computing environment. Statistically, about 10–12% of computing jobs end with a failure: network faults, service failures, authorization failures, and other error conditions trigger error messages which provide detailed information about the issue, which can be used for diagnosis and proactive fault handling. However, this analysis is complicated by the sheer scale of textual log data, and often exacerbated by the lack of a well-defined structure: human experts have to interpret the detected messages and create parsing rules manually, which is time-consuming and does not allow identifying previously unknown error conditions without further human intervention. This paper is dedicated to the description of a pipeline of methods for the unsupervised clustering of multi-source error messages. The pipeline is data-driven, based on machine learning algorithms, and executed fully automatically, allowing categorizing error messages according to textual patterns and meaning.

Self-healing of workflow activity incidents on distributed computing infrastructures

2013 ◽  
Vol 29 (8) ◽  
pp. 2284-2294 ◽  
Author(s):  
Rafael Ferreira da Silva ◽  
Tristan Glatard ◽  
Frédéric Desprez
Keyword(s):  
Distributed Computing ◽  
Self Healing ◽  
Distributed Computing Infrastructures

scholarly journals Fine-Grain Interoperability of Scientific Workflows in Distributed Computing Infrastructures

2013 ◽  
Vol 11 (3) ◽  
pp. 429-455 ◽  
Author(s):  
Kassian Plankensteiner ◽  
Radu Prodan ◽  
Matthias Janetschek ◽  
Thomas Fahringer ◽  
Johan Montagnat ◽  
...  
Keyword(s):  
Distributed Computing ◽  
Scientific Workflows ◽  
Fine Grain ◽  
Distributed Computing Infrastructures

scholarly journals Service Virtualization Using a Non-von Neumann Parallel, Distributed, and Scalable Computing Model

2012 ◽  
Vol 2012 ◽  
pp. 1-10 ◽  
Author(s):  
Rao Mikkilineni ◽  
Giovanni Morana ◽  
Daniele Zito ◽  
Marco Di Sano
Keyword(s):  
Distributed Computing ◽  
Performance Management ◽  
Virtual Machines ◽  
Dynamic Performance ◽  
Parallel Execution ◽  
Self Management ◽  
Transaction Management ◽  
Von Neumann ◽  
Computing Model ◽  
End To End

This paper describes a prototype implementing a high degree of transaction resilience in distributed software systems using a non-von Neumann computing model exploiting parallelism in computing nodes. The prototype incorporates fault, configuration, accounting, performance, and security (FCAPS) management using a signaling network overlay and allows the dynamic control of a set of distributed computing elements in a network. Each node is a computing entity endowed with self-management and signaling capabilities to collaborate with similar nodes in a network. The separation of parallel computing and management channels allows the end-to-end transaction management of computing tasks (provided by the autonomous distributed computing elements) to be implemented as network-level FCAPS management. While the new computing model is operating system agnostic, a Linux, Apache, MySQL, PHP/Perl/Python (LAMP) based services architecture is implemented in a prototype to demonstrate end-to-end transaction management with auto-scaling, self-repair, dynamic performance management and distributed transaction security assurance. The implementation is made possible by a non-von Neumann middleware library providing Linux process management through multi-threaded parallel execution of self-management and signaling abstractions. We did not use Hypervisors, Virtual machines, or layers of complex virtualization management systems in implementing this prototype.

Sign in / Sign up

Export Citation Format

Share Document