Self-Healing of Operational Workflow Incidents on Distributed Computing Infrastructures

2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012) ◽

10.1109/ccgrid.2012.24 ◽

2012 ◽

Cited By ~ 7

Author(s):

Rafael Ferreira da Silva ◽

Tristan Glatard ◽

Frédéric Desprez

Keyword(s):

Distributed Computing ◽

Self Healing ◽

Distributed Computing Infrastructures

Download Full-text

Self-healing of workflow activity incidents on distributed computing infrastructures

Future Generation Computer Systems ◽

10.1016/j.future.2013.06.012 ◽

2013 ◽

Vol 29 (8) ◽

pp. 2284-2294 ◽

Cited By ~ 14

Author(s):

Rafael Ferreira da Silva ◽

Tristan Glatard ◽

Frédéric Desprez

Keyword(s):

Distributed Computing ◽

Self Healing ◽

Distributed Computing Infrastructures

Download Full-text

Clustering error messages produced by distributed computing infrastructure during the processing of high energy physics data

International Journal of Modern Physics A ◽

10.1142/s0217751x21500706 ◽

2021 ◽

Vol 36 (10) ◽

pp. 2150070

Author(s):

Maria Grigorieva ◽

Dmitry Grin

Keyword(s):

Distributed Computing ◽

Large Scale ◽

High Energy Physics ◽

High Energy ◽

Machine Learning Algorithms ◽

Service Failures ◽

Fault Handling ◽

Scientific Experiments ◽

Computing Centers ◽

Distributed Computing Infrastructures

Large-scale distributed computing infrastructures ensure the operation and maintenance of scientific experiments at the LHC: more than 160 computing centers all over the world execute tens of millions of computing jobs per day. ATLAS — the largest experiment at the LHC — creates an enormous flow of data which has to be recorded and analyzed by a complex heterogeneous and distributed computing environment. Statistically, about 10–12% of computing jobs end with a failure: network faults, service failures, authorization failures, and other error conditions trigger error messages which provide detailed information about the issue, which can be used for diagnosis and proactive fault handling. However, this analysis is complicated by the sheer scale of textual log data, and often exacerbated by the lack of a well-defined structure: human experts have to interpret the detected messages and create parsing rules manually, which is time-consuming and does not allow identifying previously unknown error conditions without further human intervention. This paper is dedicated to the description of a pipeline of methods for the unsupervised clustering of multi-source error messages. The pipeline is data-driven, based on machine learning algorithms, and executed fully automatically, allowing categorizing error messages according to textual patterns and meaning.

Download Full-text