Scalable Parallel I/O on a Blue Gene/Q Supercomputer Using Compression, Topology-Aware Data Aggregation, and Subfiling

2014 ◽  
Author(s):  
Huy Bui ◽  
Hal Finkel ◽  
Venkatram Vishwanath ◽  
Salma Habib ◽  
Katrin Heitmann ◽  
...  
Keyword(s):  
Data Aggregation ◽  
Blue Gene

Anomaly Detection via Mining Numerical Workflow Relations from Logs

2020 ◽  
Author(s):  
Bo Zhang ◽  
Hongyu Zhang ◽  
Pablo Moscato
Keyword(s):  
Distributed Systems ◽  
Anomaly Detection ◽  
Text Messages ◽  
Distributed File System ◽  
Log Data ◽  
Sliding Windows ◽  
Novel Approach ◽  
Hadoop Distributed File System ◽  
Blue Gene ◽  
Online Anomaly Detection

<div>Complex software intensive systems, especially distributed systems, generate logs for troubleshooting. The logs are text messages recording system events, which can help engineers determine the system's runtime status. This paper proposes a novel approach named ADR (stands for Anomaly Detection by workflow Relations) that employs matrix nullspace to mine numerical relations from log data. The mined relations can be used for both offline and online anomaly detection and facilitate fault diagnosis. We have evaluated ADR on log data collected from two distributed systems, HDFS (Hadoop Distributed File System) and BGL (IBM Blue Gene/L supercomputers system). ADR successfully mined 87 and 669 numerical relations from the logs and used them to detect anomalies with high precision and recall. For online anomaly detection, ADR employs PSO (Particle Swarm Optimization) to find the optimal sliding windows' size and achieves fast anomaly detection.</div><div>The experimental results confirm that ADR is effective for both offline and online anomaly detection. </div>

An Overview of Data Aggregation Architecture for Real-Time Tracking with Sensor Networks

2006 ◽  
Author(s):  
Tian He ◽  
Lin Gu ◽  
Liqian Luo ◽  
Ting Yan ◽  
John A. Stankovic ◽  
...  
Keyword(s):  
Sensor Networks ◽  
Real Time ◽  
Data Aggregation ◽  
Real Time Tracking

Corruption Measurements: Caught Between Conceptualizing the Phenomenon and Promoting New Governance Agenda?

2021 ◽  
pp. 097226292098394
Author(s):  
Kannan Perumal
Keyword(s):  
Data Aggregation ◽  
Policy Advocacy ◽  
New Governance ◽  
Methodological Issues ◽  
Topic Evolution ◽  
Theoretical Perspectives ◽  
Control Framework ◽  
The Subject ◽  
Theoretical Issues ◽  
Insight Into

The work ‘Corruption Measurements: Caught Between Conceptualizing the Phenomenon and Promoting New Governance Agenda?’ is a qualitative study based on reviewing the literature available on the subject. It starts with the introduction that explains the evolution of the idea of measuring corruption, its relevance to governance and associated theoretical issues. The topic, ‘Evolution of Corruption Measurements’ gives an overview about different corruption indices. While the topic ‘Challenges to Corruption Measurements’ briefly introduces the challenges faced by corruption measurements, the topics ‘Conceptualizing Corruption’ and ‘Methodological Issues’ give insight into the contentions faced by corruption measurements from different theoretical perspectives. Also, explained in these sections are how the corruption measurements have conceptualized corruption over the period of three decades; and how do they keep evolving their methods in order to become more relevant in policy advocacy. Issues associated with data aggregation also are explained in-depth in this work. This work demonstrates that though continuous methodological evolution and empirical research have helped corruption measurements to improve their acceptance level, the gap that exist between corruption control framework and practice will remain a challenge to address in future if corruption measurements do not genuinely account the contextual realities.

Sign in / Sign up

Export Citation Format

Share Document