scholarly journals FLAGS: A methodology for adaptive anomaly detection and root cause analysis on sensor data streams by fusing expert knowledge with machine learning

2021 ◽  
Vol 116 ◽  
pp. 30-48
Author(s):  
Bram Steenwinckel ◽  
Dieter De Paepe ◽  
Sander Vanden Hautte ◽  
Pieter Heyvaert ◽  
Mohamed Bentefrit ◽  
...  
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Data Streams ◽  
Expert Knowledge ◽  
Root Cause Analysis ◽  
Sensor Data ◽  
Cause Analysis ◽  
Root Cause

scholarly journals Machine Learning Based Anomaly Detection as an Emerging Trend in Telecommunications

2020 ◽  
Author(s):  
Valentina Đorđević ◽  
Pavle Miloševic ◽  
Ana Poledica
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Cellular Network ◽  
Root Cause Analysis ◽  
Anomalous Behaviour ◽  
Lte Networks ◽  
Cause Analysis ◽  
Domain Experts ◽  
Root Cause ◽  
Drill Down

Research Question: This paper investigates into how machine learning can be applied for the purpose of detecting anomalies in the data describing transport component within the cellular network. Motivation: In the field of telecommunications, terabytes of data are generated each hour. This makes the manual analysis almost impossible to perform. There are thousands of components whose behaviour needs to be monitored, since anomalous behaviour could indicate a possible failure that can lead to network degradation, huge maintenance costs, and finally – a bad user experience. Our goal is to try to catch anomalous behaviour automatically, and thus help domain experts when performing drill down analysis of the degraded base stations and their key performance indicators (KPIs). Idea: The main idea of this paper is to empirically evaluate the application of machine learning for the problem of anomaly detection, in the field of telecommunications, specifically to long term evolution (LTE) networks. Data: Data used in the analysis contains information about base transceiver stations (BTS) behaviour through the time. The data are gathered from a cellular network provider located in Serbia. The data are collected on an hourly basis, for a period of two weeks, resulting in almost 700 thousand rows. The behaviour is assessed by 96 transport KPIs coming from BTS, describing the package losses, delays, transmission success rates, etc. Tools: Two main algorithms, ensemble-based Isolation Forest and autoencoder neural network, are elaborated and applied in order to identify patterns of anomalous behaviour. Findings: The results show that machine learning can be successfully applied in the field of LTE networks for the problem of anomaly detection. Machine learning can significantly reduce the time needed for the domain experts to identify anomalies within the network. In addition to time efficiency, one of the algorithms tested is able to identify anomalous KPIs separately, which is crucial when performing root cause analysis, by using drill-down approach, in order to identify which component is degraded. Contribution: This paper enriches existing research related to anomaly detection in LTE networks and provides an innovative approach to automated root-cause analysis of network degradation.

scholarly journals AUTOMATED ROOT CAUSE ANALYSIS OF NON-CONFORMITIES WITH MACHINE LEARNING ALGORITHMS

2018 ◽  
Vol 18 (4) ◽  
pp. 60-72 ◽  
Author(s):  
Tobias MUELLER ◽  
Jonathan GREIPEL ◽  
Tobias WEBER ◽  
Robert H. SCHMITT
Keyword(s):  
Machine Learning ◽  
Expert Knowledge ◽  
Learning Algorithms ◽  
Root Cause Analysis ◽  
Machine Learning Algorithms ◽  
Decision Tree Algorithm ◽  
Process Data ◽  
Cause Analysis ◽  
Root Cause ◽  
High Level

To detect root causes of non-conforming parts - parts outside the tolerance limits - in production processes a high level of expert knowledge is necessary. This results in high costs and a low flexibility in the choice of personnel to perform analyses. In modern production a vast amount of process data is available and machine learning algorithms exist which model processes empirically. Aim of this paper is to introduce a procedure for an automated root cause analysis based on machine learning algorithms to reduce the costs and the necessary expert knowledge. Therefore, a decision tree algorithm is chosen. A procedure for its application in an automated root cause analysis is presented and simulations to prove its applicability are conducted. In this paper influences affecting the success of detection are identified and simulated e.g. the necessary amount of data dependent on the amount of variables, the ratio between categories of non-conformities and OK parts as well as detectable root causes. The simulations are based on a regression model to determine the roughness of drilling holes. They prove the applicability of machine learning algorithms for an automated root cause analysis and indicate which influences have to be considered in real scenarios.

scholarly journals Assembly Line Anomaly Detection and Root Cause Analysis Using Machine Learning

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 189661-189672
Author(s):  
Osama Abdelrahman ◽  
Pantea Keikhosrokiani
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Assembly Line ◽  
Root Cause Analysis ◽  
Cause Analysis ◽  
Root Cause

scholarly journals Lifelong Machine Learning and root cause analysis for large-scale cancer patient data

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Gautam Pal ◽  
Xianbin Hong ◽  
Zhuo Wang ◽  
Hongyi Wu ◽  
Gangmin Li ◽  
...  
Keyword(s):  
Machine Learning ◽  
Lifelong Learning ◽  
Root Cause Analysis ◽  
Cause Analysis ◽  
Training Time ◽  
Root Cause ◽  
Random Decision Forest ◽  
Batch Data ◽  
Over Time ◽  
Decision Forest

Abstract Introduction This paper presents a lifelong learning framework which constantly adapts with changing data patterns over time through incremental learning approach. In many big data systems, iterative re-training high dimensional data from scratch is computationally infeasible since constant data stream ingestion on top of a historical data pool increases the training time exponentially. Therefore, the need arises on how to retain past learning and fast update the model incrementally based on the new data. Also, the current machine learning approaches do the model prediction without providing a comprehensive root cause analysis. To resolve these limitations, our framework lays foundations on an ensemble process between stream data with historical batch data for an incremental lifelong learning (LML) model. Case description A cancer patient’s pathological tests like blood, DNA, urine or tissue analysis provide a unique signature based on the DNA combinations. Our analysis allows personalized and targeted medications and achieves a therapeutic response. Model is evaluated through data from The National Cancer Institute’s Genomic Data Commons unified data repository. The aim is to prescribe personalized medicine based on the thousands of genotype and phenotype parameters for each patient. Discussion and evaluation The model uses a dimension reduction method to reduce training time at an online sliding window setting. We identify the Gleason score as a determining factor for cancer possibility and substantiate our claim through Lilliefors and Kolmogorov–Smirnov test. We present clustering and Random Decision Forest results. The model’s prediction accuracy is compared with standard machine learning algorithms for numeric and categorical fields. Conclusion We propose an ensemble framework of stream and batch data for incremental lifelong learning. The framework successively applies first streaming clustering technique and then Random Decision Forest Regressor/Classifier to isolate anomalous patient data and provides reasoning through root cause analysis by feature correlations with an aim to improve the overall survival rate. While the stream clustering technique creates groups of patient profiles, RDF further drills down into each group for comparison and reasoning for useful actionable insights. The proposed MALA architecture retains the past learned knowledge and transfer to future learning and iteratively becomes more knowledgeable over time.

Sign in / Sign up

Export Citation Format

Share Document