scholarly journals An Ensemble-Based Approach to Anomaly Detection in Marine Engine Sensor Streams for Efficient Condition Monitoring and Analysis

Sensors ◽  
2020 ◽  
Vol 20 (24) ◽  
pp. 7285
Author(s):  
Donghyun Kim ◽  
Sangbong Lee ◽  
Jihwan Lee
Keyword(s):  
Diesel Engine ◽  
Anomaly Detection ◽  
Large Scale ◽  
Abnormal Behavior ◽  
High Dimensional ◽  
Statistical Control ◽  
Marine Engine ◽  
Large Scale Dataset ◽  
Unsupervised Anomaly Detection ◽  
Sensor Streams

This study proposes an unsupervised anomaly detection method using sensor streams from the marine engine to detect the anomalous system behavior, which may be a possible sign of system failure. Previous works on marine engine anomaly detection proposed a clustering-based or statistical control chart-based approach that is unstable according to the choice of hyperparameters, or cannot fit well to the high-dimensional dataset. As a remedy to this limitation, this study adopts an ensemble-based approach to anomaly detection. The idea is to train several anomaly detectors with varying hyperparameters in parallel and then combine its result in the anomaly detection phase. Because the anomaly is detected by the combination of different detectors, it is robust to the choice of hyperparameters without loss of accuracy. To demonstrate our methodology, an actual dataset obtained from a 200,000-ton cargo vessel from a Korean shipping company that uses two-stroke diesel engine is analyzed. As a result, anomalies were successfully detected from the high-dimensional and large-scale dataset. After detecting the anomaly, clustering analysis was conducted to the anomalous observation to examine anomaly patterns. By investigating each cluster’s feature distribution, several common patterns of abnormal behavior were successfully visualized. Although we analyzed the data from two-stroke diesel engine, our method can be applied to various types of marine engine.

Generative and Autoencoder Models for Large-Scale Mutivariate Unsupervised Anomaly Detection

2021 ◽  
pp. 45-58
Author(s):  
Nabila Ounasser ◽  
Maryem Rhanoui ◽  
Mounia Mikram ◽  
Bouchra El Asri
Keyword(s):  
Anomaly Detection ◽  
Large Scale ◽  
Unsupervised Anomaly Detection

High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning

2016 ◽  
Vol 58 ◽  
pp. 121-134 ◽  
Author(s):  
Sarah M. Erfani ◽  
Sutharshan Rajasegarar ◽  
Shanika Karunasekera ◽  
Christopher Leckie
Keyword(s):  
Deep Learning ◽  
Anomaly Detection ◽  
Large Scale ◽  
High Dimensional

scholarly journals A Hybrid Deep Learning-Based Unsupervised Anomaly Detection in High Dimensional Data

2022 ◽  
Vol 70 (3) ◽  
pp. 5363-5381
Author(s):  
Amgad Muneer ◽  
Shakirah Mohd Taib ◽  
Suliman Mohamed Fati ◽  
Abdullateef O. Balogun ◽  
Izzatdin Abdul Aziz
Keyword(s):  
Deep Learning ◽  
Anomaly Detection ◽  
High Dimensional Data ◽  
High Dimensional ◽  
Unsupervised Anomaly Detection

scholarly journals Unsupervised Anomaly Detection Based on Deep Autoencoding and Clustering

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Chuanlei Zhang ◽  
Jiangtao Liu ◽  
Wei Chen ◽  
Jinyuan Shi ◽  
Minda Yao ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Dimension Reduction ◽  
Density Estimation ◽  
Original Data ◽  
Industrial Applications ◽  
Multidimensional Data ◽  
High Dimensional ◽  
Clustering Methods ◽  
Detection Scheme ◽  
Unsupervised Anomaly Detection

The unsupervised anomaly detection task based on high-dimensional or multidimensional data occupies a very important position in the field of machine learning and industrial applications; especially in the aspect of network security, the anomaly detection of network data is particularly important. The key to anomaly detection is density estimation. Although the methods of dimension reduction and density estimation have made great progress in recent years, most dimension reduction methods are difficult to retain the key information of original data or multidimensional data. Recent studies have shown that the deep autoencoder (DAE) can solve this problem well. In order to improve the performance of unsupervised anomaly detection, we propose an anomaly detection scheme based on a deep autoencoder (DAE) and clustering methods. The deep autoencoder is trained to learn the compressed representation of the input data and then feed it to clustering approach. This scheme makes full use of the advantages of the deep autoencoder (DAE) to generate low-dimensional representation and reconstruction errors for the input high-dimensional or multidimensional data and uses them to reconstruct the input samples. The proposed scheme could eliminate redundant information contained in the data, improve performance of clustering methods in identifying abnormal samples, and reduce the amount of calculation. To verify the effectiveness of the proposed scheme, massive comparison experiments have been conducted with traditional dimension reduction algorithms and clustering methods. The results of experiments demonstrate that, in most cases, the proposed scheme outperforms the traditional dimension reduction algorithms with different clustering methods.

scholarly journals Unsupervised Anomaly Detection for Network Data Streams in Industrial Control Systems

Information ◽  
2020 ◽  
Vol 11 (2) ◽  
pp. 105 ◽  
Author(s):  
Limengwei Liu ◽  
Modi Hu ◽  
Chaoqun Kang ◽  
Xiaoyong Li
Keyword(s):  
Anomaly Detection ◽  
Control Systems ◽  
Data Streams ◽  
Large Scale ◽  
Stable Operation ◽  
Time Data ◽  
Industrial Control ◽  
Industrial Control Systems ◽  
Unsupervised Anomaly Detection ◽  
Industrial Control Networks

The development and integration of information technology and industrial control networks have expanded the magnitude of new data; detecting anomalies or discovering other valid information from them is of vital importance to the stable operation of industrial control systems. This paper proposes an incremental unsupervised anomaly detection method that can quickly analyze and process large-scale real-time data. Our evaluation on the Secure Water Treatment dataset shows that the method is converging to its offline counterpart for infinitely growing data streams.

scholarly journals Effect Improved for High-Dimensional and Unbalanced Data Anomaly Detection Model Based on KNN-SMOTE-LSTM

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-17
Author(s):  
Fuguang Bao ◽  
Yongqiang Wu ◽  
Zhaogang Li ◽  
Yongzhao Li ◽  
Lili Liu ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Large Scale ◽  
Short Term Memory ◽  
Sequence Data ◽  
Binary Classification ◽  
High Dimensional ◽  
Unbalanced Data ◽  
K Nearest Neighbors ◽  
Detection Model ◽  
Model Based

High-dimensional and unbalanced data anomaly detection is common. Effective anomaly detection is essential for problem or disaster early warning and maintaining system reliability. A significant research issue related to the data analysis of the sensor is the detection of anomalies. The anomaly detection is essentially an unbalanced sequence binary classification. The data of this type contains characteristics of large scale, high complex computation, unbalanced data distribution, and sequence relationship among data. This paper uses long short-term memory networks (LSTMs) combined with historical sequence data; also, it integrates the synthetic minority oversampling technique (SMOTE) algorithm and K-nearest neighbors (kNN), and it designs and constructs an anomaly detection network model based on kNN-SMOTE-LSTM in accordance with the data characteristic of being unbalanced. This model can continuously filter out and securely generate samples to improve the performance of the model through kNN discriminant classifier and avoid the blindness and limitations of the SMOTE algorithm in generating new samples. The experiments demonstrated that the structured kNN-SMOTE-LSTM model can significantly improve the performance of the unbalanced sequence binary classification.

Sign in / Sign up

Export Citation Format

Share Document