scholarly journals Learning Representations of Network Traffic Using Deep Neural Networks for Network Anomaly Detection: A Perspective towards Oil and Gas IT Infrastructures

Symmetry ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 1882
Author(s):  
Sheraz Naseer ◽  
Rao Faizan Ali ◽  
P.D.D Dominic ◽  
Yasir Saleem
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
Oil And Gas ◽  
Resource Planning ◽  
Machine Learning Algorithms ◽  
Data Driven ◽  
Network Data ◽  
It Infrastructure ◽  
Industrial Automation ◽  
Network Anomaly Detection

Oil and Gas organizations are dependent on their IT infrastructure, which is a small part of their industrial automation infrastructure, to function effectively. The oil and gas (O&G) organizations industrial automation infrastructure landscape is complex. To perform focused and effective studies, Industrial systems infrastructure is divided into functional levels by The Instrumentation, Systems and Automation Society (ISA) Standard ANSI/ISA-95:2005. This research focuses on the ISA-95:2005 level-4 IT infrastructure to address network anomaly detection problem for ensuring the security and reliability of Oil and Gas resource planning, process planning and operations management. Anomaly detectors try to recognize patterns of anomalous behaviors from network traffic and their performance is heavily dependent on extraction time and quality of network traffic features or representations used to train the detector. Creating efficient representations from large volumes of network traffic to develop anomaly detection models is a time and resource intensive task. In this study we propose, implement and evaluate use of Deep learning to learn effective Network data representations from raw network traffic to develop data driven anomaly detection systems. Proposed methodology provides an automated and cost effective replacement of feature extraction which is otherwise a time and resource intensive task for developing data driven anomaly detectors. The ISCX-2012 dataset is used to represent ISA-95 level-4 network traffic because the O&G network traffic at this level is not much different than normal internet traffic. We trained four representation learning models using popular deep neural network architectures to extract deep representations from ISCX 2012 traffic flows. A total of sixty anomaly detectors were trained by authors using twelve conventional Machine Learning algorithms to compare the performance of aforementioned deep representations with that of a human-engineered handcrafted network data representation. The comparisons were performed using well known model evaluation parameters. Results showed that deep representations are a promising feature in engineering replacement to develop anomaly detection models for IT infrastructure security. In our future research, we intend to investigate the effectiveness of deep representations, extracted using ISA-95:2005 Level 2-3 traffic comprising of SCADA systems, for anomaly detection in critical O&G systems.

A Hybrid Technique Using PCA and Wavelets in Network Traffic Anomaly Detection

2014 ◽  
Vol 6 (1) ◽  
pp. 17-53 ◽  
Author(s):  
Stevan Novakov ◽  
Chung-Horng Lung ◽  
Ioannis Lambadaris ◽  
Nabil Seddigh
Keyword(s):  
Spectral Analysis ◽  
Statistical Analysis ◽  
Wavelet Analysis ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Hybrid Approach ◽  
Haar Wavelet ◽  
Wavelet Filtering ◽  
Analysis Technique ◽  
Network Anomaly Detection

Research into network anomaly detection has become crucial as a result of a significant increase in the number of computer attacks. Many approaches in network anomaly detection have been reported in the literature, but data or solutions typically are not freely available. Recently, a labeled network traffic flow dataset, Kyoto2006+, has been created and is publicly available. Most existing approaches using Kyoto2006+ for network anomaly detection apply various clustering techniques. This paper leverages existing well known statistical analysis and spectral analysis techniques for network anomaly detection. The first popular approach is a statistical analysis technique called Principal Component Analysis (PCA). PCA describes data in a new dimension to unlock otherwise hidden characteristics. The other well known spectral analysis technique is Haar Wavelet filtering analysis. It measures the amount and magnitude of abrupt changes in data. Both approaches have strengths and limitations. In response, this paper proposes a Hybrid PCA–Haar Wavelet Analysis. The hybrid approach first applies PCA to describe the data and then Haar Wavelet filtering for analysis. Based on prototyping and measurement, an investigation of the Hybrid PCA–Haar Wavelet Analysis technique is performed using the Kyoto2006+ dataset. The authors consider a number of parameters and present experimental results to demonstrate the effectiveness of the hybrid approach as compared to the two algorithms individually.

Network anomaly detection based on signal processing techniques

2013 ◽  
Vol 18 (1) ◽  
pp. 15-21
Author(s):  
Tomasz Andrysiak ◽  
Łukasz Saganowski ◽  
Mirosław Maszewski
Keyword(s):  
Signal Processing ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Detection Method ◽  
Matching Pursuit ◽  
Matching Pursuit Decomposition ◽  
Network Anomaly Detection ◽  
Processing Techniques ◽  
Signal Processing Techniques

Abstract The article depicts possibility of using Matching Pursuit decomposition in order to recognize unspecified hazards in network traffic. Furthermore, the work aims to present feasible enhancements to the anomaly detection method, as well as their efficiency on the basis of a wide collection of pattern test traces.

scholarly journals Visualization of Anomalies using Graph-Based Anomaly Detection

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Ramesh Paudel ◽  
Lauren Tharp ◽  
Dulce Kaiser ◽  
William Eberle ◽  
Gerald Gannod
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
False Positives ◽  
Visual Context ◽  
Traffic Flows ◽  
Security Analysts ◽  
Fast Analysis ◽  
Detection Systems ◽  
Potential Damage ◽  
Network Anomaly Detection

Network protocol analyzers such asWireshark are valuable for analyzing network traffic but pose a challenge in that it can be difficult to determine which behaviors are out of the ordinary due to the volume of data that must be analyzed. Network anomaly detection systems can provide vital insights to security analysts to supplement protocol analyzers, but this feedback can be difficult to interpret due to the complexity of the algorithms used and the lack of context to determine the reasoning for which an event was labeled as anomalous. We present an approach for visualizing anomalies using a graph-based anomaly detection methodology that aims to provide visual context to network traffic. We demonstrate the approach using network traffic flows as an approach for aiding in the investigation and triage of anomalous network events. The simplicity of a visual representation supports fast analysis of anomalous traffic to identify true positives from false positives and prevent further potential damage.

scholarly journals Network Anomaly Detection by Using a Time-Decay Closed Frequent Pattern

Information ◽  
2019 ◽  
Vol 10 (8) ◽  
pp. 262
Author(s):  
Ying Zhao ◽  
Junjun Chen ◽  
Di Wu ◽  
Jian Teng ◽  
Nabin Sharma ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
User Behavior ◽  
Frequent Pattern ◽  
Detection Methods ◽  
Frequent Patterns ◽  
Time Decay ◽  
Network Behavior ◽  
Detection Model ◽  
Network Anomaly Detection

Anomaly detection of network traffic flows is a non-trivial problem in the field of network security due to the complexity of network traffic. However, most machine learning-based detection methods focus on network anomaly detection but ignore the user anomaly behavior detection. In real scenarios, the anomaly network behavior may harm the user interests. In this paper, we propose an anomaly detection model based on time-decay closed frequent patterns to address this problem. The model mines closed frequent patterns from the network traffic of each user and uses a time-decay factor to distinguish the weight of current and historical network traffic. Because of the dynamic nature of user network behavior, a detection model update strategy is provided in the anomaly detection framework. Additionally, the closed frequent patterns can provide interpretable explanations for anomalies. Experimental results show that the proposed method can detect user behavior anomaly, and the network anomaly detection performance achieved by the proposed method is similar to the state-of-the-art methods and significantly better than the baseline methods.

scholarly journals Research Risk Factors in Monitoring Well Drilling—A Case Study Using Machine Learning Methods

Symmetry ◽  
2021 ◽  
Vol 13 (7) ◽  
pp. 1293
Author(s):  
Shamil Islamov ◽  
Alexey Grigoriev ◽  
Ilia Beloglazov ◽  
Sergey Savchenkov ◽  
Ove Tobias Gudmestad
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Oil And Gas ◽  
New Technologies ◽  
Learning Algorithms ◽  
Gas Production ◽  
Machine Learning Algorithms ◽  
Gradient Boosting ◽  
Drilling Process ◽  
Well Drilling

This article takes an approach to creating a machine learning model for the oil and gas industry. This task is dedicated to the most up-to-date issues of machine learning and artificial intelligence. One of the goals of this research was to build a model to predict the possible risks arising in the process of drilling wells. Drilling of wells for oil and gas production is a highly complex and expensive part of reservoir development. Thus, together with injury prevention, there is a goal to save cost expenditures on downtime and repair of drilling equipment. Nowadays, companies have begun to look for ways to improve the efficiency of drilling and minimize non-production time with the help of new technologies. To support decisions in a narrow time frame, it is valuable to have an early warning system. Such a decision support system will help an engineer to intervene in the drilling process and prevent high expenses of unproductive time and equipment repair due to a problem. This work describes a comparison of machine learning algorithms for anomaly detection during well drilling. In particular, machine learning algorithms will make it possible to make decisions when determining the geometry of the grid of wells—the nature of the relative position of production and injection wells at the production facility. Development systems are most often subdivided into the following: placement of wells along a symmetric grid, and placement of wells along a non-symmetric grid (mainly in rows). The tested models classify drilling problems based on historical data from previously drilled wells. To validate anomaly detection algorithms, we used historical logs of drilling problems for 67 wells at a large brownfield in Siberia, Russia. Wells with problems were selected and analyzed. It should be noted that out of the 67 wells, 20 wells were drilled without expenses for unproductive time. The experiential results illustrate that a model based on gradient boosting can classify the complications in the drilling process better than other models.

scholarly journals Model fusion of deep neural networks for anomaly detection

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Nouar AlDahoul ◽  
Hezerul Abdul Karim ◽  
Abdulaziz Saleh Ba Wazir
Keyword(s):  
Neural Networks ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Large Scale ◽  
Deep Neural Networks ◽  
Denial Of Service ◽  
Traffic Data ◽  
Model Fusion ◽  
Class Weight ◽  
Network Anomaly Detection

AbstractNetwork Anomaly Detection is still an open challenging task that aims to detect anomalous network traffic for security purposes. Usually, the network traffic data are large-scale and imbalanced. Additionally, they have noisy labels. This paper addresses the previous challenges and utilizes million-scale and highly imbalanced ZYELL’s dataset. We propose to train deep neural networks with class weight optimization to learn complex patterns from rare anomalies observed from the traffic data. This paper proposes a novel model fusion that combines two deep neural networks including binary normal/attack classifier and multi-attacks classifier. The proposed solution can detect various network attacks such as Distributed Denial of Service (DDOS), IP probing, PORT probing, and Network Mapper (NMAP) probing. The experiments conducted on a ZYELL’s real-world dataset show promising performance. It was found that the proposed approach outperformed the baseline model in terms of average macro Fβ score and false alarm rate by 17% and 5.3%, respectively.

scholarly journals Analysis of deep learning models for network anomaly detection in Internet of Things

2021 ◽  
pp. 28-37
Author(s):  
Diana Gaifilina ◽  
Igor Kotenko
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Internet Of Things ◽  
Anomaly Detection ◽  
Recurrent Neural Network ◽  
Network Traffic ◽  
Learning Models ◽  
Network Anomaly Detection ◽  
Machine Learning Models

Introduction: The article discusses the problem of choosing deep learning models for detecting anomalies in Internet of Things (IoT) network traffic. This problem is associated with the necessity to analyze a large number of security events in order to identify the abnormal behavior of smart devices. A powerful technology for analyzing such data is machine learning and, in particular, deep learning. Purpose: Development of recommendations for the selection of deep learning models for anomaly detection in IoT network traffic. Results: The main results of the research are comparative analysis of deep learning models, and recommendations on the use of deep learning models for anomaly detection in IoT network traffic. Multilayer perceptron, convolutional neural network, recurrent neural network, long short-term memory, gated recurrent units, and combined convolutional-recurrent neural network were considered the basic deep learning models. Additionally, the authors analyzed the following traditional machine learning models: naive Bayesian classifier, support vector machines, logistic regression, k-nearest neighbors, boosting, and random forest. The following metrics were used as indicators of anomaly detection efficiency: accuracy, precision, recall, and F-measure, as well as the time spent on training the model. The constructed models demonstrated a higher accuracy rate for anomaly detection in large heterogeneous traffic typical for IoT, as compared to conventional machine learning methods. The authors found that with an increase in the number of neural network layers, the completeness of detecting anomalous connections rises. This has a positive effect on the recognition of unknown anomalies, but increases the number of false positives. In some cases, preparing traditional machine learning models takes less time. This is due to the fact that the application of deep learning methods requires more resources and computing power. Practical relevance: The results obtained can be used to build systems for network anomaly detection in Internet of Things traffic.

scholarly journals Risk Factors Evaluation for Monitoring of Well Drilling

2021 ◽  
Author(s):  
Shamil Islamov ◽  
Alexey Grigoriev ◽  
Ilya Beloglazov ◽  
Sergey Savchenkov ◽  
Ove Tobias Gudmestad
Keyword(s):  
Anomaly Detection ◽  
Oil And Gas ◽  
New Technologies ◽  
Warning System ◽  
Time Frame ◽  
Gas Production ◽  
Machine Learning Algorithms ◽  
Gradient Boosting ◽  
Drilling Process ◽  
Well Drilling

Drilling of wells for oil and gas production is a highly complex and expensive part of reservoir development. Thus, together with injury prevention, there is a goal to save cost expenditures on downtime and repair of drilling equipment. Nowadays companies have begun to look for ways to improve the efficiency of drilling and minimize non-production time with the help of new technologies. To support decisions in a narrow time frame, it is valuable to have an early warning system. Such a decision support system will help an engineer to intervene in the drilling process and prevent high expenses of unproductive time and equipment repair due to a problem. This work is describing a comparison of machine learning algorithms for anomaly detection during well drilling. Tested models classify drilling problems based on historical data from previously drilled wells. To validate anomaly detection algorithms, we use historical logs of drilling problems for 67 wells at a large brownfield in Siberia, Russia. Wells with problems were selected and analyzed. It should be noted that out of the 67 wells, 20 wells were drilled without expenses for unproductive time. Experiential results illustrated that a model based on gradient boosting can classify the complications in the drilling process best of all.

scholarly journals A Deep Learning Ensemble for Network Anomaly and Cyber-Attack Detection

Sensors ◽  
2020 ◽  
Vol 20 (16) ◽  
pp. 4583 ◽  
Author(s):  
Vibekananda Dutta ◽  
Michał Choraś ◽  
Marek Pawlicki ◽  
Rafał Kozik
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Anomaly Detection ◽  
Statistical Significance ◽  
Attack Detection ◽  
Machine Learning Algorithms ◽  
Machine Learning Techniques ◽  
Second Phase ◽  
Network Intrusion ◽  
Network Anomaly Detection

Currently, expert systems and applied machine learning algorithms are widely used to automate network intrusion detection. In critical infrastructure applications of communication technologies, the interaction among various industrial control systems and the Internet environment intrinsic to the IoT technology makes them susceptible to cyber-attacks. Given the existence of the enormous network traffic in critical Cyber-Physical Systems (CPSs), traditional methods of machine learning implemented in network anomaly detection are inefficient. Therefore, recently developed machine learning techniques, with the emphasis on deep learning, are finding their successful implementations in the detection and classification of anomalies at both the network and host levels. This paper presents an ensemble method that leverages deep models such as the Deep Neural Network (DNN) and Long Short-Term Memory (LSTM) and a meta-classifier (i.e., logistic regression) following the principle of stacked generalization. To enhance the capabilities of the proposed approach, the method utilizes a two-step process for the apprehension of network anomalies. In the first stage, data pre-processing, a Deep Sparse AutoEncoder (DSAE) is employed for the feature engineering problem. In the second phase, a stacking ensemble learning approach is utilized for classification. The efficiency of the method disclosed in this work is tested on heterogeneous datasets, including data gathered in the IoT environment, namely IoT-23, LITNET-2020, and NetML-2020. The results of the evaluation of the proposed approach are discussed. Statistical significance is tested and compared to the state-of-the-art approaches in network anomaly detection.

Sign in / Sign up

Export Citation Format

Share Document