Anomaly detection driven active learning for identifying suspicious tracks and events in WAMI video

2012 ◽  
Author(s):  
David J. Miller ◽  
Aditya Natraj ◽  
Ryler Hockenbury ◽  
Katherine Dunn ◽  
Michael Sheffler ◽  
...  
Keyword(s):  
Active Learning ◽  
Anomaly Detection

scholarly journals SALAD: An Exploration of Split Active Learning based Unsupervised Network Data Stream Anomaly Detection using Autoencoders

2021 ◽  
Author(s):  
Christopher Nixon ◽  
Mohamed Sedky ◽  
Mohamed Hassan
Keyword(s):  
Machine Learning ◽  
Active Learning ◽  
Intrusion Detection ◽  
Anomaly Detection ◽  
Data Streams ◽  
Data Stream ◽  
Learning Strategy ◽  
Network Data ◽  
Anomaly Detector ◽  
Active Learning Strategy

<div>Machine learning based intrusion detection systems monitor network data streams for cyber attacks. Challenges in this space include detection of unknown attacks, adaptation to changes in the data stream such as changes in underlying behaviour, the human cost of labeling data to retrain the machine learning model and the processing and memory constraints of a real-time data stream. Failure to manage the aforementioned factors could result in missed attacks, degraded detection performance, unnecessary expense or delayed detection times. This research evaluated autoencoders, a type of feed-forward neural network, as online anomaly detectors for network data streams. The autoencoder method was combined with an active learning strategy to further reduce labeling cost and speed up training and adaptation times, resulting in a proposed Split Active Learning Anomaly Detector (SALAD) method. The proposed method was evaluated with the NSL-KDD, KDD Cup 1999, and UNSW-NB15 data sets, using the scikit-multiflow framework. Results demonstrated that a novel Adaptive Anomaly Threshold method, combined with a split active learning strategy offered superior anomaly detection performance with a labeling budget of just 20%, significantly reducing the required human expertise to annotate the network data. Processing times of the autoencoder anomaly detector method were demonstrated to be significantly lower than traditional online learning methods, allowing for greatly improved responsiveness to attacks occurring in real time. Future research areas are applying unsupervised threshold methods, multi-label classification, sample annotation, and hybrid intrusion detection.</div>

scholarly journals Class Prior Estimation in Active Positive and Unlabeled Learning

2020 ◽  
Author(s):  
Lorenzo Perini ◽  
Vincent Vercruyssen ◽  
Jesse Davis
Keyword(s):  
Active Learning ◽  
Anomaly Detection ◽  
Learning Strategy ◽  
Important Task ◽  
Positive And Unlabeled Learning ◽  
Active Learning Strategy

Estimating the proportion of positive examples (i.e., the class prior) from positive and unlabeled (PU) data is an important task that facilitates learning a classifier from such data. In this paper, we explore how to tackle this problem when the observed labels were acquired via active learning. This introduces the challenge that the observed labels were not selected completely at random, which is the primary assumption underpinning existing approaches to estimating the class prior from PU data. We analyze this new setting and design an algorithm that is able to estimate the class prior for a given active learning strategy. Empirically, we show that our approach accurately recovers the true class prior on a benchmark of anomaly detection datasets and that it does so more accurately than existing methods.

scholarly journals Active learning for anomaly detection in environmental data

2020 ◽  
Vol 134 ◽  
pp. 104869
Author(s):  
Stefania Russo ◽  
Moritz Lürig ◽  
Wenjin Hao ◽  
Blake Matthews ◽  
Kris Villez
Keyword(s):  
Active Learning ◽  
Anomaly Detection ◽  
Environmental Data

scholarly journals Network Anomaly Detection for NSL-KDD Dataset Using Deep Learning

2021 ◽  
Vol 9 (2) ◽  
pp. 821-827
Author(s):  
Kavitha S, Dr. Uma Maheswari N, Dr.R.Venkatesh
Keyword(s):  
Deep Learning ◽  
Active Learning ◽  
Intrusion Detection ◽  
Anomaly Detection ◽  
Intelligent System ◽  
Essential Element ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Learning Method ◽  
Active Learning Method

Deep learning based intrusion detection cyber security methods gained increased popularity. The essential element to provide protection to the ICT infrastructure is the intrusion detection systems (IDSs). Intelligent solutions are necessary to control the complexity and increase in the new attack types. The intelligent system (DL/ML) has been widely used with its benefits to effectively deal with complex and great dimensional data. The IDS has various attack types like known, unknown, zero day attacks are attractive to and detected using unsupervised machine learning techniques. A novel methodology has been proposed that combines the benefits of Isolation forest (One Class) Support Vector Machine (OCSVM) with active learning method to detect threats without any prior knowledge. The NSL-KDD dataset has been used to evaluate the various DL methods with active learning method. The results show that this method performs better than other techniques. The design methodology inspires the efforts to emerging anomaly detection.

Sign in / Sign up

Export Citation Format

Share Document