Anomaly Detection in Market Data Structures Via Machine Learning Algorithms

2020 ◽  
Author(s):  
Dirk Röder ◽  
Henning Mueller
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Data Structures ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Market Data

scholarly journals Adaptive Anomaly Detection Framework Model Objects in Cyberspace

2020 ◽  
Vol 2020 ◽  
pp. 1-14
Author(s):  
Hasan Alkahtani ◽  
Theyazn H. H. Aldhyani ◽  
Mohammed Al-Yaari
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Network Security ◽  
Anomaly Detection ◽  
Denial Of Service ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Ddos Attacks ◽  
Framework Model

Telecommunication has registered strong and rapid growth in the past decade. Accordingly, the monitoring of computers and networks is too complicated for network administrators. Hence, network security represents one of the biggest serious challenges that can be faced by network security communities. Taking into consideration the fact that e-banking, e-commerce, and business data will be shared on the computer network, these data may face a threat from intrusion. The purpose of this research is to propose a methodology that will lead to a high level and sustainable protection against cyberattacks. In particular, an adaptive anomaly detection framework model was developed using deep and machine learning algorithms to manage automatically-configured application-level firewalls. The standard network datasets were used to evaluate the proposed model which is designed for improving the cybersecurity system. The deep learning based on Long-Short Term Memory Recurrent Neural Network (LSTM-RNN) and machine learning algorithms namely Support Vector Machine (SVM), K-Nearest Neighbor (K-NN) algorithms were implemented to classify the Denial-of-Service attack (DoS) and Distributed Denial-of-Service (DDoS) attacks. The information gain method was applied to select the relevant features from the network dataset. These network features were significant to improve the classification algorithm. The system was used to classify DoS and DDoS attacks in four stand datasets namely KDD cup 199, NSL-KDD, ISCX, and ICI-ID2017. The empirical results indicate that the deep learning based on the LSTM-RNN algorithm has obtained the highest accuracy. The proposed system based on the LSTM-RNN algorithm produced the highest testing accuracy rate of 99.51% and 99.91% with respect to KDD Cup’99, NSL-KDD, ISCX, and ICI-Id2017 datasets, respectively. A comparative result analysis between the machine learning algorithms, namely SVM and KNN, and the deep learning algorithms based on the LSTM-RNN model is presented. Finally, it is concluded that the LSTM-RNN model is efficient and effective to improve the cybersecurity system for detecting anomaly-based cybersecurity.

Machine learning-based anomaly detection via integration of manufacturing, inspection and after-sales service data

2017 ◽  
Vol 117 (5) ◽  
pp. 927-945 ◽  
Author(s):  
Taehoon Ko ◽  
Je Hyuk Lee ◽  
Hyunchang Cho ◽  
Sungzoon Cho ◽  
Wounjoo Lee ◽  
...  
Keyword(s):  
Machine Learning ◽  
Quality Management ◽  
Data Integration ◽  
Anomaly Detection ◽  
Learning Algorithms ◽  
Perceived Quality ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Content Type ◽  
Service Data

Purpose Quality management of products is an important part of manufacturing process. One way to manage and assure product quality is to use machine learning algorithms based on relationship among various process steps. The purpose of this paper is to integrate manufacturing, inspection and after-sales service data to make full use of machine learning algorithms for estimating the products’ quality in a supervised fashion. Proposed frameworks and methods are applied to actual data associated with heavy machinery engines. Design/methodology/approach By following Lenzerini’s formula, manufacturing, inspection and after-sales service data from various sources are integrated. The after-sales service data are used to label each engine as normal or abnormal. In this study, one-class classification algorithms are used due to class imbalance problem. To address multi-dimensionality of time series data, the symbolic aggregate approximation algorithm is used for data segmentation. Then, binary genetic algorithm-based wrapper approach is applied to segmented data to find the optimal feature subset. Findings By employing machine learning-based anomaly detection models, an anomaly score for each engine is calculated. Experimental results show that the proposed method can detect defective engines with a high probability before they are shipped. Originality/value Through data integration, the actual customer-perceived quality from after-sales service is linked to data from manufacturing and inspection process. In terms of business application, data integration and machine learning-based anomaly detection can help manufacturers establish quality management policies that reflect the actual customer-perceived quality by predicting defective engines.

scholarly journals Anomaly Detection in ICS Datasets with Machine Learning Algorithms

2021 ◽  
Vol 37 (1) ◽  
pp. 33-46
Author(s):  
Sinil Mubarak ◽  
Mohamed Hadi Habaebi ◽  
Md Rafiqul Islam ◽  
Farah Diyana Abdul Rahman ◽  
Mohammad Tahir
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Learning Algorithms ◽  
Machine Learning Algorithms

scholarly journals Anomaly detection in the Zwicky Transient Facility DR3

2021 ◽  
Author(s):  
K L Malanchev ◽  
M V Pruzhinskaya ◽  
V S Korolev ◽  
P D Aleo ◽  
M V Kornilov ◽  
...  
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Domain Knowledge ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Light Curves ◽  
Image Subtraction ◽  
Scientific Application ◽  
Public Data ◽  
Expert Analysis

Abstract We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of 3 stages: feature extraction, search of outliers with machine learning algorithms and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million objects. A set of 4 automatic learning algorithms was used to identify 277 outliers, which were subsequently scrutinised by an expert. From these, 188 (68%) were found to be bogus light curves – including effects from the image subtraction pipeline as well as overlapping between a star and a known asteroid, 66 (24%) were previously reported sources whereas 23 (8%) correspond to non-catalogued objects, with the two latter cases of potential scientific interest (e. g. 1 spectroscopically confirmed RS Canum Venaticorum star, 4 supernovae candidates, 1 red dwarf flare). Moreover, using results from the expert analysis, we were able to identify a simple bi-dimensional relation which can be used to aid filtering potentially bogus light curves in future studies. We provide a complete list of objects with potential scientific application so they can be further scrutinised by the community. These results confirm the importance of combining automatic machine learning algorithms with domain knowledge in the construction of recommendation systems for astronomy. Our code is publicly available*.

Sign in / Sign up

Export Citation Format

Share Document