Bayesian Nonparametric Unsupervised Concept Drift Detection for Data Stream Mining

Data coming from different sources is referred to as data streams. Data stream mining is an online learning technique where each data point must be processed as the data arrives and discarded as the processing is completed. Progress of technologies has resulted in the monitoring these data streams in real time. Data streams has created many new challenges to the researchers in real time. The main features of this type of data are they are fast flowing, large amounts of data which are continuous and growing in nature, and characteristics of data might change in course of time which is termed as concept drift. This chapter addresses the problems in mining data streams with concept drift. Due to which, isolating the correct literature would be a grueling task for researchers and practitioners. This chapter tries to provide a solution as it would be an amalgamation of all techniques used for data stream mining with concept drift.

Download Full-text

Data Stream Mining Using Ensemble Classifier

Collaborative Filtering Using Data Mining and Analysis - Advances in Data Mining and Database Management ◽

10.4018/978-1-5225-0489-4.ch013 ◽

2017 ◽

pp. 236-249

Author(s):

Snehlata Sewakdas Dongre ◽

Latesh G. Malik

Keyword(s):

Collaborative Filtering ◽

Data Stream ◽

Concept Drift ◽

Ensemble Classifier ◽

Ensemble Classification ◽

Data Stream Mining ◽

Main Concern ◽

Stream Mining ◽

Stream Classification ◽

Data Stream Classification

A data stream is giant amount of data which is generated uncontrollably at a rapid rate from many applications like call detail records, log records, sensors applications etc. Data stream mining has grasped the attention of so many researchers. A rising problem in Data Streams is the handling of concept drift. To be a good algorithm it should adapt the changes and handle the concept drift properly. Ensemble classification method is the group of classifiers which works in collaborative manner. Overall this chapter will cover all the aspects of the data stream classification. The mission of this chapter is to discuss various techniques which use collaborative filtering for the data stream mining. The main concern of this chapter is to make reader familiar with the data stream domain and data stream mining. Instead of single classifier the group of classifiers is used to enhance the accuracy of classification. The collaborative filtering will play important role here how the different classifiers work collaborative within the ensemble to achieve a goal.

Download Full-text

Data stream mining: methods and challenges for handling concept drift

SN Applied Sciences ◽

10.1007/s42452-019-1433-0 ◽

2019 ◽

Vol 1 (11) ◽

Cited By ~ 5

Author(s):

Scott Wares ◽

John Isaacs ◽

Eyad Elyan

Keyword(s):

Data Stream ◽

Concept Drift ◽

Relevant Literature ◽

Streaming Data ◽

Future Research ◽

Stream Mining ◽

Detection Algorithms ◽

The Past ◽

Concept Drift Detection ◽

The Impact

Abstract Mining and analysing streaming data is crucial for many applications, and this area of research has gained extensive attention over the past decade. However, there are several inherent problems that continue to challenge the hardware and the state-of-the art algorithmic solutions. Examples of such problems include the unbound size, varying speed and unknown data characteristics of arriving instances from a data stream. The aim of this research is to portray key challenges faced by algorithmic solutions for stream mining, particularly focusing on the prevalent issue of concept drift. A comprehensive discussion of concept drift and its inherent data challenges in the context of stream mining is presented, as is a critical, in-depth review of relevant literature. Current issues with the evaluative procedure for concept drift detectors is also explored, highlighting problems such as a lack of established base datasets and the impact of temporal dependence on concept drift detection. By exposing gaps in the current literature, this study suggests recommendations for future research which should aid in the progression of stream mining and concept drift detection algorithms.

Download Full-text

COLLABORATIVE DATA STREAM MINING IN UBIQUITOUS ENVIRONMENTS USING DYNAMIC CLASSIFIER SELECTION

International Journal of Information Technology & Decision Making ◽

10.1142/s0219622013500375 ◽

2013 ◽

Vol 12 (06) ◽

pp. 1287-1308 ◽

Cited By ~ 4

Author(s):

JOÃO BÁRTOLO GOMES ◽

MOHAMED MEDHAT GABER ◽

PEDRO A. C. SOUSA ◽

ERNESTINA MENASALVAS

Keyword(s):

Classification Accuracy ◽

Data Stream ◽

Concept Drift ◽

Feature Space ◽

Data Stream Mining ◽

Stream Mining ◽

Stream Classification ◽

Classifier Selection ◽

Local Accuracy ◽

Real World Datasets

In ubiquitous data stream mining, different devices often aim to learn concepts that are similar to some extent. In many applications, such as spam filtering or news recommendation, the data stream underlying concept (e.g., interesting mail/news) is likely to change over time. Therefore, the resultant model must be continuously adapted to such changes. This paper presents a novel Collaborative Data Stream Mining (Coll-Stream) approach that explores the similarities in the knowledge available from other devices to improve local classification accuracy. Coll-Stream integrates the community knowledge using an ensemble method where the classifiers are selected and weighted based on their local accuracy for different partitions of the feature space. We evaluate Coll-Stream classification accuracy in situations with concept drift, noise, partition granularity and concept similarity in relation to the local underlying concept. The experimental results show that Coll-Stream resultant model achieves stability and accuracy in a variety of situations using both synthetic and real-world datasets.

Download Full-text

Online Incremental Learning for High Bandwidth Network Traffic Classification

Applied Computational Intelligence and Soft Computing ◽

10.1155/2016/1465810 ◽

2016 ◽

Vol 2016 ◽

pp. 1-13 ◽

Cited By ~ 5

Author(s):

H. R. Loo ◽

S. B. Joseph ◽

M. N. Marsono

Keyword(s):

Network Traffic ◽

Data Stream ◽

Concept Drift ◽

Distance Measures ◽

Manhattan Distance ◽

Data Stream Mining ◽

Traffic Classification ◽

Stream Mining ◽

Network Traffic Classification ◽

High Bandwidth

Data stream mining techniques are able to classify evolving data streams such as network traffic in the presence of concept drift. In order to classify high bandwidth network traffic in real-time, data stream mining classifiers need to be implemented on reconfigurable high throughput platform, such as Field Programmable Gate Array (FPGA). This paper proposes an algorithm for online network traffic classification based on the concept of incrementalk-means clustering to continuously learn from both labeled and unlabeled flow instances. Two distance measures for incrementalk-means (Euclidean and Manhattan) distance are analyzed to measure their impact on the network traffic classification in the presence of concept drift. The experimental results on real datasets show that the proposed algorithm exhibits consistency, up to 94% average accuracy for both distance measures, even in the presence of concept drifts. The proposed incrementalk-means classification using Manhattan distance can classify network traffic 3 times faster than Euclidean distance at 671 thousands flow instances per second.

Download Full-text