Outlier Detection Method based on Improved K-means Clustering Algorithm

Outlier detection has important applications in various data mining domains such as fraud detection, intrusion detection, customers' behavior and employees' performance analysis. Outliers are characterized by being significantly or "interestingly" different from the rest of the data. In this paper, a novel cluster-based outlier detection method is proposed using a humoral-mediated clustering algorithm (HAIS) based on concepts of antibody secretion in natural immune systems. The proposed method finds meaningful clusters as well as outliers simultaneously. This is an iterative approach where only clusters above threshold (larger sized clusters) are carried forward to the next cycle of cluster formation while removing small sized clusters. This paper also demonstrates through experimental results that the mere existence of outliers severely affects the clustering outcome, and removing those outliers can result in better clustering solutions. The feasibility of the method is demonstrated through simulated datasets, current datasets from the literature as well as a real-world doctors' performance evaluation dataset where the task is to identify potentially under-performing doctors. The results indicate that HAIS has capabilities of detecting single point as well as cluster-based outliers.

Download Full-text

Outlier Detection Method on UCI Repository Dataset by Entropy Based Rough K-means

Defence Science Journal ◽

10.14429/dsj.66.9463 ◽

2016 ◽

Vol 66 (2) ◽

pp. 113 ◽

Cited By ~ 2

Author(s):

Ashok P. ◽

G.M Kadhar Nawaz

Keyword(s):

Outlier Detection ◽

Clustering Algorithm ◽

Rough Set Theory ◽

Detection Method ◽

Selection Method ◽

Cluster Validity ◽

Upper Approximation ◽

Yeast Dataset ◽

Protein Localisation ◽

Selection Of

<p>Rough set theory is used to handle uncertainty and incomplete information by applying two sets, lower and upper approximation. In this paper, the clustering process is improved by adapting the preliminary centroid selection method on rough K-means (RKM) algorithm. The entropy based rough K-means (ERKM) method is developed by adapting entropy based preliminary centroids selection on RKM and executed and also validated by cluster validity indexes. An example shows that the ERKM performs effectively by selection of entropy based preliminary centroid. In addition, Outlier detection is an important task in data mining and very much different from the rest of the objects in the cluster. Entropy based rough outlier factor (EROF) method is used to detect outlier effectively for yeast dataset. An example shows that EROF detects outlier effectively on protein localisation sites and ERKM clustering algorithm performed effectively. Further, experimental readings show that the ERKM and EROF method outperformed the other methods.</p><p> </p>

Download Full-text