Dimensionality Reduction of Production Data Using PCA and DBSCAN Techniques

2020 ◽  
Author(s):  
Umadevi S ◽  
NirmalaSugirthaRajini
Keyword(s):  
Data Mining ◽  
Data Analysis ◽  
Dimensionality Reduction ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Production Data ◽  
Related Data ◽  
Dbscan Clustering ◽  
Analysis Process ◽  
Research Article

Now a day’s data mining concepts are applied in various fields like medical, agriculture, production, etc. Creation of cluster is one of the major problems in data analysis process. Various clustering algorithms are used for data analysis purpose which depends upon the applications. DBSCAN is the famous method to create cluster. This article describes DBSCAN clustering concept applied on production database. The main objective of this research article is to collect and group the related data from large amount of data and remove the unwanted data. This clustering algorithm removes the unwanted attributes and groups the related data based upon density value.

scholarly journals Ensemble Hybrid K- Means and DBSCAN Clustering Algorithm – HDKA for Cancer Dataset

2019 ◽  
Vol 8 (4) ◽  
pp. 6036-6040
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Experimental Results ◽  
Cancer Dataset ◽  
Dbscan Clustering ◽  
Selection Of

Data Mining is the foremost vital space of analysis and is pragmatically utilized in totally different domains, It becomes a highly demanding field because huge amounts of data have been collected in various applications. The database can be clustered in more number of ways depending on the clustering algorithm used, parameter settings and other factors. Multiple clustering algorithms can be combined to get the final partitioning of data which provides better clustering results. In this paper, Ensemble hybrid KMeans and DBSCAN (HDKA) algorithm has been proposed to overcome the drawbacks of DBSCAN and KMeans clustering algorithms. The performance of the proposed algorithm improves the selection of centroid points through the centroid selection strategy.For experimental results we have used two dataset Colon and Leukemia from UCI machine learning repository.

K-MEANS CLUSTERING ALGORITHM FOR SERVICE DATA ANALYSIS BASED ON CUSTOMERS COMBINATION

2018 ◽  
Vol 3 (1) ◽  
pp. 001
Author(s):  
Zulhendra Zulhendra ◽  
Gunadi Widi Nurcahyo ◽  
Julius Santony
Keyword(s):  
Data Mining ◽  
Data Analysis ◽  
Clustering Algorithm ◽  
Customer Complaints ◽  
Using Data ◽  
Clustering Data ◽  
Service Data ◽  
Selection Of

In this study using Data Mining, namely K-Means Clustering. Data Mining can be used in searching for a large enough data analysis that aims to enable Indocomputer to know and classify service data based on customer complaints using Weka Software. In this study using the algorithm K-Means Clustering to predict or classify complaints about hardware damage on Payakumbuh Indocomputer. And can find out the data of Laptop brands most do service on Indocomputer Payakumbuh as one of the recommendations to consumers for the selection of Laptops.

scholarly journals A Novel Clustering Algorithm Based on Agent Technology for VANET

2016 ◽  
Vol 8 (2) ◽  
pp. 1 ◽  
Author(s):  
Samira Harrabi ◽  
Ines Ben Jaafar ◽  
Khaled Ghedira
Keyword(s):  
Traffic Congestion ◽  
Ad Hoc Network ◽  
Clustering Algorithm ◽  
Ad Hoc ◽  
Mobile Ad Hoc Network ◽  
Clustering Algorithms ◽  
High Mobility ◽  
Agent Technology ◽  
Related Data ◽  
Network Overhead

Vehicular Ad-hoc Network (VANET) is a sub-family of Mobile Ad-hoc Network (MANET).The means goal of VANET is to provide communications between nearby nodes or between nodes and fixed infrastructure. Despite that VANET is considered as a subclass of MANET, it has for particularity the high mobility of vehicles producing the frequent changes of network topology that involve changing of road, varying node density and locations of vehicles existing in this road. That‘s why, the most proposed clustering algorithms for MANET are unsuitable for VANET. Various searches have been recently published deal with clustering for VANETs. But most of them are focused on minimizing network overhead value, number of created clusters and had not considered the vehicles interests which defined as any related data used to differentiate vehicle from another (such as traffic congestion, looking for free parking space etc). In this paper, we propose a novel clustering algorithm based on agent technology to solve the problems mentioned above and improve routing in VANET. Experimental part show promising results regarding the adoption of the proposed approach.

scholarly journals Preprocessing Profiling Model for Visual Analytics

2020 ◽  
Author(s):  
Alessandra Maciel Paz Milani ◽  
Fernando V. Paulovich ◽  
Isabel Harb Manssour
Keyword(s):  
Data Mining ◽  
Data Analysis ◽  
Visual Analytics ◽  
Data Preprocessing ◽  
Interview Study ◽  
Raw Data ◽  
Important Stage ◽  
Analysis Process

Analyzing and managing raw data are still a challenging part of the data analysis process, mainly regarding data preprocessing. Although we can find studies proposing design implications or recommendations for visualization solutions in the data analysis scope, they do not focus on challenges during the preprocessing phase. Likewise, the current Visual Analytics processes do not consider preprocessing an equally important stage in their process. Thus, with this study, we aim to contribute to the discussion of how we can use and combine methods of visualization and data mining to assist data analysts during the preprocessing activities. To achieve that, we introduce the Preprocessing Profiling Model for Visual Analytics, which contemplates a set of features to inspire the implementation of new solutions. In turn, these features were designed considering a list of insights we obtained during an interview study with thirteen data analysts. Our contributions can be summarized as offering resources to promote a shift to a visual preprocessing.

scholarly journals Improved minimum-minimum roughness algorithm for clustering categorical data

2021 ◽  
Vol 8 (10) ◽  
pp. 43-50
Author(s):  
Truong et al. ◽  
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Hierarchical Clustering ◽  
Categorical Data ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Experimental Results ◽  
Data Sets ◽  
Top Down ◽  
Hierarchical Clustering Algorithm

Clustering is a fundamental technique in data mining and machine learning. Recently, many researchers are interested in the problem of clustering categorical data and several new approaches have been proposed. One of the successful and pioneering clustering algorithms is the Minimum-Minimum Roughness algorithm (MMR) which is a top-down hierarchical clustering algorithm and can handle the uncertainty in clustering categorical data. However, MMR tends to choose the category with less value leaf node with more objects, leading to undesirable clustering results. To overcome such shortcomings, this paper proposes an improved version of the MMR algorithm for clustering categorical data, called IMMR (Improved Minimum-Minimum Roughness). Experimental results on actual data sets taken from UCI show that the IMMR algorithm outperforms MMR in clustering categorical data.

scholarly journals Comparison of dimensionality reduction and clustering methods for SARS-CoV-2 genome

2021 ◽  
Vol 10 (4) ◽  
pp. 2170-2180
Author(s):  
Untari N. Wisesty ◽  
Tati Rajab Mengko
Keyword(s):  
Dimensionality Reduction ◽  
Dimensional Reduction ◽  
Clustering Algorithm ◽  
Sequence Data ◽  
Clustering Algorithms ◽  
Gaussian Mixture Models ◽  
Reduction Process ◽  
Principal Component ◽  
Gaussian Mixture ◽  
Clustering Methods

This paper aims to conduct an analysis of the SARS-CoV-2 genome variation was carried out by comparing the results of genome clustering using several clustering algorithms and distribution of sequence in each cluster. The clustering algorithms used are K-means, Gaussian mixture models, agglomerative hierarchical clustering, mean-shift clustering, and DBSCAN. However, the clustering algorithm has a weakness in grouping data that has very high dimensions such as genome data, so that a dimensional reduction process is needed. In this research, dimensionality reduction was carried out using principal component analysis (PCA) and autoencoder method with three models that produce 2, 10, and 50 features. The main contributions achieved were the dimensional reduction and clustering scheme of SARS-CoV-2 sequence data and the performance analysis of each experiment on each scheme and hyper parameters for each method. Based on the results of experiments conducted, PCA and DBSCAN algorithm achieve the highest silhouette score of 0.8770 with three clusters when using two features. However, dimensionality reduction using autoencoder need more iterations to converge. On the testing process with Indonesian sequence data, more than half of them enter one cluster and the rest are distributed in the other two clusters.

scholarly journals Canonical PSO Based K-Means Clustering Approach for Real Datasets

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Lopamudra Dey ◽  
Sanjay Chakraborty
Keyword(s):  
Data Mining ◽  
Air Pollution ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Real Life ◽  
Cluster Validity ◽  
Validity Assessment ◽  
Different Types ◽  
Clustering Approach ◽  
Validity Measure

“Clustering” the significance and application of this technique is spread over various fields. Clustering is an unsupervised process in data mining, that is why the proper evaluation of the results and measuring the compactness and separability of the clusters are important issues. The procedure of evaluating the results of a clustering algorithm is known as cluster validity measure. Different types of indexes are used to solve different types of problems and indices selection depends on the kind of available data. This paper first proposes Canonical PSO based K-means clustering algorithm and also analyses some important clustering indices (intercluster, intracluster) and then evaluates the effects of those indices on real-time air pollution database, wholesale customer, wine, and vehicle datasets using typical K-means, Canonical PSO based K-means, simple PSO based K-means, DBSCAN, and Hierarchical clustering algorithms. This paper also describes the nature of the clusters and finally compares the performances of these clustering algorithms according to the validity assessment. It also defines which algorithm will be more desirable among all these algorithms to make proper compact clusters on this particular real life datasets. It actually deals with the behaviour of these clustering algorithms with respect to validation indexes and represents their results of evaluation in terms of mathematical and graphical forms.

A Data Distribution View of Clustering Algorithms

2011 ◽  
pp. 374-381 ◽  
Author(s):  
Junjie Wu ◽  
Jian Chen ◽  
Hui Xiong
Keyword(s):  
Data Mining ◽  
Cluster Analysis ◽  
Clustering Analysis ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Data Distribution ◽  
Point Of View ◽  
Group Method ◽  
Data Sets ◽  
Distribution Point

Cluster analysis (Jain & Dubes, 1988) provides insight into the data by dividing the objects into groups (clusters), such that objects in a cluster are more similar to each other than objects in other clusters. Cluster analysis has long played an important role in a wide variety of fields, such as psychology, bioinformatics, pattern recognition, information retrieval, machine learning, and data mining. Many clustering algorithms, such as K-means and Unweighted Pair Group Method with Arithmetic Mean (UPGMA), have been wellestablished. A recent research focus on clustering analysis is to understand the strength and weakness of various clustering algorithms with respect to data factors. Indeed, people have identified some data characteristics that may strongly affect clustering analysis including high dimensionality and sparseness, the large size, noise, types of attributes and data sets, and scales of attributes (Tan, Steinbach, & Kumar, 2005). However, further investigation is expected to reveal whether and how the data distributions can have the impact on the performance of clustering algorithms. Along this line, we study clustering algorithms by answering three questions: 1. What are the systematic differences between the distributions of the resultant clusters by different clustering algorithms? 2. How can the distribution of the “true” cluster sizes make impact on the performances of clustering algorithms? 3. How to choose an appropriate clustering algorithm in practice? The answers to these questions can guide us for the better understanding and the use of clustering methods. This is noteworthy, since 1) in theory, people seldom realized that there are strong relationships between the clustering algorithms and the cluster size distributions, and 2) in practice, how to choose an appropriate clustering algorithm is still a challenging task, especially after an algorithm boom in data mining area. This chapter thus tries to fill this void initially. To this end, we carefully select two widely used categories of clustering algorithms, i.e., K-means and Agglomerative Hierarchical Clustering (AHC), as the representative algorithms for illustration. In the chapter, we first show that K-means tends to generate the clusters with a relatively uniform distribution on the cluster sizes. Then we demonstrate that UPGMA, one of the robust AHC methods, acts in an opposite way to K-means; that is, UPGMA tends to generate the clusters with high variation on the cluster sizes. Indeed, the experimental results indicate that the variations of the resultant cluster sizes by K-means and UPGMA, measured by the Coefficient of Variation (CV), are in the specific intervals, say [0.3, 1.0] and [1.0, 2.5] respectively. Finally, we put together K-means and UPGMA for a further comparison, and propose some rules for the better choice of the clustering schemes from the data distribution point of view.

Research on Parallel DBSCAN Algorithm Design Based on MapReduce

2011 ◽  
Vol 301-303 ◽  
pp. 1133-1138 ◽  
Author(s):  
Yan Xiang Fu ◽  
Wei Zhong Zhao ◽  
Hui Fang Ma
Keyword(s):  
Data Clustering ◽  
Large Scale ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Algorithm Design ◽  
Document Retrieval ◽  
Commodity Hardware ◽  
Dbscan Clustering ◽  
Dbscan Algorithm ◽  
Parallel Clustering

Data clustering has been received considerable attention in many applications, such as data mining, document retrieval, image segmentation and pattern classification. The enlarging volumes of information emerging by the progress of technology, makes clustering of very large scale of data a challenging task. In order to deal with the problem, more researchers try to design efficient parallel clustering algorithms. In this paper, we propose a parallel DBSCAN clustering algorithm based on Hadoop, which is a simple yet powerful parallel programming platform. The experimental results demonstrate that the proposed algorithm can scale well and efficiently process large datasets on commodity hardware.

scholarly journals A Comparison of K-Means and Mean Shift Algorithms

2021 ◽  
Author(s):  
Mehak Nigar Shumaila
Keyword(s):  
Cluster Analysis ◽  
Data Analysis ◽  
Time Complexity ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Mean Shift ◽  
Prediction Performance ◽  
Learning Problem ◽  
Cluster A ◽  
Formation Of Groups

Clustering, or otherwise known as cluster analysis, is a learning problem that takes place without any human supervision. This technique has often been utilized, much efficiently, in data analysis, and serves for observing and identifying interesting, useful, or desired patterns in the said data. The clustering technique functions by performing a structured division of the data involved, in similar objects based on the characteristics that it identifies. This process results in the formation of groups, and each group that is formed, is called a cluster. A single said cluster consists of objects from the data, that have similarities among other objects found in the same cluster, and resemble differences when compared to objects identified from the data that now exist in other clusters. The process of clustering is very significant in various aspects of data analysis, as it determines and presents the intrinsic grouping of objects present in the data, based on their attributes, in a batch of unlabeled raw data. A textbook or otherwise said, good criteria, does not exist in this method of cluster analysis. That is because this process is so different and so customizable for every user, that needs it in his/her various and different needs. There is no outright best clustering algorithm, as it massively depends on the user’s scenario and needs. This paper is intended to compare and study two different clustering algorithms. The algorithms under investigation are k-mean and mean shift. These algorithms are compared according to the following factors: time complexity, training, prediction performance and accuracy of the clustering algorithms.

Sign in / Sign up

Export Citation Format

Share Document