scholarly journals Detection of Electricity Theft Behavior Based on Improved Synthetic Minority Oversampling Technique and Random Forest Classifier

Energies ◽  
2020 ◽  
Vol 13 (8) ◽  
pp. 2039 ◽  
Author(s):  
Zhengwei Qu ◽  
Hongwen Li ◽  
Yunjing Wang ◽  
Jiaxi Zhang ◽  
Ahmed Abu-Siada ◽  
...  
Keyword(s):  
Random Forest ◽  
Smart Grids ◽  
Clustering Algorithm ◽  
Optimal Number ◽  
Cluster Center ◽  
Complex Data ◽  
Electricity Theft ◽  
Positive Data ◽  
Data Error ◽  
Detection Technologies

Effective detection of electricity theft is essential to maintain power system reliability. With the development of smart grids, traditional electricity theft detection technologies have become ineffective to deal with the increasingly complex data on the users’ side. To improve the auditing efficiency of grid enterprises, a new electricity theft detection method based on improved synthetic minority oversampling technique (SMOTE) and improve random forest (RF) method is proposed in this paper. The data of normal and electricity theft users were classified as positive data (PD) and negative data (ND), respectively. In practice, the number of ND was far less than PD, which made the dataset composed of these two types of data become unbalanced. An improved SOMTE based on K-means clustering algorithm (K-SMOTE) was firstly presented to balance the dataset. The cluster center of ND was determined by K-means method. Then, the ND were interpolated by SMOTE on the basis of the cluster center to balance the entire data. Finally, the RF classifier was trained with the balanced dataset, and the optimal number of decision trees in RF was decided according to the convergence of out-of-bag data error (OOB error). Electricity theft behaviors on the user side were detected by the trained RF classifier.

scholarly journals Data Analysis Using Representation Theory and Clustering Algorithms

2021 ◽  
Vol 19 ◽  
pp. 310-320
Author(s):  
Suboh Alkhushayni ◽  
Taeyoung Choi ◽  
Du’a Alzaleq
Keyword(s):  
Data Analysis ◽  
Random Forest ◽  
Hierarchical Clustering ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Optimal Number ◽  
Categorical Variables ◽  
Common Disease ◽  
Agglomerative Hierarchical Clustering ◽  
Data Set

This work aims to expand the knowledge of the area of data analysis through both persistence homology, as well as representations of directed graphs. To be specific, we looked for how we can analyze homology cluster groups using agglomerative Hierarchical Clustering algorithms and methods. Additionally, the Wine data, which is offered in R studio, was analyzed using various cluster algorithms such as Hierarchical Clustering, K-Means Clustering, and PAM Clustering. The goal of the analysis was to find out which cluster's method is proper for a given numerical data set. By testing the data, we tried to find the agglomerative hierarchical clustering method that will be the optimal clustering algorithm among these three; K-Means, PAM, and Random Forest methods. By comparing each model's accuracy value with cultivar coefficients, we came with a conclusion that K-Means methods are the most helpful when working with numerical variables. On the other hand, PAM clustering and Gower with random forest are the most beneficial approaches when working with categorical variables. All these tests can determine the optimal number of clustering groups, given the data set, and by doing the proper analysis. Using those the project, we can apply our method to several industrial areas such that clinical, business, and others. For example, people can make different groups based on each patient who has a common disease, required therapy, and other things in the clinical society. Additionally, for the business area, people can expect to get several clustered groups based on the marginal profit, marginal cost, or other economic indicators.

scholarly journals Data analysis using representation theory and clustering algorithms

2020 ◽  
Vol 9 (4) ◽  
pp. 887
Author(s):  
Suboh Alkhushayni ◽  
Taeyoung Choi ◽  
Du'a Alzaleq
Keyword(s):  
Data Analysis ◽  
Random Forest ◽  
Hierarchical Clustering ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Optimal Number ◽  
Categorical Variables ◽  
Common Disease ◽  
Agglomerative Hierarchical Clustering ◽  
Data Set

This work aims to expand the knowledge of the area of data analysis through persistence homology and representations of directed graphs. To be specific, we looked for how we can analyze homology cluster groups using agglomerative Hierarchical Clustering algorithms and methods. Additionally, the Wine data, which is offered in R studio, was analyzed using various cluster algorithms such as Hierarchical Clustering, K-Means Clustering, and PAM Clustering. The goal of the analysis was to find out which cluster's method is proper for a given numerical dataset. We tried to find the agglomerative hierarchical clustering method by testing the data that will be the optimal clustering algorithm among these three; K-Means, PAM, and Random Forest methods. By comparing each model's accuracy value with cultivar coefficients, we concluded that K-Means methods are the most helpful when working with numerical variables. On the other hand, PAM clustering and Gower with Random Forest are the most beneficial approaches when using categorical variables. These tests can determine the optimal number of clustering groups, given the data set, and by doing the proper analysis. Using those the project, we can apply our method to several industrial areas such that clinical, business, and others. For example, people can make different groups based on each patient who has a common disease, required therapy, and other things in the clinical society. Additionally, people can expect to get several clustered groups based on the marginal profit, marginal cost, or other economic indicators for the business area.

Implementation of data mining as a support of business application strategy

2018 ◽  
Vol 5 (1) ◽  
pp. 47-55
Author(s):  
Florensia Unggul Damayanti
Keyword(s):  
Data Mining ◽  
Random Forest ◽  
Business Strategy ◽  
Input Parameter ◽  
Data Mining Algorithm ◽  
Complex Data ◽  
Business Decision ◽  
Marketing Department ◽  
Business Application ◽  
Complex Data Sets

Data mining help industries create intelligent decision on complex problems. Data mining algorithm can be applied to the data in order to forecasting, identity pattern, make rules and recommendations, analyze the sequence in complex data sets and retrieve fresh insights. Yet, increasing of technology and various techniques among data mining availability data give opportunity to industries to explore and gain valuable information from their data and use the information to support business decision making. This paper implement classification data mining in order to retrieve knowledge in customer databases to support marketing department while planning strategy for predict plan premium. The dataset decompose into conceptual analytic to identify characteristic data that can be used as input parameter of data mining model. Business decision and application is characterized by processing step, processing characteristic and processing outcome (Seng, J.L., Chen T.C. 2010). This paper set up experimental of data mining based on J48 and Random Forest classifiers and put a light on performance evaluation between J48 and random forest in the context of dataset in insurance industries. The experiment result are about classification accuracy and efficiency of J48 and Random Forest , also find out the most attribute that can be used to predict plan premium in context of strategic planning to support business strategy.

A Hard C-Means Clustering Algorithm Incorporating Membership KL Divergence and Local Data Information for Noisy Image Segmentation

2017 ◽  
Vol 32 (04) ◽  
pp. 1850012 ◽  
Author(s):  
R. R. Gharieb ◽  
G. Gendy ◽  
H. Selim
Keyword(s):  
Image Segmentation ◽  
Membership Function ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Cluster Center ◽  
Local Data ◽  
Cluster Membership ◽  
Kl Divergence ◽  
Clustering Approach ◽  
Center Distance

In this paper, the standard hard C-means (HCM) clustering approach to image segmentation is modified by incorporating weighted membership Kullback–Leibler (KL) divergence and local data information into the HCM objective function. The membership KL divergence, used for fuzzification, measures the proximity between each cluster membership function of a pixel and the locally-smoothed value of the membership in the pixel vicinity. The fuzzification weight is a function of the pixel to cluster-centers distances. The used pixel to a cluster-center distance is composed of the original pixel data distance plus a fraction of the distance generated from the locally-smoothed pixel data. It is shown that the obtained membership function of a pixel is proportional to the locally-smoothed membership function of this pixel multiplied by an exponentially distributed function of the minus pixel distance relative to the minimum distance provided by the nearest cluster-center to the pixel. Therefore, since incorporating the locally-smoothed membership and data information in addition to the relative distance, which is more tolerant to additive noise than the absolute distance, the proposed algorithm has a threefold noise-handling process. The presented algorithm, named local data and membership KL divergence based fuzzy C-means (LDMKLFCM), is tested by synthetic and real-world noisy images and its results are compared with those of several FCM-based clustering algorithms.

Random Forest Model for Trip End Identification Using Cellular Phone and Points of Interest Data

2021 ◽  
pp. 036119812110315
Author(s):  
Fei Yang ◽  
Yanchen Wang ◽  
Peter J. Jin ◽  
Dingbang Li ◽  
Zhenxing Yao
Keyword(s):  
Random Forest ◽  
Clustering Algorithm ◽  
Subjective Experience ◽  
Average Distance ◽  
Cellular Phone ◽  
Random Forest Model ◽  
Support Vector ◽  
Rule Based ◽  
Forest Model ◽  
Points Of Interest

Cellular phone data has been proven to be valuable in the analysis of residents’ travel patterns. Existing studies mostly identify the trip ends through rule-based or clustering algorithms. These methods largely depend on subjective experience and users’ communication behaviors. Moreover, limited by privacy policy, the accuracy of these methods is difficult to assess. In this paper, points of interest data is applied to supplement cellular phone data’s missing information generated by users’ behaviors. Specifically, a random forest model for trip end identification is proposed using multi-dimensional attributes. A field data acquisition test is designed and conducted with communication operators to implement synchronized cellular phone data and real trip information collection. The proposed identification approach is empirically evaluated with real trip information. Results show that the overall trip end detection precision and recall reach 95.2% and 88.7% with an average distance error of 269 m, and the time errors of the trip ends are less than 10 min. Compared with the rule-based approach, clustering algorithm, naive Bayes method, and support vector machine, the proposed method has better performance in accuracy and consistency.

Cloud-Precipitation Hybrid Regimes and their Projection onto IMERG Precipitation Data

2021 ◽  
Author(s):  
Daeho Jin ◽  
Lazaros Oreopoulos ◽  
Dongmin Lee ◽  
Jackson Tan ◽  
Nayeong Cho
Keyword(s):  
Clustering Algorithm ◽  
Relative Weight ◽  
Heavy Precipitation ◽  
Optimal Number ◽  
Hybrid Regimes ◽  
Precipitation Regimes ◽  
Moderate Resolution ◽  
Application Potential ◽  
Moderate Resolution Imaging Spectroradiometer ◽  
Cloud Top Pressure

AbstractIn order to better understand cloud-precipitation relationships, we extend the concept of cloud regimes (CRs) developed from two-dimensional joint histograms of cloud optical thickness and cloud top pressure from the Moderate Resolution Imaging Spectroradiometer (MODIS), to include precipitation information. Taking advantage of the high-resolution Integrated Multi-satellitE Retrievals for GPM (IMERG) precipitation dataset, we derive cloud-precipitation “hybrid” regimes by implementing a k-means clustering algorithm with advanced initialization and objective measures to determine the optimal number of clusters. By expressing the variability of precipitation rates within 1-degree grid cells as histograms and varying the relative weight of cloud and precipitation information in the clustering algorithm, we obtain several editions of hybrid cloud-precipitation regimes (CPRs), and examine their characteristics.In the deep tropics, when precipitation is weighted weakly, the cloud part centroids of the hybrid regimes resemble their counterparts of cloud-only regimes, but combined clustering tightens the cloud-precipitation relationship by decreasing each regime’s precipitation variability. As precipitation weight progressively increases, the shape of the cloud part centroids becomes blunter, while the precipitation part sharpens. When cloud and precipitation are weighted equally, the CPRs representing high clouds with intermediate to heavy precipitation exhibit distinct enough features in the precipitation parts of the centroids to allow us to project them onto the 30-min IMERG domain. Such a projection overcomes the temporal sparseness of MODIS cloud observations associated with substantial rainfall, suggesting great application potential for convection-focused studies where characterization of the diurnal cycle is essential.

Improved Fuzzy FCM-LI Algorithm

2013 ◽  
Vol 765-767 ◽  
pp. 670-673
Author(s):  
Li Bo Hou
Keyword(s):  
Real Time ◽  
Clustering Algorithm ◽  
Feature Analysis ◽  
Cluster Center ◽  
High Dimensional ◽  
Fuzzy C Means ◽  
Sample Data ◽  
Fuzzy C Means Clustering ◽  
Fcm Clustering ◽  
Np Hard Problem

Fuzzy C-means (FCM) clustering algorithm is one of the widely applied algorithms in non-supervision of pattern recognition. However, FCM algorithm in the iterative process requires a lot of calculations, especially when feature vectors has high-dimensional, Use clustering algorithm to sub-heap, not only inefficient, but also may lead to "the curse of dimensionality." For the problem, This paper analyzes the fuzzy C-means clustering algorithm in high dimensional feature of the process, the problem of cluster center is an np-hard problem, In order to improve the effectiveness and Real-time of fuzzy C-means clustering algorithm in high dimensional feature analysis, Combination of landmark isometric (L-ISOMAP) algorithm, Proposed improved algorithm FCM-LI. Preliminary analysis of the samples, Use clustering results and the correlation of sample data, using landmark isometric (L-ISOMAP) algorithm to reduce the dimension, further analysis on the basis, obtained the final results. Finally, experimental results show that the effectiveness and Real-time of FCM-LI algorithm in high dimensional feature analysis.

Sign in / Sign up

Export Citation Format

Share Document