scholarly journals A Feature Selection Algorithm Performance Metric for Comparative Analysis

Algorithms ◽  
2021 ◽  
Vol 14 (3) ◽  
pp. 100
Author(s):  
Werner Mostert ◽  
Katherine M. Malan ◽  
Andries P. Engelbrecht
Keyword(s):  
Feature Selection ◽  
Comparative Analysis ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Algorithm Selection ◽  
Algorithm Performance ◽  
Performance Space ◽  
Performance Metric ◽  
Real World Datasets ◽  
Selection Algorithms

This study presents a novel performance metric for feature selection algorithms that is unbiased and can be used for comparative analysis across feature selection problems. The baseline fitness improvement (BFI) measure quantifies the potential value gained by applying feature selection. The BFI measure can be used to compare the performance of feature selection algorithms across datasets by measuring the change in classifier performance as a result of feature selection, with respect to the baseline where all features are included. Empirical results are presented to show that there is performance complementarity for a suite of feature selection algorithms on a variety of real world datasets. The BFI measure is a normalised performance metric that can be used to correlate problem characteristics with feature selection algorithm performance, across multiple datasets. This ability paves the way towards describing the performance space of the per-instance algorithm selection problem for feature selection algorithms.

A NOVEL FEATURE SELECTION ALGORITHM WITH SUPERVISED MUTUAL INFORMATION FOR CLASSIFICATION

2013 ◽  
Vol 22 (04) ◽  
pp. 1350027
Author(s):  
JAGANATHAN PALANICHAMY ◽  
KUPPUCHAMY RAMASAMY
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Feature Selection ◽  
Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class A ◽  
Selection Algorithms ◽  
The Relationship ◽  
Class Variable

Feature selection is essential in data mining and pattern recognition, especially for database classification. During past years, several feature selection algorithms have been proposed to measure the relevance of various features to each class. A suitable feature selection algorithm normally maximizes the relevancy and minimizes the redundancy of the selected features. The mutual information measure can successfully estimate the dependency of features on the entire sampling space, but it cannot exactly represent the redundancies among features. In this paper, a novel feature selection algorithm is proposed based on maximum relevance and minimum redundancy criterion. The mutual information is used to measure the relevancy of each feature with class variable and calculate the redundancy by utilizing the relationship between candidate features, selected features and class variables. The effectiveness is tested with ten benchmarked datasets available in UCI Machine Learning Repository. The experimental results show better performance when compared with some existing algorithms.

scholarly journals A Mixed Feature Selection Method Considering Interaction

2015 ◽  
Vol 2015 ◽  
pp. 1-10 ◽  
Author(s):  
Zilin Zeng ◽  
Hongjun Zhang ◽  
Rui Zhang ◽  
Youliang Zhang
Keyword(s):  
Feature Selection ◽  
Rough Sets ◽  
Feature Selection Method ◽  
Feature Space ◽  
Feature Interaction ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Neighborhood Rough Sets ◽  
Real World Datasets ◽  
Neighborhood Interaction

Feature interaction has gained considerable attention recently. However, many feature selection methods considering interaction are only designed for categorical features. This paper proposes a mixed feature selection algorithm based on neighborhood rough sets that can be used to search for interacting features. In this paper, feature relevance, feature redundancy, and feature interaction are defined in the framework of neighborhood rough sets, the neighborhood interaction weight factor reflecting whether a feature is redundant or interactive is proposed, and a neighborhood interaction weight based feature selection algorithm (NIWFS) is brought forward. To evaluate the performance of the proposed algorithm, we compare NIWFS with other three feature selection algorithms, including INTERACT, NRS, and NMI, in terms of the classification accuracies and the number of selected features with C4.5 and IB1. The results from ten real world datasets indicate that NIWFS not only deals with mixed datasets directly, but also reduces the dimensionality of feature space with the highest average accuracies.

Proportional Hybrid Mechanism for Population Based Feature Selection Algorithm

2017 ◽  
Vol 16 (05) ◽  
pp. 1309-1338 ◽  
Author(s):  
Pin Wang ◽  
Yongming Li ◽  
Bohan Chen ◽  
Xianling Hu ◽  
Jin Yan ◽  
...  
Keyword(s):  
Feature Selection ◽  
High Efficiency ◽  
Search Algorithm ◽  
Population Based ◽  
Time Cost ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Filter Model ◽  
Hybrid Mechanism ◽  
Selection Algorithms

Feature selection is an important research field for pattern classification, data mining, etc. Population-based optimization algorithms (POA) have high parallelism and are widely used as search algorithm for feature selection. Population-based feature selection algorithms (PFSA) involve compromise between precision and time cost. In order to optimize the PFSA, the feature selection models need to be improved. Feature selection algorithms broadly fall into two categories: the filter model and the wrapper model. The filter model is fast but less precise; while the wrapper model is more precise but generally computationally more intensive. In this paper, we proposed a new mechanism — proportional hybrid mechanism (PHM) to combine the advantages of filter and wrapper models. The mechanism can be applied in PFSA to improve their performance. Genetic algorithm (GA) has been applied in many kinds of feature selection problems as search algorithm because of its high efficiency and implicit parallelism. Therefore, GAs are used in this paper. In order to validate the mechanism, seven datasets from university of California Irvine (UCI) database and artificial toy datasets are tested. The experiments are carried out for different GAs, classifiers, and evaluation criteria, the results show that with the introduction of PHM, the GA-based feature selection algorithm can be improved in both time cost and classification accuracy. Moreover, the comparison of GA-based, PSO-based and some other feature selection algorithms demonstrate that the PHM can be used in other population-based feature selection algorithms and obtain satisfying results.

scholarly journals A Feature Selection Algorithm Integrating Maximum Classification Information and Minimum Interaction Feature Dependency Information

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Li Zhang
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Information Gain ◽  
Small Sample ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class Labels ◽  
Minimum Interaction ◽  
Classification Information ◽  
Selection Algorithms

Feature selection is the key step in the analysis of high-dimensional small sample data. The core of feature selection is to analyse and quantify the correlation between features and class labels and the redundancy between features. However, most of the existing feature selection algorithms only consider the classification contribution of individual features and ignore the influence of interfeature redundancy and correlation. Therefore, this paper proposes a feature selection algorithm for nonlinear dynamic conditional relevance (NDCRFS) through the study and analysis of the existing feature selection algorithm ideas and method. Firstly, redundancy and relevance between features and between features and class labels are discriminated by mutual information, conditional mutual information, and interactive mutual information. Secondly, the selected features and candidate features are dynamically weighted utilizing information gain factors. Finally, to evaluate the performance of this feature selection algorithm, NDCRFS was validated against 6 other feature selection algorithms on three classifiers, using 12 different data sets, for variability and classification metrics between the different algorithms. The experimental results show that the NDCRFS method can improve the quality of the feature subsets and obtain better classification results.

Feature Selection Algorithm Using Relative Odds for Data Mining Classification

2020 ◽  
pp. 81-106 ◽  
Author(s):  
Donald Douglas Atsa'am
Keyword(s):  
Feature Selection ◽  
Binary Classification ◽  
Initial Step ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Classification Problems ◽  
Odds Ratios ◽  
Relative Odds ◽  
Importance Ranking ◽  
Selection Algorithms

A filter feature selection algorithm is developed and its performance tested. In the initial step, the algorithm dichotomizes the dataset then separately computes the association between each predictor and the class variable using relative odds (odds ratios). The value of the odds ratios becomes the importance ranking of the corresponding explanatory variable in determining the output. Logistic regression classification is deployed to test the performance of the new algorithm in comparison with three existing feature selection algorithms: the Fisher index, Pearson's correlation, and the varImp function. A number of experimental datasets are employed, and in most cases, the subsets selected by the new algorithm produced models with higher classification accuracy than the subsets suggested by the existing feature selection algorithms. Therefore, the proposed algorithm is a reliable alternative in filter feature selection for binary classification problems.

scholarly journals Feature Selection Combining Information Theory View and Algebraic View in the Neighborhood Decision System

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 704
Author(s):  
Jiucheng Xu ◽  
Kanglin Qu ◽  
Meng Yuan ◽  
Jie Yang
Keyword(s):  
Information Theory ◽  
Feature Selection ◽  
Rough Set ◽  
Rough Set Theory ◽  
Classification Performance ◽  
Joint Entropy ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Selection Algorithms ◽  
Theory View

Feature selection is one of the core contents of rough set theory and application. Since the reduction ability and classification performance of many feature selection algorithms based on rough set theory and its extensions are not ideal, this paper proposes a feature selection algorithm that combines the information theory view and algebraic view in the neighborhood decision system. First, the neighborhood relationship in the neighborhood rough set model is used to retain the classification information of continuous data, to study some uncertainty measures of neighborhood information entropy. Second, to fully reflect the decision ability and classification performance of the neighborhood system, the neighborhood credibility and neighborhood coverage are defined and introduced into the neighborhood joint entropy. Third, a feature selection algorithm based on neighborhood joint entropy is designed, which improves the disadvantage that most feature selection algorithms only consider information theory definition or algebraic definition. Finally, experiments and statistical analyses on nine data sets prove that the algorithm can effectively select the optimal feature subset, and the selection result can maintain or improve the classification performance of the data set.

A dynamic recursive feature elimination framework (dRFE) to further refine a set of OMIC biomarkers

2021 ◽  
Author(s):  
Yuanyuan Han ◽  
Lan Huang ◽  
Fengfeng Zhou
Keyword(s):  
Experimental Data ◽  
Feature Selection ◽  
The Other ◽  
Supplementary Information ◽  
Recursive Feature Elimination ◽  
Supplementary Data ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class Labels ◽  
Selection Algorithms

Abstract Motivation A feature selection algorithm may select the subset of features with the best associations with the class labels. The recursive feature elimination (RFE) is a heuristic feature screening framework and has been widely used to select the biological OMIC biomarkers. This study proposed a dynamic recursive feature elimination (dRFE) framework with more flexible feature elimination operations. The proposed dRFE was comprehensively compared with 11 existing feature selection algorithms and five classifiers on the eight difficult transcriptome datasets from a previous study, the ten newly collected transcriptome datasets and the five methylome datasets. Results The experimental data suggested that the regular RFE framework did not perform well, and dRFE outperformed the existing feature selection algorithms in most cases. The dRFE-detected features achieved Acc = 1.0000 for the two methylome datasets GSE53045 and GSE66695. The best prediction accuracies of the dRFE-detected features were 0.9259, 0.9424 and 0.8601 for the other three methylome datasets GSE74845, GSE103186 and GSE80970, respectively. Four transcriptome datasets received Acc = 1.0000 using the dRFE-detected features, and the prediction accuracies for the other six newly collected transcriptome datasets were between 0.6301 and 0.9917. Availability and implementation The experiments in this study are implemented and tested using the programming language Python version 3.7.6. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals A Multi-Objective Multi-Label Feature Selection Algorithm Based on Shapley Value

Entropy ◽  
2021 ◽  
Vol 23 (8) ◽  
pp. 1094
Author(s):  
Hongbin Dong ◽  
Jing Sun ◽  
Xiaohang Sun
Keyword(s):  
Feature Selection ◽  
Shapley Value ◽  
Feature Selection Method ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Multi Objective ◽  
True Label ◽  
Mutation Operators ◽  
Real World Datasets ◽  
Crossover And Mutation

Multi-label learning is dedicated to learning functions so that each sample is labeled with a true label set. With the increase of data knowledge, the feature dimensionality is increasing. However, high-dimensional information may contain noisy data, making the process of multi-label learning difficult. Feature selection is a technical approach that can effectively reduce the data dimension. In the study of feature selection, the multi-objective optimization algorithm has shown an excellent global optimization performance. The Pareto relationship can handle contradictory objectives in the multi-objective problem well. Therefore, a Shapley value-fused feature selection algorithm for multi-label learning (SHAPFS-ML) is proposed. The method takes multi-label criteria as the optimization objectives and the proposed crossover and mutation operators based on Shapley value are conducive to identifying relevant, redundant and irrelevant features. The comparison of experimental results on real-world datasets reveals that SHAPFS-ML is an effective feature selection method for multi-label classification, which can reduce the classification algorithm’s computational complexity and improve the classification accuracy.

scholarly journals Naive Bayes-Guided Bat Algorithm for Feature Selection

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Ahmed Majid Taha ◽  
Aida Mustapha ◽  
Soong-Der Chen
Keyword(s):  
Feature Selection ◽  
Classification Accuracy ◽  
Naive Bayes ◽  
Bat Algorithm ◽  
Naïve Bayes ◽  
Bayes Classifier ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Benchmark Datasets ◽  
Selection Algorithms

When the amount of data and information is said to double in every 20 months or so, feature selection has become highly important and beneficial. Further improvements in feature selection will positively affect a wide array of applications in fields such as pattern recognition, machine learning, or signal processing. Bio-inspired method called Bat Algorithm hybridized with a Naive Bayes classifier has been presented in this work. The performance of the proposed feature selection algorithm was investigated using twelve benchmark datasets from different domains and was compared to three other well-known feature selection algorithms. Discussion focused on four perspectives: number of features, classification accuracy, stability, and feature generalization. The results showed that BANB significantly outperformed other algorithms in selecting lower number of features, hence removing irrelevant, redundant, or noisy features while maintaining the classification accuracy. BANB is also proven to be more stable than other methods and is capable of producing more general feature subsets.

Sign in / Sign up

Export Citation Format

Share Document