scholarly journals An Improved Feature Selection Algorithm Based on Parzen Window and Conditional Mutual Information

2013 ◽  
Author(s):  
Deng-chao He ◽  
Wen-ning Hao ◽  
Gang Chen ◽  
Da-wei Jin
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Conditional Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Parzen Window

An Improved Feature Selection Algorithm Based on Parzen Window and Conditional Mutual Information

2013 ◽  
Vol 347-350 ◽  
pp. 2614-2619
Author(s):  
Deng Chao He ◽  
Wen Ning Hao ◽  
Gang Chen ◽  
Da Wei Jin
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Probability Density Functions ◽  
Density Functions ◽  
Continuous Variables ◽  
Conditional Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Parzen Window ◽  
Selection For

In this paper, an improved feature selection algorithm by conditional mutual information with Parzen window was proposed, which adopted conditional mutual information as an evaluation criterion of feature selection in order to overcome the deficiency of feature redundant and used Parzen window to estimate the probability density functions and calculate the conditional mutual information of continuous variables, in such a way as to achieve feature selection for continuous data.

scholarly journals Accelerating Causal Inference and Feature Selection Methods through G-Test Computation Reuse

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1501
Author(s):  
Camil Băncioiu ◽  
Remus Brad
Keyword(s):  
Feature Selection ◽  
Causal Inference ◽  
Selection Methods ◽  
Conditional Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Inference Method ◽  
Information Theoretic ◽  
Theoretic Concept ◽  
Test Computation

This article presents a novel and remarkably efficient method of computing the statistical G-test made possible by exploiting a connection with the fundamental elements of information theory: by writing the G statistic as a sum of joint entropy terms, its computation is decomposed into easily reusable partial results with no change in the resulting value. This method greatly improves the efficiency of applications that perform a series of G-tests on permutations of the same features, such as feature selection and causal inference applications because this decomposition allows for an intensive reuse of these partial results. The efficiency of this method is demonstrated by implementing it as part of an experiment involving IPC–MB, an efficient Markov blanket discovery algorithm, applicable both as a feature selection algorithm and as a causal inference method. The results show outstanding efficiency gains for IPC–MB when the G-test is computed with the proposed method, compared to the unoptimized G-test, but also when compared to IPC–MB++, a variant of IPC–MB which is enhanced with an AD–tree, both static and dynamic. Even if this proposed method of computing the G-test is presented here in the context of IPC–MB, it is in fact bound neither to IPC–MB in particular, nor to feature selection or causal inference applications in general, because this method targets the information-theoretic concept that underlies the G-test, namely conditional mutual information. This aspect grants it wide applicability in data sciences.

A NOVEL FEATURE SELECTION ALGORITHM WITH SUPERVISED MUTUAL INFORMATION FOR CLASSIFICATION

2013 ◽  
Vol 22 (04) ◽  
pp. 1350027
Author(s):  
JAGANATHAN PALANICHAMY ◽  
KUPPUCHAMY RAMASAMY
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Feature Selection ◽  
Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class A ◽  
Selection Algorithms ◽  
The Relationship ◽  
Class Variable

Feature selection is essential in data mining and pattern recognition, especially for database classification. During past years, several feature selection algorithms have been proposed to measure the relevance of various features to each class. A suitable feature selection algorithm normally maximizes the relevancy and minimizes the redundancy of the selected features. The mutual information measure can successfully estimate the dependency of features on the entire sampling space, but it cannot exactly represent the redundancies among features. In this paper, a novel feature selection algorithm is proposed based on maximum relevance and minimum redundancy criterion. The mutual information is used to measure the relevancy of each feature with class variable and calculate the redundancy by utilizing the relationship between candidate features, selected features and class variables. The effectiveness is tested with ten benchmarked datasets available in UCI Machine Learning Repository. The experimental results show better performance when compared with some existing algorithms.

Sign in / Sign up

Export Citation Format

Share Document