scholarly journals Analysis of Information-Based Nonparametric Variable Selection Criteria

Entropy ◽  
2020 ◽  
Vol 22 (9) ◽  
pp. 974
Author(s):  
Małgorzata Łazęcka ◽  
Jan Mielniczuk
Keyword(s):  
Feature Extraction ◽  
Mutual Information ◽  
Variable Selection ◽  
Selection Criteria ◽  
Selection Process ◽  
Gaussian Mixture ◽  
Conditional Mutual Information ◽  
Tree Model ◽  
Mixing Distribution ◽  
Explicit Formulae

We consider a nonparametric Generative Tree Model and discuss a problem of selecting active predictors for the response in such scenario. We investigated two popular information-based selection criteria: Conditional Infomax Feature Extraction (CIFE) and Joint Mutual information (JMI), which are both derived as approximations of Conditional Mutual Information (CMI) criterion. We show that both criteria CIFE and JMI may exhibit different behavior from CMI, resulting in different orders in which predictors are chosen in variable selection process. Explicit formulae for CMI and its two approximations in the generative tree model are obtained. As a byproduct, we establish expressions for an entropy of a multivariate gaussian mixture and its mutual information with mixing distribution.

Mutual information-based multi-output tree learning algorithm

2021 ◽  
Vol 25 (6) ◽  
pp. 1525-1545
Author(s):  
Hyun-Seok Kang ◽  
Chi-Hyuck Jun
Keyword(s):  
Mutual Information ◽  
Variable Selection ◽  
Time Complexity ◽  
Learning Algorithm ◽  
Regression Tree ◽  
Classification And Regression Tree ◽  
Tree Model ◽  
Industrial Systems ◽  
Output Dimension ◽  
Cart Algorithm

A tree model with low time complexity can support the application of artificial intelligence to industrial systems. Variable selection based tree learning algorithms are more time efficient than existing Classification and Regression Tree (CART) algorithms. To our best knowledge, there is no attempt to deal with categorical input variable in variable selection based multi-output tree learning. Also, in the case of multi-output regression tree, a conventional variable selection based algorithm is not suitable to large datasets. We propose a mutual information-based multi-output tree learning algorithm that consists of variable selection and split optimization. The proposed method discretizes each variable based on k-means into 2–4 clusters and selects the variable for splitting based on the discretized variables using mutual information. This variable selection component has relatively low time complexity and can be applied regardless of output dimension and types. The proposed split optimization component is more efficient than an exhaustive search. The performance of the proposed tree learning algorithm is similar to or better than that of a multi-output version of CART algorithm on a specific dataset. In addition, with a large dataset, the time complexity of the proposed algorithm is significantly reduced compared to a CART algorithm.

scholarly journals A Comparative Survey of Feature Extraction and Machine Learning Methods in Diverse Acoustic Environments

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1274
Author(s):  
Daniel Bonet-Solà ◽  
Rosa Ma Alsina-Pagès
Keyword(s):  
Machine Learning ◽  
Feature Extraction ◽  
Best Practice ◽  
Nearest Neighbor ◽  
Gaussian Mixture ◽  
Machine Learning Algorithms ◽  
Multimedia Retrieval ◽  
Natural Environments ◽  
K Nearest Neighbor ◽  
Acoustic Environments

Acoustic event detection and analysis has been widely developed in the last few years for its valuable application in monitoring elderly or dependant people, for surveillance issues, for multimedia retrieval, or even for biodiversity metrics in natural environments. For this purpose, sound source identification is a key issue to give a smart technological answer to all the aforementioned applications. Diverse types of sounds and variate environments, together with a number of challenges in terms of application, widen the choice of artificial intelligence algorithm proposal. This paper presents a comparative study on combining several feature extraction algorithms (Mel Frequency Cepstrum Coefficients (MFCC), Gammatone Cepstrum Coefficients (GTCC), and Narrow Band (NB)) with a group of machine learning algorithms (k-Nearest Neighbor (kNN), Neural Networks (NN), and Gaussian Mixture Model (GMM)), tested over five different acoustic environments. This work has the goal of detailing a best practice method and evaluate the reliability of this general-purpose algorithm for all the classes. Preliminary results show that most of the combinations of feature extraction and machine learning present acceptable results in most of the described corpora. Nevertheless, there is a combination that outperforms the others: the use of GTCC together with kNN, and its results are further analyzed for all the corpora.

Sign in / Sign up

Export Citation Format

Share Document