scholarly journals An Improved Normalized Mutual Information Variable Selection Algorithm for Neural Network-Based Soft Sensors

Sensors ◽  
2019 ◽  
Vol 19 (24) ◽  
pp. 5368 ◽  
Author(s):  
Kai Sun ◽  
Pengxin Tian ◽  
Huanning Qi ◽  
Fengying Ma ◽  
Genke Yang
Keyword(s):  
Mutual Information ◽  
Variable Selection ◽  
Optimal Solution ◽  
Selection Algorithm ◽  
Soft Sensors ◽  
Model Accuracy ◽  
Normalized Mutual Information ◽  
Entropy Information ◽  
Local Optimal Solution ◽  
Selection Algorithms

In this paper, normalized mutual information feature selection (NMIFS) and tabu search (TS) are integrated to develop a new variable selection algorithm for soft sensors. NMIFS is applied to select influential variables contributing to the output variable and avoids selecting redundant variables by calculating mutual information (MI). A TS based strategy is designed to prevent NMIFS from falling into a local optimal solution. The proposed algorithm performs the variable selection by combining the entropy information and MI and validating error information of artificial neural networks (ANNs); therefore, it has advantages over previous MI-based variable selection algorithms. Several simulation datasets with different scales, correlations and noise parameters are implemented to demonstrate the performance of the proposed algorithm. A set of actual production data from a power plant is also used to check the performance of these algorithms. The experiments showed that the developed variable selection algorithm presents better model accuracy with fewer selected variables, compared with other state-of-the-art methods. The application of this algorithm to soft sensors can achieve reliable results.

A NOVEL FEATURE SELECTION ALGORITHM WITH SUPERVISED MUTUAL INFORMATION FOR CLASSIFICATION

2013 ◽  
Vol 22 (04) ◽  
pp. 1350027
Author(s):  
JAGANATHAN PALANICHAMY ◽  
KUPPUCHAMY RAMASAMY
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Feature Selection ◽  
Mutual Information ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class A ◽  
Selection Algorithms ◽  
The Relationship ◽  
Class Variable

Feature selection is essential in data mining and pattern recognition, especially for database classification. During past years, several feature selection algorithms have been proposed to measure the relevance of various features to each class. A suitable feature selection algorithm normally maximizes the relevancy and minimizes the redundancy of the selected features. The mutual information measure can successfully estimate the dependency of features on the entire sampling space, but it cannot exactly represent the redundancies among features. In this paper, a novel feature selection algorithm is proposed based on maximum relevance and minimum redundancy criterion. The mutual information is used to measure the relevancy of each feature with class variable and calculate the redundancy by utilizing the relationship between candidate features, selected features and class variables. The effectiveness is tested with ten benchmarked datasets available in UCI Machine Learning Repository. The experimental results show better performance when compared with some existing algorithms.

scholarly journals A Feature Selection Algorithm Integrating Maximum Classification Information and Minimum Interaction Feature Dependency Information

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Li Zhang
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Information Gain ◽  
Small Sample ◽  
Selection Algorithm ◽  
Feature Selection Algorithm ◽  
Class Labels ◽  
Minimum Interaction ◽  
Classification Information ◽  
Selection Algorithms

Feature selection is the key step in the analysis of high-dimensional small sample data. The core of feature selection is to analyse and quantify the correlation between features and class labels and the redundancy between features. However, most of the existing feature selection algorithms only consider the classification contribution of individual features and ignore the influence of interfeature redundancy and correlation. Therefore, this paper proposes a feature selection algorithm for nonlinear dynamic conditional relevance (NDCRFS) through the study and analysis of the existing feature selection algorithm ideas and method. Firstly, redundancy and relevance between features and between features and class labels are discriminated by mutual information, conditional mutual information, and interactive mutual information. Secondly, the selected features and candidate features are dynamically weighted utilizing information gain factors. Finally, to evaluate the performance of this feature selection algorithm, NDCRFS was validated against 6 other feature selection algorithms on three classifiers, using 12 different data sets, for variability and classification metrics between the different algorithms. The experimental results show that the NDCRFS method can improve the quality of the feature subsets and obtain better classification results.

Efficient Management of Fragmented Replica in Data Grids

2012 ◽  
Vol 4 (2) ◽  
pp. 63-70 ◽  
Author(s):  
Chao-Chin Wu ◽  
Lien-Fu Lai ◽  
Jia-Xian Lai
Keyword(s):  
Optimal Solution ◽  
Selection Algorithm ◽  
Data Grids ◽  
Server Selection ◽  
Efficient Management ◽  
Replica Selection ◽  
Retrieval Efficiency ◽  
Simulation Results ◽  
Replication Technique ◽  
Selection Algorithms

Rather than replicating a file completely, Chang et al. proposed a fragmented replication technique to cope with the problem that only partial content of the replica are required in a local application. Furthermore, they also proposed two server selection algorithms for replica retrieval. However, their algorithms do not always find an optimal solution. To address the problem, in this paper, the authors propose a replica selection algorithm to improve the fragmented replica retrieval efficiency in this paper. It is a heuristic considering not only the transmission time but also the number of available servers for each block. Simulation results show that the proposed algorithm can improve the retrieval efficiency up to 12%.

scholarly journals Feature Selection with Conditional Mutual Information Considering Feature Interaction

Symmetry ◽  
2019 ◽  
Vol 11 (7) ◽  
pp. 858 ◽  
Author(s):  
Jun Liang ◽  
Liang Hou ◽  
Zhenhua Luan ◽  
Weiping Huang
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Classification Performance ◽  
Feature Interaction ◽  
Conditional Mutual Information ◽  
Selection Algorithm ◽  
Benchmark Datasets ◽  
Feature Relevance ◽  
New Feature ◽  
Selection Algorithms

Feature interaction is a newly proposed feature relevance relationship, but the unintentional removal of interactive features can result in poor classification performance for this relationship. However, traditional feature selection algorithms mainly focus on detecting relevant and redundant features while interactive features are usually ignored. To deal with this problem, feature relevance, feature redundancy and feature interaction are redefined based on information theory. Then a new feature selection algorithm named CMIFSI (Conditional Mutual Information based Feature Selection considering Interaction) is proposed in this paper, which makes use of conditional mutual information to estimate feature redundancy and interaction, respectively. To verify the effectiveness of our algorithm, empirical experiments are conducted to compare it with other several representative feature selection algorithms. The results on both synthetic and benchmark datasets indicate that our algorithm achieves better results than other methods in most cases. Further, it highlights the necessity of dealing with feature interaction.

scholarly journals Development of a Novel Soft Sensor with Long Short-Term Memory Network and Normalized Mutual Information Feature Selection

2020 ◽  
Vol 2020 ◽  
pp. 1-11 ◽  
Author(s):  
Dongfeng Li ◽  
Zhirui Li ◽  
Kai Sun
Keyword(s):  
Feature Selection ◽  
Mutual Information ◽  
Variable Selection ◽  
Short Term Memory ◽  
Dynamic Modelling ◽  
Soft Sensor ◽  
Short Term ◽  
Term Memory ◽  
Normalized Mutual Information ◽  
Long Short Term Memory

In this paper, a novel soft sensor is developed by combining long short-term memory (LSTM) network with normalized mutual information feature selection (NMIFS). In the proposed algorithm, LSTM is designed to handle time series with high nonlinearity and dynamics of industrial processes. NMIFS is conducted to perform the input variable selection for LSTM to simplify the excessive complexity of the model. The developed soft sensor combines the excellent dynamic modelling of LSTM and precise variable selection of NMIFS. Simulations on two actual production datasets are used to demonstrate the performance of the proposed algorithm. The developed soft sensor could precisely predict the objective variables and has better performance than other methods.

Sign in / Sign up

Export Citation Format

Share Document