scholarly journals Monte Carlo Tree Search-Based Recursive Algorithm for Feature Selection in High-Dimensional Datasets

Entropy ◽  
2020 ◽  
Vol 22 (10) ◽  
pp. 1093
Author(s):  
Muhammad Umar Chaudhry ◽  
Muhammad Yasir ◽  
Muhammad Nabeel Asghar ◽  
Jee-Hyong Lee
Keyword(s):  
Monte Carlo ◽  
Feature Selection ◽  
Classification Accuracy ◽  
Recursive Algorithm ◽  
Feature Subset Selection ◽  
Great Success ◽  
Feature Subset ◽  
Tree Search ◽  
Monte Carlo Tree Search ◽  
The Impact

The complexity and high dimensionality are the inherent concerns of big data. The role of feature selection has gained prime importance to cope with the issue by reducing dimensionality of datasets. The compromise between the maximum classification accuracy and the minimum dimensions is as yet an unsolved puzzle. Recently, Monte Carlo Tree Search (MCTS)-based techniques have been invented that have attained great success in feature selection by constructing a binary feature selection tree and efficiently focusing on the most valuable features in the features space. However, one challenging problem associated with such approaches is a tradeoff between the tree search and the number of simulations. In a limited number of simulations, the tree might not meet the sufficient depth, thus inducing biasness towards randomness in feature subset selection. In this paper, a new algorithm for feature selection is proposed where multiple feature selection trees are built iteratively in a recursive fashion. The state space of every successor feature selection tree is less than its predecessor, thus increasing the impact of tree search in selecting best features, keeping the MCTS simulations fixed. In this study, experiments are performed on 16 benchmark datasets for validation purposes. We also compare the performance with state-of-the-art methods in literature both in terms of classification accuracy and the feature selection ratio.

A New Hybrid Feature Subset Selection Framework Based on Binary Genetic Algorithm and Information Theory

2019 ◽  
Vol 18 (03) ◽  
pp. 1950020 ◽  
Author(s):  
Alok Kumar Shukla ◽  
Pradeep Singh ◽  
Manu Vardhan
Keyword(s):  
Genetic Algorithm ◽  
Feature Selection ◽  
Classification Accuracy ◽  
B Cell Lymphoma ◽  
Feature Subset Selection ◽  
Classification Model ◽  
Significant Feature ◽  
Support Vector ◽  
Feature Subset ◽  
Binary Genetic Algorithm

The explosion of the high-dimensional dataset in the scientific repository has been encouraging interdisciplinary research on data mining, pattern recognition and bioinformatics. The fundamental problem of the individual Feature Selection (FS) method is extracting informative features for classification model and to seek for the malignant disease at low computational cost. In addition, existing FS approaches overlook the fact that for a given cardinality, there can be several subsets with similar information. This paper introduces a novel hybrid FS algorithm, called Filter-Wrapper Feature Selection (FWFS) for a classification problem and also addresses the limitations of existing methods. In the proposed model, the front-end filter ranking method as Conditional Mutual Information Maximization (CMIM) selects the high ranked feature subset while the succeeding method as Binary Genetic Algorithm (BGA) accelerates the search in identifying the significant feature subsets. One of the merits of the proposed method is that, unlike an exhaustive method, it speeds up the FS procedure without lancing of classification accuracy on reduced dataset when a learning model is applied to the selected subsets of features. The efficacy of the proposed (FWFS) method is examined by Naive Bayes (NB) classifier which works as a fitness function. The effectiveness of the selected feature subset is evaluated using numerous classifiers on five biological datasets and five UCI datasets of a varied dimensionality and number of instances. The experimental results emphasize that the proposed method provides additional support to the significant reduction of the features and outperforms the existing methods. For microarray data-sets, we found the lowest classification accuracy is 61.24% on SRBCT dataset and highest accuracy is 99.32% on Diffuse large B-cell lymphoma (DLBCL). In UCI datasets, the lowest classification accuracy is 40.04% on the Lymphography using k-nearest neighbor (k-NN) and highest classification accuracy is 99.05% on the ionosphere using support vector machine (SVM).

scholarly journals A Hybrid Feature Selection Method for Improve the Accuracy of Medical Classification Process

2021 ◽  
Vol 11 (1) ◽  
pp. 50-55
Author(s):  
Maria Mohammad Yousef ◽  
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Dimensionality Reduction ◽  
Classification Accuracy ◽  
Fitness Function ◽  
Machine Learning Algorithms ◽  
Feature Subset Selection ◽  
High Dimensionality ◽  
Support Vector ◽  
Feature Subset

Generally, medical dataset classification has become one of the biggest problems in data mining research. Every database has a given number of features but it is observed that some of these features can be redundant and can be harmful as well as disrupt the process of classification and this problem is known as a high dimensionality problem. Dimensionality reduction in data preprocessing is critical for increasing the performance of machine learning algorithms. Besides the contribution of feature subset selection in dimensionality reduction gives a significant improvement in classification accuracy. In this paper, we proposed a new hybrid feature selection approach based on (GA assisted by KNN) to deal with issues of high dimensionality in biomedical data classification. The proposed method first applies the combination between GA and KNN for feature selection to find the optimal subset of features where the classification accuracy of the k-Nearest Neighbor (kNN) method is used as the fitness function for GA. After selecting the best-suggested subset of features, Support Vector Machine (SVM) are used as the classifiers. The proposed method experiments on five medical datasets of the UCI Machine Learning Repository. It is noted that the suggested technique performs admirably on these databases, achieving higher classification accuracy while using fewer features.

scholarly journals MOTiFS: Monte Carlo Tree Search Based Feature Selection

Entropy ◽  
2018 ◽  
Vol 20 (5) ◽  
pp. 385 ◽  
Author(s):  
Muhammad Chaudhry ◽  
Jee-Hyong Lee
Keyword(s):  
Monte Carlo ◽  
Feature Selection ◽  
Tree Search ◽  
Monte Carlo Tree Search

scholarly journals Sentiment Analysis Using Hybrid Feature Selection Techniques

2020 ◽  
Vol 4 (1) ◽  
pp. 29
Author(s):  
Sasan Sarbast Abdulkhaliq ◽  
Aso Mohammad Darwesh
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Feature Selection ◽  
Classification Accuracy ◽  
Feature Selection Method ◽  
Selection Method ◽  
Machine Learning Algorithms ◽  
Feature Subset Selection ◽  
Support Vector ◽  
Feature Subset

Nowadays, people from every part of the world use social media and social networks to express their feelings toward different topics and aspects. One of the trendiest social media is Twitter, which is a microblogging website that provides a platform for its users to share their views and feelings about products, services, events, etc., in public. Which makes Twitter one of the most valuable sources for collecting and analyzing data by researchers and developers to reveal people sentiment about different topics and services, such as products of commercial companies, services, well-known people such as politicians and athletes, through classifying those sentiments into positive and negative. Classification of people sentiment could be automated through using machine learning algorithms and could be enhanced through using appropriate feature selection methods. We collected most recent tweets about (Amazon, Trump, Chelsea FC, CR7) using Twitter-Application Programming Interface and assigned sentiment score using lexicon rule-based approach, then proposed a machine learning model to improve classification accuracy through using hybrid feature selection method, namely, filter-based feature selection method Chi-square (Chi-2) plus wrapper-based binary coordinate ascent (Chi-2 + BCA) to select optimal subset of features from term frequency-inverse document frequency (TF-IDF) generated features for classification through support vector machine (SVM), and Bag of words generated features for logistic regression (LR) classifiers using different n-gram ranges. After comparing the hybrid (Chi-2+BCA) method with (Chi-2) selected features, and also with the classifiers without feature subset selection, results show that the hybrid feature selection method increases classification accuracy in all cases. The maximum attained accuracy with LR is 86.55% using (1 + 2 + 3-g) range, with SVM is 85.575% using the unigram range, both in the CR7 dataset.

scholarly journals Feature Selection for Regression Based on Gamma Test Nested Monte Carlo Tree Search

Entropy ◽  
2021 ◽  
Vol 23 (10) ◽  
pp. 1331
Author(s):  
Ying Li ◽  
Guohe Li ◽  
Lingun Guo
Keyword(s):  
Monte Carlo ◽  
Feature Selection ◽  
Feature Space ◽  
Relevant Information ◽  
Tree Search ◽  
Gamma Test ◽  
Monte Carlo Tree Search ◽  
Reward Function ◽  
Original Feature

This paper investigates the Nested Monte Carlo Tree Search (NMCTS) for feature selection on regression tasks. NMCTS starts out with an empty subset and uses search results of lower nesting level simulation. Level 0 is based on random moves until the path reaches the leaf node. In order to accomplish feature selection on the regression task, the Gamma test is introduced to play the role of the reward function at the end of the simulation. The concept Vratio of the Gamma test is also combined with the original UCT-tuned1 and the design of stopping conditions in the selection and simulation phases. The proposed GNMCTS method was tested on seven numeric datasets and compared with six other feature selection methods. It shows better performance than the vanilla MCTS framework and maintains the relevant information in the original feature space. The experimental results demonstrate that GNMCTS is a robust and effective tool for feature selection. It can accomplish the task well in a reasonable computation budget.

scholarly journals Neural Architecture Search Using Deep Neural Networks and Monte Carlo Tree Search

2020 ◽  
Vol 34 (06) ◽  
pp. 9983-9991
Author(s):  
Linnan Wang ◽  
Yiyang Zhao ◽  
Yuu Jinnai ◽  
Yuandong Tian ◽  
Rodrigo Fonseca
Keyword(s):  
Neural Networks ◽  
Monte Carlo ◽  
Large Scale ◽  
Random Search ◽  
Great Success ◽  
Tree Search ◽  
Style Transfer ◽  
Monte Carlo Tree Search ◽  
Neural Architecture ◽  
Network Evaluation

Neural Architecture Search (NAS) has shown great success in automating the design of neural networks, but the prohibitive amount of computations behind current NAS methods requires further investigations in improving the sample efficiency and the network evaluation cost to get better results in a shorter time. In this paper, we present a novel scalable Monte Carlo Tree Search (MCTS) based NAS agent, named AlphaX, to tackle these two aspects. AlphaX improves the search efficiency by adaptively balancing the exploration and exploitation at the state level, and by a Meta-Deep Neural Network (DNN) to predict network accuracies for biasing the search toward a promising region. To amortize the network evaluation cost, AlphaX accelerates MCTS rollouts with a distributed design and reduces the number of epochs in evaluating a network by transfer learning, which is guided with the tree structure in MCTS. In 12 GPU days and 1000 samples, AlphaX found an architecture that reaches 97.84% top-1 accuracy on CIFAR-10, and 75.5% top-1 accuracy on ImageNet, exceeding SOTA NAS methods in both the accuracy and sampling efficiency. Particularly, we also evaluate AlphaX on NASBench-101, a large scale NAS dataset; AlphaX is 3x and 2.8x more sample efficient than Random Search and Regularized Evolution in finding the global optimum. Finally, we show the searched architecture improves a variety of vision applications from Neural Style Transfer, to Image Captioning and Object Detection.

scholarly journals Analysis of the Impact of Randomization of Search-Control Parameters in Monte-Carlo Tree Search

2021 ◽  
Vol 72 ◽  
pp. 717-757
Author(s):  
Chiara F. Sironi ◽  
Mark H. M. Winands
Keyword(s):  
Monte Carlo ◽  
Optimal Parameter ◽  
Parameter Tuning ◽  
Tree Search ◽  
Control Parameters ◽  
Monte Carlo Tree Search ◽  
The One ◽  
Parameter Values ◽  
Search Control ◽  
The Impact

Monte-Carlo Tree Search (MCTS) has been applied successfully in many domains, including games. However, its performance is not uniform on all domains, and it also depends on how parameters that control the search are set. Parameter values that are optimal for a task might be sub-optimal for another. In a domain that tackles many games with different characteristics, like general game playing (GGP), selecting appropriate parameter settings is not a trivial task. Games are unknown to the player, thus, finding optimal parameters for a given game in advance is not feasible. Previous work has looked into tuning parameter values online, while the game is being played, showing some promising results. This tuning approach looks for optimal parameter values, balancing exploitation of values that performed well so far in the search and exploration of less sampled values. Continuously changing parameter values while performing the search, combined also with exploration of multiple values, introduces some randomization in the process. In addition, previous research indicates that adding randomization to certain components of MCTS might increase the diversification of the search and improve the performance. Therefore, this article investigates the effect of randomly selecting values for MCTS search-control parameters online among predefined sets of reasonable values. For the GGP domain, this article evaluates four different online parameter randomization strategies by comparing them with other methods to set parameter values: online parameter tuning, offline parameter tuning and sub-optimal parameter choices. Results on a set of 14 heterogeneous abstract games show that randomizing parameter values before each simulation has a positive effect on the search in some of the tested games, with respect to using fixed offline-tuned parameters. Moreover, results show a clear distinction between games for which online parameter tuning works best and games for which online randomization works best. In addition, the overall performance of online parameter randomization is closer to the one of online parameter turning than the one of sub-optimal parameter values, showing that online randomization is a reasonable parameter selection strategy. When analyzing the structure of the search trees generated by agents that use the different parameters selection strategies, it is clear that randomization causes MCTS to become more explorative, which is helpful for alignment games that present many winning paths in their trees. Online parameter tuning, instead, seems more suitable for games that present narrow winning paths and many losing paths.

Sign in / Sign up

Export Citation Format

Share Document