scholarly journals Optimization of Decision Trees with Hypotheses for Knowledge Representation

Electronics ◽  
2021 ◽  
Vol 10 (13) ◽  
pp. 1580
Author(s):  
Mohammad Azad ◽  
Igor Chikalov ◽  
Shahid Hussain ◽  
Mikhail Moshkov
Keyword(s):  
Dynamic Programming ◽  
Knowledge Representation ◽  
Decision Trees ◽  
Boolean Functions ◽  
Computer Experiments ◽  
Data Sets ◽  
Exact Learning ◽  
Equivalence Queries ◽  
Programming Algorithms

In this paper, we consider decision trees that use two types of queries: queries based on one attribute each and queries based on hypotheses about values of all attributes. Such decision trees are similar to the ones studied in exact learning, where membership and equivalence queries are allowed. We present dynamic programming algorithms for minimization of the depth and number of nodes of above decision trees and discuss results of computer experiments on various data sets and randomly generated Boolean functions. Decision trees with hypotheses generally have less complexity, i.e., they are more understandable and more suitable as a means for knowledge representation.

scholarly journals Entropy-Based Greedy Algorithm for Decision Trees Using Hypotheses

Entropy ◽  
2021 ◽  
Vol 23 (7) ◽  
pp. 808
Author(s):  
Mohammad Azad ◽  
Igor Chikalov ◽  
Shahid Hussain ◽  
Mikhail Moshkov
Keyword(s):  
Decision Trees ◽  
Greedy Algorithm ◽  
Boolean Functions ◽  
Computer Experiments ◽  
Data Sets ◽  
Exact Learning ◽  
Equivalence Queries

In this paper, we consider decision trees that use both conventional queries based on one attribute each and queries based on hypotheses of values of all attributes. Such decision trees are similar to those studied in exact learning, where membership and equivalence queries are allowed. We present greedy algorithm based on entropy for the construction of the above decision trees and discuss the results of computer experiments on various data sets and randomly generated Boolean functions.

scholarly journals Decision Rules Derived from Optimal Decision Trees with Hypotheses

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1641
Author(s):  
Mohammad Azad ◽  
Igor Chikalov ◽  
Shahid Hussain ◽  
Mikhail Moshkov ◽  
Beata Zielosko
Keyword(s):  
Decision Trees ◽  
Decision Rules ◽  
Computer Experiments ◽  
Optimal Decision ◽  
Equivalence Queries ◽  
Minimum Number ◽  
Minimum Depth ◽  
Decision Tables ◽  
Programming Algorithms ◽  
Better Than

Conventional decision trees use queries each of which is based on one attribute. In this study, we also examine decision trees that handle additional queries based on hypotheses. This kind of query is similar to the equivalence queries considered in exact learning. Earlier, we designed dynamic programming algorithms for the computation of the minimum depth and the minimum number of internal nodes in decision trees that have hypotheses. Modification of these algorithms considered in the present paper permits us to build decision trees with hypotheses that are optimal relative to the depth or relative to the number of the internal nodes. We compare the length and coverage of decision rules extracted from optimal decision trees with hypotheses and decision rules extracted from optimal conventional decision trees to choose the ones that are preferable as a tool for the representation of information. To this end, we conduct computer experiments on various decision tables from the UCI Machine Learning Repository. In addition, we also consider decision tables for randomly generated Boolean functions. The collected results show that the decision rules derived from decision trees with hypotheses in many cases are better than the rules extracted from conventional decision trees.

scholarly journals On the Depth of Decision Trees with Hypotheses

Entropy ◽  
2022 ◽  
Vol 24 (1) ◽  
pp. 116
Author(s):  
Mikhail Moshkov
Keyword(s):  
Information Systems ◽  
Decision Trees ◽  
Rough Set Theory ◽  
Test Theory ◽  
Complexity Classes ◽  
Worst Case ◽  
Exact Learning ◽  
Equivalence Queries ◽  
Problem Description ◽  
Binary Information

In this paper, based on the results of rough set theory, test theory, and exact learning, we investigate decision trees over infinite sets of binary attributes represented as infinite binary information systems. We define the notion of a problem over an information system and study three functions of the Shannon type, which characterize the dependence in the worst case of the minimum depth of a decision tree solving a problem on the number of attributes in the problem description. The considered three functions correspond to (i) decision trees using attributes, (ii) decision trees using hypotheses (an analog of equivalence queries from exact learning), and (iii) decision trees using both attributes and hypotheses. The first function has two possible types of behavior: logarithmic and linear (this result follows from more general results published by the author earlier). The second and the third functions have three possible types of behavior: constant, logarithmic, and linear (these results were published by the author earlier without proofs that are given in the present paper). Based on the obtained results, we divided the set of all infinite binary information systems into four complexity classes. In each class, the type of behavior for each of the considered three functions does not change.

Efficient Execution of Dynamic Programming Algorithms on Apache Spark

2020 ◽  
Author(s):  
Mohammad Mahdi Javanmard ◽  
Zafar Ahmad ◽  
Jaroslaw Zola ◽  
Louis-Noel Pouchet ◽  
Rezaul Chowdhury ◽  
...  
Keyword(s):  
Dynamic Programming ◽  
Apache Spark ◽  
Programming Algorithms ◽  
Efficient Execution

Rule Extraction from Decision Trees Ensembles: New Algorithms Based on Heuristic Search and Sparse Group Lasso Methods

2017 ◽  
Vol 16 (06) ◽  
pp. 1707-1727 ◽  
Author(s):  
Morteza Mashayekhi ◽  
Robin Gras
Keyword(s):  
Decision Trees ◽  
Predictive Accuracy ◽  
Weight Vector ◽  
Rule Extraction ◽  
Group Lasso ◽  
Hill Climbing ◽  
Data Sets ◽  
Sparse Group Lasso ◽  
Rule Set ◽  
Interpretable Models

Decision trees are examples of easily interpretable models whose predictive accuracy is normally low. In comparison, decision tree ensembles (DTEs) such as random forest (RF) exhibit high predictive accuracy while being regarded as black-box models. We propose three new rule extraction algorithms from DTEs. The RF[Formula: see text]DHC method, a hill climbing method with downhill moves (DHC), is used to search for a rule set that decreases the number of rules dramatically. In the RF[Formula: see text]SGL and RF[Formula: see text]MSGL methods, the sparse group lasso (SGL) method, and the multiclass SGL (MSGL) method are employed respectively to find a sparse weight vector corresponding to the rules generated by RF. Experimental results with 24 data sets show that the proposed methods outperform similar state-of-the-art methods, in terms of human comprehensibility, by greatly reducing the number of rules and limiting the number of antecedents in the retained rules, while preserving the same level of accuracy.

scholarly journals An Explainable Bayesian Decision Tree Algorithm

2021 ◽  
Vol 7 ◽  
Author(s):  
Giuseppe Nuti ◽  
Lluís Antoni Jiménez Rugama ◽  
Andreea-Ingrid Cross
Keyword(s):  
Monte Carlo ◽  
Decision Tree ◽  
Decision Trees ◽  
Medical Industry ◽  
Data Sets ◽  
Bayesian Decision ◽  
Decision Tree Algorithm ◽  
Probabilistic Framework ◽  
Classification Problems ◽  
Tree Algorithm

Bayesian Decision Trees provide a probabilistic framework that reduces the instability of Decision Trees while maintaining their explainability. While Markov Chain Monte Carlo methods are typically used to construct Bayesian Decision Trees, here we provide a deterministic Bayesian Decision Tree algorithm that eliminates the sampling and does not require a pruning step. This algorithm generates the greedy-modal tree (GMT) which is applicable to both regression and classification problems. We tested the algorithm on various benchmark classification data sets and obtained similar accuracies to other known techniques. Furthermore, we show that we can statistically analyze how was the GMT derived from the data and demonstrate this analysis with a financial example. Notably, the GMT allows for a technique that provides explainable simpler models which is often a prerequisite for applications in finance or the medical industry.

Time-Space Tradeoffs for Dynamic Programming Algorithms in Trees and Bounded Treewidth Graphs

2015 ◽  
pp. 349-360 ◽  
Author(s):  
Niranka Banerjee ◽  
Sankardeep Chakraborty ◽  
Venkatesh Raman ◽  
Sasanka Roy ◽  
Saket Saurabh
Keyword(s):  
Dynamic Programming ◽  
Bounded Treewidth ◽  
Time Space ◽  
Programming Algorithms
Sign in / Sign up

Export Citation Format

Share Document