scholarly journals An Analysis of Reduced Error Pruning

2001 ◽  
Vol 15 ◽  
pp. 163-187 ◽  
Author(s):  
T. Elomaa ◽  
M. Kaariainen
Keyword(s):  
Decision Tree ◽  
Sample Size ◽  
Decision Trees ◽  
General Analysis ◽  
Leaf Node ◽  
Top Down ◽  
Class Label ◽  
Decision Tree Learning ◽  
Pruning Algorithm ◽  
Specific Analysis

Top-down induction of decision trees has been observed to suffer from the inadequate functioning of the pruning phase. In particular, it is known that the size of the resulting tree grows linearly with the sample size, even though the accuracy of the tree does not improve. Reduced Error Pruning is an algorithm that has been used as a representative technique in attempts to explain the problems of decision tree learning. In this paper we present analyses of Reduced Error Pruning in three different settings. First we study the basic algorithmic properties of the method, properties that hold independent of the input decision tree and pruning examples. Then we examine a situation that intuitively should lead to the subtree under consideration to be replaced by a leaf node, one in which the class label and attribute values of the pruning examples are independent of each other. This analysis is conducted under two different assumptions. The general analysis shows that the pruning probability of a node fitting pure noise is bounded by a function that decreases exponentially as the size of the tree grows. In a specific analysis we assume that the examples are distributed uniformly to the tree. This assumption lets us approximate the number of subtrees that are pruned because they do not receive any pruning examples. This paper clarifies the different variants of the Reduced Error Pruning algorithm, brings new insight to its algorithmic properties, analyses the algorithm with less imposed assumptions than before, and includes the previously overlooked empty subtrees to the analysis.

scholarly journals LEARNING HYPERPLANES THAT CAPTURES THE GEOMETRIC STRUCTURE OF CLASS REGIONS

2013 ◽  
pp. 7-12
Author(s):  
PRAMOD PATIL ◽  
ALKA LONDHE ◽  
PARAG KULKARNI
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Geometric Structure ◽  
Gini Index ◽  
Decision Tree Algorithm ◽  
Top Down ◽  
Angle Bisector ◽  
Eigen Value ◽  
Tree Algorithms ◽  
Left And Right

Most of the decision tree algorithms rely on impurity measures to evaluate the goodness of hyperplanes at each node while learning a decision tree in a top-down fashion. These impurity measures are not differentiable with relation to the hyperplane parameters. Therefore the algorithms for decision tree learning using impurity measures need to use some search techniques for finding the best hyperplane at every node. These impurity measures don’t properly capture the geometric structures of the data. In this paper a Two-Class algorithm for learning oblique decision trees is proposed. Aggravated by this, the algorithm uses a strategy, to evaluate the hyperplanes in such a way that the (linear) geometric structure in the data is taken into consideration. At each node of the decision tree, algorithm finds the clustering hyperplanes for both the classes. The clustering hyperplanes are obtained by solving the generalized Eigen-value problem. Then the data is splitted based on angle bisector and recursively learn the left and right sub-trees of the node. Since, in general, there will be two angle bisectors; one is selected which is better based on an impurity measure gini index. Thus the algorithm combines the ideas of linear tendencies in data and purity of nodes to find better decision trees. This idea leads to small decision trees and better performance.

Evolutionary Algorithms for Global Decision Tree Induction

2018 ◽  
pp. 2132-2141 ◽  
Author(s):  
Marek Kretowski ◽  
Marcin Czajkowski
Keyword(s):  
Evolutionary Algorithms ◽  
Decision Tree ◽  
Knowledge Discovery ◽  
Decision Trees ◽  
Optimal Solution ◽  
Top Down ◽  
Local Optima ◽  
Optimal Decisions ◽  
Decision Tree Induction ◽  
Regression Functions

Decision trees represent one of the main predictive techniques in knowledge discovery. This chapter describes evolutionary induced trees, which are emerging alternatives to the greedy top-down solutions. Most typical tree-based system searches only for locally optimal decisions at each node and do not guarantee the optimal solution. Application of evolutionary algorithms to the problem of decision tree induction allows searching for the structure of the tree, tests in internal nodes and regression functions in the leaves (for model trees) at the same time. As a result, such globally induced decision tree is able to avoid local optima and usually leads to better prediction than the greedy counterparts.

New classification technique: fuzzy oblique decision tree

2018 ◽  
Vol 41 (8) ◽  
pp. 2185-2195
Author(s):  
Yuliang Cai ◽  
Huaguang Zhang ◽  
Qiang He ◽  
Shaoxin Sun
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Main Idea ◽  
Fuzzy Rule ◽  
Decision Function ◽  
Fuzzy Rules ◽  
Leaf Node ◽  
Biomedical Data ◽  
Data Set ◽  
Afs Theory

Based on axiomatic fuzzy set (AFS) theory and fuzzy information entropy, a novel fuzzy oblique decision tree (FODT) algorithm is proposed in this paper. Traditional axis-parallel decision trees only consider a single feature at each non-leaf node, while oblique decision trees partition the feature space with an oblique hyperplane. By contrast, the FODT takes dynamic mining fuzzy rules as a decision function. The main idea of the FODT is to use these fuzzy rules to construct leaf nodes for each class in each layer of the tree; the samples that cannot be covered by the fuzzy rules are then put into an additional node – the only non-leaf node in this layer. Construction of the FODT consists of four major steps: (a) generation of fuzzy membership functions automatically by AFS theory according to the raw data distribution; (b) extraction of dynamically fuzzy rules in each non-leaf node by the fuzzy rule extraction algorithm (FREA); (c) construction of the FODT by the fuzzy rules obtained from step (b); and (d) determination of the optimal threshold [Formula: see text] to generate a final tree. Compared with five traditional decision trees (C4.5, LADtree (LAD), Best-first tree (BFT), SimpleCart (SC) and NBTree (NBT)) and a recently obtained fuzzy rules decision tree (FRDT) on eight UCI machine learning data sets and one biomedical data set (ALLAML), the experimental results demonstrate that the proposed algorithm outperforms the other decision trees in both classification accuracy and tree size.

scholarly journals Top-down decision tree learning as information based boosting

2003 ◽  
Vol 292 (2) ◽  
pp. 447-464 ◽  
Author(s):  
Eiji Takimoto ◽  
Akira Maruoka
Keyword(s):  
Decision Tree ◽  
Top Down ◽  
Decision Tree Learning

Evolutionary Algorithms for Global Decision Tree Induction

2019 ◽  
pp. 668-679
Author(s):  
Marek Kretowski ◽  
Marcin Czajkowski
Keyword(s):  
Evolutionary Algorithms ◽  
Decision Tree ◽  
Knowledge Discovery ◽  
Decision Trees ◽  
Optimal Solution ◽  
Top Down ◽  
Local Optima ◽  
Optimal Decisions ◽  
Decision Tree Induction ◽  
Regression Functions

Decision trees represent one of the main predictive techniques in knowledge discovery. This chapter describes evolutionary-induced trees, which are emerging alternatives to the greedy top-down solutions. Most typical tree-based systems search only for locally optimal decisions at each node and do not guarantee the optimal solution. Application of evolutionary algorithms to the problem of decision tree induction allows searching for the structure of the tree, tests in internal nodes, and regression functions in the leaves (for model trees) at the same time. As a result, such globally induced decision trees are able to avoid local optima and usually lead to better prediction than the greedy counterparts.

scholarly journals Secure training of decision trees with continuous attributes

2021 ◽  
Vol 2021 (1) ◽  
pp. 167-187
Author(s):  
Mark Abspoel ◽  
Daniel Escudero ◽  
Nikolaj Volgushev
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Large Scale ◽  
Learning Algorithm ◽  
A Priori ◽  
Secret Data ◽  
Decision Tree Learning ◽  
Starting Point ◽  
Practical Feasibility ◽  
Medical Dataset

AbstractWe apply multiparty computation (MPC) techniques to show, given a database that is secret-shared among multiple mutually distrustful parties, how the parties may obliviously construct a decision tree based on the secret data. We consider data with continuous attributes (i.e., coming from a large domain), and develop a secure version of a learning algorithm similar to the C4.5 or CART algorithms. Previous MPC-based work only focused on decision tree learning with discrete attributes (De Hoogh et al. 2014). Our starting point is to apply an existing generic MPC protocol to a standard decision tree learning algorithm, which we then optimize in several ways. We exploit the fact that even if we allow the data to have continuous values, which a priori might require fixed or floating point representations, the output of the tree learning algorithm only depends on the relative ordering of the data. By obliviously sorting the data we reduce the number of comparisons needed per node to O(N log2N) from the naive O(N2), where N is the number of training records in the dataset, thus making the algorithm feasible for larger datasets. This does however introduce a problem when duplicate values occur in the dataset, but we manage to overcome this problem with a relatively cheap subprotocol. We show a procedure to convert a sorting network into a permutation network of smaller complexity, resulting in a round complexity of O(log N) per layer in the tree. We implement our algorithm in the MP-SPDZ framework and benchmark our implementation for both passive and active three-party computation using arithmetic modulo 264. We apply our implementation to a large scale medical dataset of ≈ 290 000 rows using random forests, and thus demonstrate practical feasibility of using MPC for privacy-preserving machine learning based on decision trees for large datasets.

Decision Trees

2011 ◽  
pp. 203-208
Author(s):  
John Wang ◽  
Dajin Wang
Keyword(s):  
Decision Making ◽  
Decision Tree ◽  
Decision Theory ◽  
Decision Trees ◽  
Decision Making Process ◽  
Decision Tree Learning ◽  
Learning Methods ◽  
Business Functions

Decision trees are part of the decision theory and are excellent tools in the decision-making process. Majority of decision tree learning methods were developed within the last 30 years by scholars like Quinlan, Mitchell, and Breiman, just to name a few (Ozgulbas & Koyuncugil, 2006). There are a number of methods and sophisticated software used to graphically present decision trees. Decision trees have a great number of benefits and are widely used in many business functions as well as different industries. However there are also disagreements and various concerns as to how useful decision trees really are. As technology evolves so do decision trees. Therefore not only do many controversies arise but also solutions and new proposals to these arguments.

Automated Development of Clinical Strategies Using Multistage Decision Analysis

1986 ◽  
Vol 25 (04) ◽  
pp. 207-214 ◽  
Author(s):  
P. Glasziou
Keyword(s):  
Decision Tree ◽  
Decision Analysis ◽  
Decision Trees ◽  
Optimal Strategy ◽  
Cholestatic Jaundice ◽  
Clinical Strategies ◽  
Simultaneous Study

SummaryThe development of investigative strategies by decision analysis has been achieved by explicitly drawing the decision tree, either by hand or on computer. This paper discusses the feasibility of automatically generating and analysing decision trees from a description of the investigations and the treatment problem. The investigation of cholestatic jaundice is used to illustrate the technique.Methods to decrease the number of calculations required are presented. It is shown that this method makes practical the simultaneous study of at least half a dozen investigations. However, some new problems arise due to the possible complexity of the resulting optimal strategy. If protocol errors and delays due to testing are considered, simpler strategies become desirable. Generation and assessment of these simpler strategies are discussed with examples.

Sign in / Sign up

Export Citation Format

Share Document