An Incremental and Interactive Decision Tree Learning Algorithm for a Practical Diagnostic Supporting Workbench

AbstractWe present a novel decision tree-based synthesis algorithm of ranking functions for verifying program termination. Our algorithm is integrated into the workflow of CounterExample Guided Inductive Synthesis (CEGIS). CEGIS is an iterative learning model where, at each iteration, (1) a synthesizer synthesizes a candidate solution from the current examples, and (2) a validator accepts the candidate solution if it is correct, or rejects it providing counterexamples as part of the next examples. Our main novelty is in the design of a synthesizer: building on top of a usual decision tree learning algorithm, our algorithm detects cycles in a set of example transitions and uses them for refining decision trees. We have implemented the proposed method and obtained promising experimental results on existing benchmark sets of (non-)termination verification problems that require synthesis of piecewise-defined lexicographic affine ranking functions.

Download Full-text

The research of decision tree learning algorithm in technology of data mining classification

Journal of Convergence Information Technology ◽

10.4156/jcit.vol7.issue10.25 ◽

2012 ◽

Vol 7 (10) ◽

pp. 216-223 ◽

Cited By ~ 2

Author(s):

Guangxian Ji

Keyword(s):

Data Mining ◽

Decision Tree ◽

Learning Algorithm ◽

Decision Tree Learning

Download Full-text

Secure training of decision trees with continuous attributes

Proceedings on Privacy Enhancing Technologies ◽

10.2478/popets-2021-0010 ◽

2021 ◽

Vol 2021 (1) ◽

pp. 167-187

Author(s):

Mark Abspoel ◽

Daniel Escudero ◽

Nikolaj Volgushev

Keyword(s):

Decision Tree ◽

Decision Trees ◽

Large Scale ◽

Learning Algorithm ◽

A Priori ◽

Secret Data ◽

Decision Tree Learning ◽

Starting Point ◽

Practical Feasibility ◽

Medical Dataset

AbstractWe apply multiparty computation (MPC) techniques to show, given a database that is secret-shared among multiple mutually distrustful parties, how the parties may obliviously construct a decision tree based on the secret data. We consider data with continuous attributes (i.e., coming from a large domain), and develop a secure version of a learning algorithm similar to the C4.5 or CART algorithms. Previous MPC-based work only focused on decision tree learning with discrete attributes (De Hoogh et al. 2014). Our starting point is to apply an existing generic MPC protocol to a standard decision tree learning algorithm, which we then optimize in several ways. We exploit the fact that even if we allow the data to have continuous values, which a priori might require fixed or floating point representations, the output of the tree learning algorithm only depends on the relative ordering of the data. By obliviously sorting the data we reduce the number of comparisons needed per node to O(N log2N) from the naive O(N2), where N is the number of training records in the dataset, thus making the algorithm feasible for larger datasets. This does however introduce a problem when duplicate values occur in the dataset, but we manage to overcome this problem with a relatively cheap subprotocol. We show a procedure to convert a sorting network into a permutation network of smaller complexity, resulting in a round complexity of O(log N) per layer in the tree. We implement our algorithm in the MP-SPDZ framework and benchmark our implementation for both passive and active three-party computation using arithmetic modulo 264. We apply our implementation to a large scale medical dataset of ≈ 290 000 rows using random forests, and thus demonstrate practical feasibility of using MPC for privacy-preserving machine learning based on decision trees for large datasets.

Download Full-text