Optimal Decision Tree Synthesis for Efficient Neighborhood Computation

2009 ◽  
pp. 92-101 ◽  
Author(s):  
Costantino Grana ◽  
Daniele Borghesani
Keyword(s):  
Decision Tree ◽  
Optimal Decision ◽  
Tree Synthesis

scholarly journals Optimal Decision Trees on Simplicial Complexes

10.37236/1900 ◽  
2005 ◽  
Vol 12 (1) ◽  
Author(s):  
Jakob Jonsson
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Simplicial Complex ◽  
Elementary Theory ◽  
Simplicial Complexes ◽  
Optimal Decision ◽  
Property A ◽  
Recursive Definition ◽  
Topological Combinatorics ◽  
Definition Of

We consider topological aspects of decision trees on simplicial complexes, concentrating on how to use decision trees as a tool in topological combinatorics. By Robin Forman's discrete Morse theory, the number of evasive faces of a given dimension $i$ with respect to a decision tree on a simplicial complex is greater than or equal to the $i$th reduced Betti number (over any field) of the complex. Under certain favorable circumstances, a simplicial complex admits an "optimal" decision tree such that equality holds for each $i$; we may hence read off the homology directly from the tree. We provide a recursive definition of the class of semi-nonevasive simplicial complexes with this property. A certain generalization turns out to yield the class of semi-collapsible simplicial complexes that admit an optimal discrete Morse function in the analogous sense. In addition, we develop some elementary theory about semi-nonevasive and semi-collapsible complexes. Finally, we provide explicit optimal decision trees for several well-known simplicial complexes.

scholarly journals Near-Optimal Sparse Neural Trees for Supervised Learning

2021 ◽  
Author(s):  
Tanujit Chakraborty
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Decision Tree ◽  
Classification Tree ◽  
Mathematical Formulation ◽  
Machine Learning Algorithms ◽  
Optimal Decision ◽  
Feed Forward ◽  
Feed Forward Network ◽  
Tree Algorithms

Decision tree algorithms have been among the most popular algorithms for interpretable (transparent) machine learning since the early 1980s. On the other hand, deep learning methods have boosted the capacity of machine learning algorithms and are now being used for non-trivial applications in various applied domains. But training a fully-connected deep feed-forward network by gradient-descent backpropagation is slow and requires arbitrary choices regarding the number of hidden units and layers. In this paper, we propose near-optimal neural regression trees, intending to make it much faster than deep feed-forward networks and for which it is not essential to specify the number of hidden units in the hidden layers of the neural network in advance. The key idea is to construct a decision tree and then simulate the decision tree with a neural network. This work aims to build a mathematical formulation of neural trees and gain the complementary benefits of both sparse optimal decision trees and neural trees. We propose near-optimal sparse neural trees (NSNT) that is shown to be asymptotically consistent and robust in nature. Additionally, the proposed NSNT model obtain a fast rate of convergence which is near-optimal up to some logarithmic factor. We comprehensively benchmark the proposed method on a sample of 80 datasets (40 classification datasets and 40 regression datasets) from the UCI machine learning repository. We establish that the proposed method is likely to outperform the current state-of-the-art methods (random forest, XGBoost, optimal classification tree, and near-optimal nonlinear trees) for the majority of the datasets.

scholarly journals Learning Optimal Decision Trees with SAT

2018 ◽  
Author(s):  
Nina Narodytska ◽  
Alexey Ignatiev ◽  
Filipe Pereira ◽  
Joao Marques-Silva
Keyword(s):  
Machine Learning ◽  
Decision Tree ◽  
Decision Trees ◽  
Practical Interest ◽  
Training Data ◽  
Fundamental Importance ◽  
Optimal Decision ◽  
Past Work ◽  
Computational Problem ◽  
Natural Mapping

Explanations of machine learning (ML) predictions are of fundamental importance in different settings. Moreover, explanations should be succinct, to enable easy understanding by humans.  Decision trees represent an often used approach for developing explainable ML models, motivated by the natural mapping between decision tree paths and rules. Clearly, smaller trees correlate well with smaller rules, and so one  challenge is to devise solutions for computing smallest size decision trees given training data. Although simple to formulate, the computation of smallest size decision trees turns out to be an extremely challenging computational problem, for which no practical solutions are known. This paper develops a SAT-based model for computing smallest-size decision trees given training data. In sharp contrast with past work, the proposed SAT model is shown to scale for publicly available datasets of practical interest.

scholarly journals Near-optimal Sparse Neural Trees

2021 ◽  
Author(s):  
Tanujit Chakraborty ◽  
Tanmoy Chakraborty
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Decision Tree ◽  
Classification Tree ◽  
Mathematical Formulation ◽  
Machine Learning Algorithms ◽  
Optimal Decision ◽  
Feed Forward ◽  
Feed Forward Network ◽  
Tree Algorithms

Decision tree algorithms have been among the most popular algorithms for interpretable (transparent) machine learning since the early 1980s. On the other hand, deep learning methods have boosted the capacity of machine learning algorithms and are now being used for non-trivial applications in various applied domains. But training a fully-connected deep feed-forward network by gradient-descent backpropagation is slow and requires arbitrary choices regarding the number of hidden units and layers. In this paper, we propose near-optimal neural regression trees, intending to make it much faster than deep feed-forward networks and for which it is not essential to specify the number of hidden units in the hidden layers of the neural network in advance. The key idea is to construct a decision tree and then simulate the decision tree with a neural network. This work aims to build a mathematical formulation of neural trees and gain the complementary benefits of both sparse optimal decision trees and neural trees. We propose near-optimal sparse neural trees (NSNT) that is shown to be asymptotically consistent and robust in nature. Additionally, the proposed NSNT model obtain a fast rate of convergence which is near-optimal upto some logarithmic factor. We comprehensively benchmark the proposed method on a sample of 80 datasets (40 classification datasets and 40 regression datasets) from the UCI machine learning repository. We establish that the proposed method is likely to outperform the current state-of-the-art methods (random forest, XGBoost, optimal classification tree, and near-optimal nonlinear trees) for the majority of the datasets.

Sign in / Sign up

Export Citation Format

Share Document