Learning optimal decision trees using constraint programming

Constraints ◽  
2020 ◽  
Vol 25 (3-4) ◽  
pp. 226-250
Author(s):  
Hélène Verhaeghe ◽  
Siegfried Nijssen ◽  
Gilles Pesant ◽  
Claude-Guy Quimper ◽  
Pierre Schaus
Keyword(s):  
Decision Trees ◽  
Constraint Programming ◽  
Optimal Decision

scholarly journals Learning Optimal Decision Trees using Constraint Programming (Extended Abstract)

2020 ◽  
Author(s):  
Hélène Verhaeghe ◽  
Siegfried Nijssen ◽  
Gilles Pesant ◽  
Claude-Guy Quimper ◽  
Pierre Schaus
Keyword(s):  
Decision Trees ◽  
Constraint Programming ◽  
Greedy Algorithms ◽  
Building Blocks ◽  
Optimal Decision ◽  
Programming Approach ◽  
Learning Problem ◽  
New Approach ◽  
Good Classification ◽  
Additional Constraints

Decision trees are among the most popular classification models in machine learning. Traditionally, they are learned using greedy algorithms. However, such algorithms have their disadvantages: it is difficult to limit the size of the decision trees while maintaining a good classification accuracy, and it is hard to impose additional constraints on the models that are learned. For these reasons, there has been a recent interest in exact and flexible algorithms for learning decision trees. In this paper, we introduce a new approach to learn decision trees using constraint programming. Compared to earlier approaches, we show that our approach obtains better performance, while still being sufficiently flexible to allow for the inclusion of constraints. Our approach builds on three key building blocks: (1) the use of AND/OR search, (2) the use of caching, (3) the use of the CoverSize global constraint proposed recently for the problem of itemset mining. This allows our constraint programming approach to deal in a much more efficient way with the decompositions in the learning problem.

scholarly journals Optimal Decision Trees on Simplicial Complexes

10.37236/1900 ◽  
2005 ◽  
Vol 12 (1) ◽  
Author(s):  
Jakob Jonsson
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Simplicial Complex ◽  
Elementary Theory ◽  
Simplicial Complexes ◽  
Optimal Decision ◽  
Property A ◽  
Recursive Definition ◽  
Topological Combinatorics ◽  
Definition Of

We consider topological aspects of decision trees on simplicial complexes, concentrating on how to use decision trees as a tool in topological combinatorics. By Robin Forman's discrete Morse theory, the number of evasive faces of a given dimension $i$ with respect to a decision tree on a simplicial complex is greater than or equal to the $i$th reduced Betti number (over any field) of the complex. Under certain favorable circumstances, a simplicial complex admits an "optimal" decision tree such that equality holds for each $i$; we may hence read off the homology directly from the tree. We provide a recursive definition of the class of semi-nonevasive simplicial complexes with this property. A certain generalization turns out to yield the class of semi-collapsible simplicial complexes that admit an optimal discrete Morse function in the analogous sense. In addition, we develop some elementary theory about semi-nonevasive and semi-collapsible complexes. Finally, we provide explicit optimal decision trees for several well-known simplicial complexes.

scholarly journals PyDL8.5: a Library for Learning Optimal Decision Trees

2020 ◽  
Author(s):  
Gaël Aglin ◽  
Siegfried Nijssen ◽  
Pierre Schaus
Keyword(s):  
Machine Learning ◽  
Decision Trees ◽  
Efficient Algorithm ◽  
Optimal Decision ◽  
Learning Tasks ◽  
Explainable Ai ◽  
Classification Tasks ◽  
Interpretable Models ◽  
Limited Depth

Decision Trees (DTs) are widely used Machine Learning (ML) models with a broad range of applications. The interest in these models has increased even further in the context of Explainable AI (XAI), as decision trees of limited depth are very interpretable models. However, traditional algorithms for learning DTs are heuristic in nature; they may produce trees that are of suboptimal quality under depth constraints. We introduce PyDL8.5, a Python library to infer depth-constrained Optimal Decision Trees (ODTs). PyDL8.5 provides an interface for DL8.5, an efficient algorithm for inferring depth-constrained ODTs. The library provides an easy-to-use scikit-learn compatible interface. It cannot only be used for classification tasks, but also for regression, clustering, and other tasks. We introduce an interface that allows users to easily implement these other learning tasks. We provide a number of examples of how to use this library.

Prediction of irrigation event occurrence at farm level using optimal decision trees

2019 ◽  
Vol 157 ◽  
pp. 173-180 ◽  
Author(s):  
R. González Perea ◽  
E. Camacho Poyato ◽  
P. Montesinos ◽  
J.A. Rodríguez Díaz
Keyword(s):  
Decision Trees ◽  
Irrigation Event ◽  
Optimal Decision ◽  
Farm Level ◽  
Event Occurrence

scholarly journals Experimental Study of Totally Optimal Decision Trees

2019 ◽  
Vol 165 (3-4) ◽  
pp. 245-261
Author(s):  
Abdulla Aldilaijan ◽  
Mohammad Azad ◽  
Mikhail Moshkov
Keyword(s):  
Experimental Study ◽  
Decision Trees ◽  
Optimal Decision

scholarly journals Soil Mapping Based on Globally Optimal Decision Trees and Digital Imitations of Traditional Approaches

2020 ◽  
Vol 9 (11) ◽  
pp. 664
Author(s):  
Arseniy Zhogolev ◽  
Igor Savin
Keyword(s):  
Random Forest ◽  
Decision Trees ◽  
Regional Scale ◽  
Soil Formation ◽  
Test Site ◽  
Soil Mapping ◽  
Optimal Decision ◽  
Formation Factor ◽  
Trend Prediction ◽  
Soil Geography

Most digital soil mapping (DSM) approaches aim at complete statistical model extraction. The value of the explicit rules of soil delineation formulated by soil-mapping experts is often underestimated. These rules can be used for expert testing of the notional consistency of soil maps, soil trend prediction, soil geography investigations, and other applications. We propose an approach that imitates traditional soil mapping by constructing compact globally optimal decision trees (EVTREE) for the covariates of traditionally used soil formation factor maps. We evaluated our approach by regional-scale soil mapping at a test site in the Belgorod region of Russia. The notional consistency and compactness of the decision trees created by EVTREE were found to be suitable for expert-based analysis and improvement. With a large sample set, the accuracy of the predictions was slightly lower for EVTREE (59%) than for CART (67%) and much lower than for Random Forest (87%). With smaller sample sets of 1785 and 1000 points, EVTREE produced comparable or more accurate predictions and much more accurate models of soil geography than CART or Random Forest.

Sign in / Sign up

Export Citation Format

Share Document