Better Short than Greedy: Interpretable Models through Optimal Rule Boosting

Decision trees are examples of easily interpretable models whose predictive accuracy is normally low. In comparison, decision tree ensembles (DTEs) such as random forest (RF) exhibit high predictive accuracy while being regarded as black-box models. We propose three new rule extraction algorithms from DTEs. The RF[Formula: see text]DHC method, a hill climbing method with downhill moves (DHC), is used to search for a rule set that decreases the number of rules dramatically. In the RF[Formula: see text]SGL and RF[Formula: see text]MSGL methods, the sparse group lasso (SGL) method, and the multiclass SGL (MSGL) method are employed respectively to find a sparse weight vector corresponding to the rules generated by RF. Experimental results with 24 data sets show that the proposed methods outperform similar state-of-the-art methods, in terms of human comprehensibility, by greatly reducing the number of rules and limiting the number of antecedents in the retained rules, while preserving the same level of accuracy.

Download Full-text

Selecting a sequence of last successes in independent trials

Journal of Applied Probability ◽

10.1239/jap/1014842544 ◽

2000 ◽

Vol 37 (2) ◽

pp. 389-399 ◽

Cited By ~ 13

Author(s):

F. Thomas Bruss ◽

Davy Paindaveine

Keyword(s):

Approximate Solution ◽

Convenient Method ◽

Probability Space ◽

Maximum Probability ◽

Optimal Rule ◽

Unknown Number ◽

Multiple Stopping ◽

Indicator Functions ◽

Independent Indicator ◽

Independent Trials

Let I1,I2,…,In be a sequence of independent indicator functions defined on a probability space (Ω, A, P). We say that index k is a success time if Ik = 1. The sequence I1,I2,…,In is observed sequentially. The objective of this article is to predict the lth last success, if any, with maximum probability at the time of its occurrence. We find the optimal rule and discuss briefly an algorithm to compute it in an efficient way. This generalizes the result of Bruss (1998) for l = 1, and is equivalent to the problem of (multiple) stopping with l stops on the last l successes. We then extend the model to a larger class allowing for an unknown number N of indicator functions, and present, in particular, a convenient method for an approximate solution if the success probabilities are small. We also discuss some applications of the results.

Download Full-text

Detecting and Monitoring Brain Disorders Using Smartphones and Machine Learning

10.1101/2020.10.03.20206235 ◽

2020 ◽

Cited By ~ 2

Author(s):

Rich Colbaugh ◽

Kristin Glass ◽

Volv Global

Keyword(s):

Machine Learning ◽

Neurological Diseases ◽

Disease Status ◽

Sensor Data ◽

Care Seeking ◽

Brain Disorders ◽

Real Patient ◽

Data Collection And Analysis ◽

Interpretable Models ◽

Undiagnosed Diseases

AbstractThe ubiquity of smartphones in modern life suggests the possibility to use them to continuously monitor patients, for instance to detect undiagnosed diseases or track treatment progress. Such data collection and analysis may be especially beneficial to patients with i.) mental disorders, as these individuals can experience intermittent symptoms and impaired decision-making, which may impede diagnosis and care-seeking, and ii.) progressive neurological diseases, as real-time monitoring could facilitate earlier diagnosis and more effective treatment. This paper presents a new method of leveraging passively-collected smartphone data and machine learning to detect and monitor brain disorders such as depression and Parkinson’s disease. Crucially, the algorithm is able learn accurate, interpretable models from small numbers of labeled examples (i.e., smartphone users for whom sensor data has been gathered and disease status has been determined). Predictive modeling is achieved by learning from both real patient data and ‘synthetic’ patients constructed via adversarial learning. The proposed approach is shown to outperform state-of-the-art techniques in experiments involving disparate brain disorders and multiple patient datasets.

Download Full-text

Automated, predictive, and interpretable inference of C. elegans escape dynamics

10.1101/426387 ◽

2018 ◽

Author(s):

Bryan C. Daniels ◽

William S. Ryu ◽

Ilya Nemenman

Keyword(s):

Artificial Intelligence ◽

Escape Response ◽

Escape Behavior ◽

Dynamical Variable ◽

Motor Systems ◽

Natural Systems ◽

C Elegans ◽

Escape Dynamics ◽

Functional Logic ◽

Interpretable Models

AbstractThe roundworm C. elegans exhibits robust escape behavior in response to rapidly rising temperature. The behavior lasts for a few seconds, shows history dependence, involves both sensory and motor systems, and is too complicated to model mechanistically using currently available knowledge. Instead we model the process phenomenologically, and we use the Sir Isaac dynamical inference platform to infer the model in a fully automated fashion directly from experimental data. The inferred model requires incorporation of an unobserved dynamical variable, and is biologically interpretable. The model makes accurate predictions about the dynamics of the worm behavior, and it can be used to characterize the functional logic of the dynamical system underlying the escape response. This work illustrates the power of modern artificial intelligence to aid in discovery of accurate and interpretable models of complex natural systems.

Download Full-text

Flexible Bayesian Nonlinear Model Configuration

Journal of Artificial Intelligence Research ◽

10.1613/jair.1.13047 ◽

2021 ◽

Vol 72 ◽

pp. 901-942

Author(s):

Aliaksandr Hubin ◽

Geir Storvik ◽

Florian Frommlet

Keyword(s):

Regression Models ◽

Nonlinear Models ◽

Model Averaging ◽

Predictive Performance ◽

Machine Learning Algorithms ◽

Parametric Models ◽

Monte Carlo Algorithm ◽

Wide Range ◽

Nonlinear Features ◽

Interpretable Models

Regression models are used in a wide range of applications providing a powerful scientific tool for researchers from different fields. Linear, or simple parametric, models are often not sufficient to describe complex relationships between input variables and a response. Such relationships can be better described through flexible approaches such as neural networks, but this results in less interpretable models and potential overfitting. Alternatively, specific parametric nonlinear functions can be used, but the specification of such functions is in general complicated. In this paper, we introduce a flexible approach for the construction and selection of highly flexible nonlinear parametric regression models. Nonlinear features are generated hierarchically, similarly to deep learning, but have additional flexibility on the possible types of features to be considered. This flexibility, combined with variable selection, allows us to find a small set of important features and thereby more interpretable models. Within the space of possible functions, a Bayesian approach, introducing priors for functions based on their complexity, is considered. A genetically modi ed mode jumping Markov chain Monte Carlo algorithm is adopted to perform Bayesian inference and estimate posterior probabilities for model averaging. In various applications, we illustrate how our approach is used to obtain meaningful nonlinear models. Additionally, we compare its predictive performance with several machine learning algorithms.

Download Full-text

PyDL8.5: a Library for Learning Optimal Decision Trees

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/750 ◽

2020 ◽

Author(s):

Gaël Aglin ◽

Siegfried Nijssen ◽

Pierre Schaus

Keyword(s):

Machine Learning ◽

Decision Trees ◽

Efficient Algorithm ◽

Optimal Decision ◽

Learning Tasks ◽

Explainable Ai ◽

Classification Tasks ◽

Interpretable Models ◽

Limited Depth

Decision Trees (DTs) are widely used Machine Learning (ML) models with a broad range of applications. The interest in these models has increased even further in the context of Explainable AI (XAI), as decision trees of limited depth are very interpretable models. However, traditional algorithms for learning DTs are heuristic in nature; they may produce trees that are of suboptimal quality under depth constraints. We introduce PyDL8.5, a Python library to infer depth-constrained Optimal Decision Trees (ODTs). PyDL8.5 provides an interface for DL8.5, an efficient algorithm for inferring depth-constrained ODTs. The library provides an easy-to-use scikit-learn compatible interface. It cannot only be used for classification tasks, but also for regression, clustering, and other tasks. We introduce an interface that allows users to easily implement these other learning tasks. We provide a number of examples of how to use this library.

Download Full-text