scholarly journals A Rule Extraction Technique Applied to Ensembles of Neural Networks, Random Forests, and Gradient-Boosted Trees

Algorithms ◽  
2021 ◽  
Vol 14 (12) ◽  
pp. 339
Author(s):  
Guido Bologna
Keyword(s):  
Neural Networks ◽  
Decision Trees ◽  
Random Forests ◽  
Rule Extraction ◽  
Complex Problem ◽  
Extraction Technique ◽  
Classification Problems ◽  
Learning Ensembles ◽  
Fold Cross Validation ◽  
Natural Way

In machine learning, ensembles of models based on Multi-Layer Perceptrons (MLPs) or decision trees are considered successful models. However, explaining their responses is a complex problem that requires the creation of new methods of interpretation. A natural way to explain the classifications of the models is to transform them into propositional rules. In this work, we focus on random forests and gradient-boosted trees. Specifically, these models are converted into an ensemble of interpretable MLPs from which propositional rules are produced. The rule extraction method presented here allows one to precisely locate the discriminating hyperplanes that constitute the antecedents of the rules. In experiments based on eight classification problems, we compared our rule extraction technique to “Skope-Rules” and other state-of-the-art techniques. Experiments were performed with ten-fold cross-validation trials, with propositional rules that were also generated from ensembles of interpretable MLPs. By evaluating the characteristics of the extracted rules in terms of complexity, fidelity, and accuracy, the results obtained showed that our rule extraction technique is competitive. To the best of our knowledge, this is one of the few works showing a rule extraction technique that has been applied to both ensembles of decision trees and neural networks.

A STUDY ON RULE EXTRACTION FROM SEVERAL COMBINED NEURAL NETWORKS

2001 ◽  
Vol 11 (03) ◽  
pp. 247-255 ◽  
Author(s):  
GUIDO BOLOGNA
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Decision Trees ◽  
Polynomial Time ◽  
Public Domain ◽  
Rule Extraction ◽  
Data Sets ◽  
Np Hard ◽  
The Public ◽  
A New Technique

The problem of rule extraction from neural networks is NP-hard. This work presents a new technique to extract "if-then-else" rules from ensembles of DIMLP neural networks. Rules are extracted in polynomial time with respect to the dimensionality of the problem, the number of examples, and the size of the resulting network. Further, the degree of matching between extracted rules and neural network responses is 100%. Ensembles of DIMLP networks were trained on four data sets in the public domain. Extracted rules were on average significantly more accurate than those extracted from C4.5 decision trees.

A COMPARATIVE STUDY OF DATA SAMPLING TECHNIQUES FOR CONSTRUCTING NEURAL NETWORK ENSEMBLES

2009 ◽  
Vol 19 (02) ◽  
pp. 67-89 ◽  
Author(s):  
M. A. H. AKHAND ◽  
MD. MONIRUL ISLAM ◽  
KAZUYUKI MURASE
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Decision Trees ◽  
Ensemble Methods ◽  
Sampling Techniques ◽  
Data Sampling ◽  
Classification Problems ◽  
Negative Correlation Learning ◽  
Neural Network Ensembles ◽  
Network Ensembles

Ensembles with several classifiers (such as neural networks or decision trees) are widely used to improve the generalization performance over a single classifier. Proper diversity among component classifiers is considered an important parameter for ensemble construction so that failure of one may be compensated by others. Among various approaches, data sampling, i.e., different data sets for different classifiers, is found more effective than other approaches. A number of ensemble methods have been proposed under the umbrella of data sampling in which some are constrained to neural networks or decision trees and others are commonly applicable to both types of classifiers. We studied prominent data sampling techniques for neural network ensembles, and then experimentally evaluated their effectiveness on a common test ground. Based on overlap and uncover, the relation between generalization and diversity is presented. Eight ensemble methods were tested on 30 benchmark classification problems. We found that bagging and boosting, the pioneer ensemble methods, are still better than most of the other proposed methods. However, negative correlation learning that implicitly encourages different networks to different training spaces is shown as better or at least comparable to bagging and boosting that explicitly create different training spaces.

scholarly journals COMBINING SPECTRAL AND TEXTURE FEATURES USING RANDOM FOREST ALGORITHM: EXTRACTING IMPERVIOUS SURFACE AREA IN WUHAN

2016 ◽  
Vol XLI-B7 ◽  
pp. 351-358
Author(s):  
Zhenfeng Shao ◽  
Yuan Zhang ◽  
Lei Zhang ◽  
Yang Song ◽  
Minjun Peng
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Decision Trees ◽  
Surface Area ◽  
Random Forests ◽  
Impervious Surface ◽  
Impervious Surfaces ◽  
Remote Sensing Images ◽  
Impervious Surface Area

Impervious surface area (ISA) is one of the most important indicators of urban environments. At present, based on multi-resolution remote sensing images, numerous approaches have been proposed to extract impervious surface, using statistical estimation, sub-pixel classification and spectral mixture analysis method of sub-pixel analysis. Through these methods, impervious surfaces can be effectively applied to regional-scale planning and management. However, for the large scale region, high resolution remote sensing images can provide more details, and therefore they will be more conducive to analysis environmental monitoring and urban management. Since the purpose of this study is to map impervious surfaces more effectively, three classification algorithms (random forests, decision trees, and artificial neural networks) were tested for their ability to map impervious surface. Random forests outperformed the decision trees, and artificial neural networks in precision. Combining the spectral indices and texture, random forests is applied to impervious surface extraction with a producer’s accuracy of 0.98, a user’s accuracy of 0.97, and an overall accuracy of 0.98 and a kappa coefficient of 0.97.

scholarly journals COMBINING SPECTRAL AND TEXTURE FEATURES USING RANDOM FOREST ALGORITHM: EXTRACTING IMPERVIOUS SURFACE AREA IN WUHAN

2016 ◽  
Vol XLI-B7 ◽  
pp. 351-358 ◽  
Author(s):  
Zhenfeng Shao ◽  
Yuan Zhang ◽  
Lei Zhang ◽  
Yang Song ◽  
Minjun Peng
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Decision Trees ◽  
Surface Area ◽  
Random Forests ◽  
Impervious Surface ◽  
Impervious Surfaces ◽  
Remote Sensing Images ◽  
Impervious Surface Area

Impervious surface area (ISA) is one of the most important indicators of urban environments. At present, based on multi-resolution remote sensing images, numerous approaches have been proposed to extract impervious surface, using statistical estimation, sub-pixel classification and spectral mixture analysis method of sub-pixel analysis. Through these methods, impervious surfaces can be effectively applied to regional-scale planning and management. However, for the large scale region, high resolution remote sensing images can provide more details, and therefore they will be more conducive to analysis environmental monitoring and urban management. Since the purpose of this study is to map impervious surfaces more effectively, three classification algorithms (random forests, decision trees, and artificial neural networks) were tested for their ability to map impervious surface. Random forests outperformed the decision trees, and artificial neural networks in precision. Combining the spectral indices and texture, random forests is applied to impervious surface extraction with a producer’s accuracy of 0.98, a user’s accuracy of 0.97, and an overall accuracy of 0.98 and a kappa coefficient of 0.97.

scholarly journals Predictors of progression through the cascade of care to a cure for hepatitis C patients using decision trees and random forests

2021 ◽  
pp. 104461
Author(s):  
Jasmine Ye Nakayama ◽  
Joyce Ho ◽  
Emily Cartwright ◽  
Roy Simpson ◽  
Vicki Stover Hertzberg
Keyword(s):  
Hepatitis C ◽  
Decision Trees ◽  
Random Forests ◽  
Cascade Of Care

scholarly journals Fast Motion Model of Road Vehicles with Artificial Neural Networks

Electronics ◽  
2021 ◽  
Vol 10 (8) ◽  
pp. 928
Author(s):  
Ferenc Hegedüs ◽  
Péter Gáspár ◽  
Tamás Bécsi
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Dynamic Model ◽  
Piecewise Linear ◽  
Complex Problem ◽  
Computational Effort ◽  
Road Vehicles ◽  
Vehicle Motion ◽  
Artificial Neural ◽  
Original Dynamic

Nonlinear optimization-based motion planning algorithms have been successfully used for dynamically feasible trajectory planning of road vehicles. However, the main drawback of these methods is their significant computational effort and thus high runtime, which makes real-time application a complex problem. Addressing this field, this paper proposes an algorithm for fast simulation of road vehicle motion based on artificial neural networks that can be used in optimization-based trajectory planners. The neural networks are trained with supervised learning techniques to predict the future state of the vehicle based on its current state and driving inputs. Learning data is provided for a wide variety of randomly generated driving scenarios by simulation of a dynamic vehicle model. The realistic random driving maneuvers are created on the basis of piecewise linear travel velocity and road curvature profiles that are used for the planning of public roads. The trained neural networks are then used in a feedback loop with several variables being calculated by additional numerical integration to provide all the outputs of the original dynamic model. The presented model can be capable of short-term vehicle motion simulation with sufficient precision while having a considerably faster runtime than the original dynamic model.

Sign in / Sign up

Export Citation Format

Share Document