EVOLVING DECISION TREES IN HARDWARE

2009 ◽  
Vol 18 (06) ◽  
pp. 1033-1060 ◽  
Author(s):  
RASTISLAV J. R. STRUHARIK ◽  
LADISLAV A. NOVAK
Keyword(s):  
Integrated Circuits ◽  
Decision Tree ◽  
Decision Trees ◽  
Adaptive Learning ◽  
Software Evolution ◽  
Fpga Implementation ◽  
Software Implementation ◽  
Navigation Systems ◽  
Field Programmable ◽  
Tree Inference

This paper, according to the best of our knowledge, provides the very first solution to the hardware implementation of the complete decision tree inference algorithm. Evolving decision trees in hardware is motivated by a significant improvement in the evolution time compared to the time needed for software evolution and efficient use of decision trees in various embedded applications (robotic navigation systems, image processing systems, etc.), where run-time adaptive learning is of particular interest. Several architectures for the hardware evolution of single oblique or nonlinear decision trees and ensembles comprised from oblique or nonlinear decision trees are presented. Proposed architectures are suitable for the implementation using both Field Programmable Gate Arrays (FPGA) and Application Specific Integrated Circuits (ASIC). Results of experiments obtained using 29 datasets from the standard UCI Machine Learning Repository database suggest that the FPGA implementations offer significant improvement in inference time when compared with the traditional software implementations. In the case of single decision tree evolution, FPGA implementation of H_DTS2 architecture has on average 26 times shorter inference time when compared to the software implementation, whereas FPGA implementation of H_DTE2 architecture has on average 693 times shorter inference time than the software implementation.

HARDWARE IMPLEMENTATION OF DECISION TREE ENSEMBLES

2013 ◽  
Vol 22 (05) ◽  
pp. 1350032 ◽  
Author(s):  
RASTISLAV J. R. STRUHARIK ◽  
LADISLAV A. NOVAK
Keyword(s):  
Integrated Circuits ◽  
Decision Trees ◽  
General Purpose ◽  
Software Implementation ◽  
Support Vector ◽  
Combination Rules ◽  
Field Programmable ◽  
Axis Parallel ◽  
Programmable Gate Arrays ◽  
Hardware Architectures

In this paper, several hardware architectures for the realization of ensembles of axis-parallel, oblique and nonlinear decision trees (DTs) are presented. Hardware architectures for the implementation of a number of ensemble combination rules are also presented. These architectures are universal and can be used to combine predictions from any type of classifiers, such as decision trees, artificial neural networks (ANNs) and support vector machines (SVMs). Proposed architectures are suitable for the implementation using Field Programmable Gate Arrays (FPGA) and Application Specific Integrated Circuits (ASIC). Experiment results obtained using 29 datasets from the standard UCI Machine Learning Repository database suggest that the FPGA implementations offer significant improvement in the classification time in comparison with the traditional software implementations. Greatest improvement can be achieved using the SP2-P architecture implemented on the FPGA achieving 416.53 times faster classification speed on average, compared with the software implementation. This result has been achieved on the FPGA working at 135.51 MHz on average, which is 33.21 times slower than the operating frequency of the general purpose computer on which the software implementation has been executed.

scholarly journals FPGA Implementation of Crossover Module of Genetic Algorithm

Information ◽  
2019 ◽  
Vol 10 (6) ◽  
pp. 184
Author(s):  
Narges Attarmoghaddam ◽  
Kin Fun Li ◽  
Awos Kanan
Keyword(s):  
Genetic Algorithm ◽  
Processing Speed ◽  
Travelling Salesman Problem ◽  
Fpga Implementation ◽  
Software Implementation ◽  
Crossover Operator ◽  
Travelling Salesman ◽  
Hardware Realization ◽  
Field Programmable ◽  
Simulation Results

This paper proposes a hardware realization of the crossover module in the genetic algorithm for the travelling salesman problem (TSP). In order to enhance performance, we employ a combination of pipelining and parallelization with a genetic algorithm (GA) processor to improve processing speed, as compared to software implementation. Simulation results showed that the proposed architecture is six times faster than the similar existing architecture. The presented field-programmable gate array (FPGA) implementation of PMX crossover operator is more than 400 times faster than in software.

Automated Development of Clinical Strategies Using Multistage Decision Analysis

1986 ◽  
Vol 25 (04) ◽  
pp. 207-214 ◽  
Author(s):  
P. Glasziou
Keyword(s):  
Decision Tree ◽  
Decision Analysis ◽  
Decision Trees ◽  
Optimal Strategy ◽  
Cholestatic Jaundice ◽  
Clinical Strategies ◽  
Simultaneous Study

SummaryThe development of investigative strategies by decision analysis has been achieved by explicitly drawing the decision tree, either by hand or on computer. This paper discusses the feasibility of automatically generating and analysing decision trees from a description of the investigations and the treatment problem. The investigation of cholestatic jaundice is used to illustrate the technique.Methods to decrease the number of calculations required are presented. It is shown that this method makes practical the simultaneous study of at least half a dozen investigations. However, some new problems arise due to the possible complexity of the resulting optimal strategy. If protocol errors and delays due to testing are considered, simpler strategies become desirable. Generation and assessment of these simpler strategies are discussed with examples.

scholarly journals Performance Improvement of Decision Tree: A Robust Classifier Using Tabu Search Algorithm

2021 ◽  
Vol 11 (15) ◽  
pp. 6728
Author(s):  
Muhammad Asfand Hafeez ◽  
Muhammad Rashid ◽  
Hassan Tariq ◽  
Zain Ul Abideen ◽  
Saud S. Alotaibi ◽  
...  
Keyword(s):  
Machine Learning ◽  
Tabu Search ◽  
Decision Tree ◽  
Decision Trees ◽  
Search Algorithm ◽  
Learning Algorithms ◽  
Performance Comparison ◽  
Machine Learning Algorithms ◽  
Supervised Machine Learning ◽  
Tabu Search Algorithm

Classification and regression are the major applications of machine learning algorithms which are widely used to solve problems in numerous domains of engineering and computer science. Different classifiers based on the optimization of the decision tree have been proposed, however, it is still evolving over time. This paper presents a novel and robust classifier based on a decision tree and tabu search algorithms, respectively. In the aim of improving performance, our proposed algorithm constructs multiple decision trees while employing a tabu search algorithm to consistently monitor the leaf and decision nodes in the corresponding decision trees. Additionally, the used tabu search algorithm is responsible to balance the entropy of the corresponding decision trees. For training the model, we used the clinical data of COVID-19 patients to predict whether a patient is suffering. The experimental results were obtained using our proposed classifier based on the built-in sci-kit learn library in Python. The extensive analysis for the performance comparison was presented using Big O and statistical analysis for conventional supervised machine learning algorithms. Moreover, the performance comparison to optimized state-of-the-art classifiers is also presented. The achieved accuracy of 98%, the required execution time of 55.6 ms and the area under receiver operating characteristic (AUROC) for proposed method of 0.95 reveals that the proposed classifier algorithm is convenient for large datasets.

A Practical Tutorial for Decision Tree Induction

2021 ◽  
Vol 54 (1) ◽  
pp. 1-38
Author(s):  
Víctor Adrián Sosa Hernández ◽  
Raúl Monroy ◽  
Miguel Angel Medina-Pérez ◽  
Octavio Loyola-González ◽  
Francisco Herrera
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Machine Learning Techniques ◽  
Evaluation Measures ◽  
Decision Tree Induction ◽  
Learning Techniques ◽  
Tree Models ◽  
Evaluation Measure ◽  
Main Components ◽  
Support Decision Making

Experts from different domains have resorted to machine learning techniques to produce explainable models that support decision-making. Among existing techniques, decision trees have been useful in many application domains for classification. Decision trees can make decisions in a language that is closer to that of the experts. Many researchers have attempted to create better decision tree models by improving the components of the induction algorithm. One of the main components that have been studied and improved is the evaluation measure for candidate splits. In this article, we introduce a tutorial that explains decision tree induction. Then, we present an experimental framework to assess the performance of 21 evaluation measures that produce different C4.5 variants considering 110 databases, two performance measures, and 10× 10-fold cross-validation. Furthermore, we compare and rank the evaluation measures by using a Bayesian statistical analysis. From our experimental results, we present the first two performance rankings in the literature of C4.5 variants. Moreover, we organize the evaluation measures into two groups according to their performance. Finally, we introduce meta-models that automatically determine the group of evaluation measures to produce a C4.5 variant for a new database and some further opportunities for decision tree models.

scholarly journals Evolutionary Algorithm for Improving Decision Tree with Global Discretization in Manufacturing

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2849
Author(s):  
Sungbum Jun
Keyword(s):  
Decision Tree ◽  
Evolutionary Algorithm ◽  
Decision Trees ◽  
Manufacturing Systems ◽  
Ensemble Methods ◽  
Machine Learning Techniques ◽  
Learning Techniques ◽  
Industrial Internet ◽  
Tree Models ◽  
Real World Datasets

Due to the recent advance in the industrial Internet of Things (IoT) in manufacturing, the vast amount of data from sensors has triggered the need for leveraging such big data for fault detection. In particular, interpretable machine learning techniques, such as tree-based algorithms, have drawn attention to the need to implement reliable manufacturing systems, and identify the root causes of faults. However, despite the high interpretability of decision trees, tree-based models make a trade-off between accuracy and interpretability. In order to improve the tree’s performance while maintaining its interpretability, an evolutionary algorithm for discretization of multiple attributes, called Decision tree Improved by Multiple sPLits with Evolutionary algorithm for Discretization (DIMPLED), is proposed. The experimental results with two real-world datasets from sensors showed that the decision tree improved by DIMPLED outperformed the performances of single-decision-tree models (C4.5 and CART) that are widely used in practice, and it proved competitive compared to the ensemble methods, which have multiple decision trees. Even though the ensemble methods could produce slightly better performances, the proposed DIMPLED has a more interpretable structure, while maintaining an appropriate performance level.

Induction of fuzzy decision trees and its refinement using gradient projected-neuro-fuzzy decision tree

2014 ◽  
Vol 6 (4) ◽  
pp. 346 ◽  
Author(s):  
Swathi Jamjala Narayanan ◽  
Rajen B. Bhatt ◽  
Ilango Paramasivam ◽  
M. Khalid ◽  
B.K. Tripathy
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Fuzzy Decision ◽  
Fuzzy Decision Tree ◽  
Neuro Fuzzy ◽  
Fuzzy Decision Trees

scholarly journals Forest Pruning Based on Branch Importance

2017 ◽  
Vol 2017 ◽  
pp. 1-11 ◽  
Author(s):  
Xiangkui Jiang ◽  
Chang-an Wu ◽  
Huaping Guo
Keyword(s):  
Decision Tree ◽  
Decision Trees ◽  
Selection Algorithm ◽  
Ensemble Size ◽  
Ensemble Pruning ◽  
Generalization Ability ◽  
Ensemble Selection ◽  
Novel Strategy

A forest is an ensemble with decision trees as members. This paper proposes a novel strategy to pruning forest to enhance ensemble generalization ability and reduce ensemble size. Unlike conventional ensemble pruning approaches, the proposed method tries to evaluate the importance of branches of trees with respect to the whole ensemble using a novel proposed metric called importance gain. The importance of a branch is designed by considering ensemble accuracy and the diversity of ensemble members, and thus the metric reasonably evaluates how much improvement of the ensemble accuracy can be achieved when a branch is pruned. Our experiments show that the proposed method can significantly reduce ensemble size and improve ensemble accuracy, no matter whether ensembles are constructed by a certain algorithm such as bagging or obtained by an ensemble selection algorithm, no matter whether each decision tree is pruned or unpruned.

Novel Approach for Battery Type Determination: A Mere Electrical Alternative

2021 ◽  
Author(s):  
İsmail Can Dikmen ◽  
Teoman Karadağ
Keyword(s):  
Integrated Circuits ◽  
Decision Tree ◽  
Statistical Significance ◽  
High Capacity ◽  
Electrical Energy ◽  
Machine Learning Algorithms ◽  
Battery Management ◽  
Separation Function ◽  
Novel Approach ◽  
Tree Algorithms

Abstract Today, the storage of electrical energy is one of the most important technical challenges. The increasing number of high capacity, high-power applications, especially electric vehicles and grid energy storage, points to the fact that we will be faced with a large amount of batteries that will need to be recycled and separated in the near future. An alternative method to the currently used methods for separating these batteries according to their chemistry is discussed in this study. This method can be applied even on integrated circuits due to its ease of implementation and low operational cost. In this respect, it is also possible to use it in multi-chemistry battery management systems to detect the chemistry of the connected battery. For the implementation of the method, the batteries are connected to two different loads alternately. In this way, current and voltage values ​​are measured for two different loads without allowing the battery to relax. The obtained data is pre-processed with a separation function developed based on statistical significance. In machine learning algorithms, artificial neural network and decision tree algorithms are trained with processed data and used to determine battery chemistry with 100% accuracy. The efficiency and ease of implementation of the decision tree algorithm in such a categorization method are presented comparatively.

Sign in / Sign up

Export Citation Format

Share Document