A Rule Extraction Technique Applied to Ensembles of Neural Networks, Random Forests, and Gradient-Boosted Trees

In machine learning, ensembles of models based on Multi-Layer Perceptrons (MLPs) or decision trees are considered successful models. However, explaining their responses is a complex problem that requires the creation of new methods of interpretation. A natural way to explain the classifications of the models is to transform them into propositional rules. In this work, we focus on random forests and gradient-boosted trees. Specifically, these models are converted into an ensemble of interpretable MLPs from which propositional rules are produced. The rule extraction method presented here allows one to precisely locate the discriminating hyperplanes that constitute the antecedents of the rules. In experiments based on eight classification problems, we compared our rule extraction technique to “Skope-Rules” and other state-of-the-art techniques. Experiments were performed with ten-fold cross-validation trials, with propositional rules that were also generated from ensembles of interpretable MLPs. By evaluating the characteristics of the extracted rules in terms of complexity, fidelity, and accuracy, the results obtained showed that our rule extraction technique is competitive. To the best of our knowledge, this is one of the few works showing a rule extraction technique that has been applied to both ensembles of decision trees and neural networks.

Random Subspace Ensembles of Fully Convolutional Network for Time Series Classification

Time series classification (TSC) task is one of the most significant topics in data mining. Among all methods for this issue, the deep-learning-based shows superior performance for its good adaption to raw series data and automatic extraction of features. However, rare eyes are kept on composing ensembles of these superior individual classifiers to achieve further breakthroughs. The existing deep learning ensembles NNE did a heavy work of combining 60 individuals but did not maximize the deserving improvement, since it merely pays attention to the diversity of individuals but ignores their accuracy. In this paper, we propose to construct an ensemble of Full Convolutional Neural Networks (FCN) by Random Subspace Method (RSM), named RSM-FCN. FCN is a simple but outstanding individual classifier and RSM is suitable for high dimensional data such as time series, but there are few instances. Thus, the combination of these strengths, RSM-FCN provides a highly cost-effective approach to yield promising results. Experiments on the UCR dataset demonstrate the effectiveness and reasonability of the proposed method.

Ensemble Surrogate Models for Fast LIB Performance Predictions

Battery Cell design and control have been widely explored through modeling and simulation. On the one hand, Doyle’s pseudo-two-dimensional (P2D) model and Single Particle Models are among the most popular electrochemical models capable of predicting battery performance and therefore guiding cell characterization. On the other hand, empirical models obtained, for example, by Machine Learning (ML) methods represent a simpler and computationally more efficient complement to electrochemical models and have been widely used for Battery Management System (BMS) control purposes. This article proposes ML-based ensemble models to be used for the estimation of the performance of an LIB cell across a wide range of input material characteristics and parameters and evaluates 1. Deep Learning ensembles for simulation convergence classification and 2. structured regressors for battery energy and power predictions. The results represent an improvement on state-of-the-art LIB surrogate models and indicate that deep ensembles represent a promising direction for battery modeling and design.

A quantitative assessment of site-level factors in influencing Chukar (Alectoris chukar) introduction outcomes

Chukar partridges (Alectoris chukar) are popular game birds that have been introduced throughout the world. Propagules of varying magnitudes have been used to try and establish populations into novel locations, though the relationship between propagule size and species establishment remains speculative. Previous qualitative studies argue that site-level factors are of importance when determining where to release Chukar. We utilized machine learning ensembles to evaluate bioclimatic and topographic data from native and naturalized regions to produce predictive species distribution models (SDMs) and evaluate the relationship between establishment and site-level factors for the conterminous United States. Predictions were then compared to a distribution map based on recorded occurrences to determine model prediction performance. SDM predictions scored an average of 88% accuracy and suitability favored states where Chukars were successfully introduced and are present. Our study shows that the use of quantitative models in evaluating environmental variables and that site-level factors are strong indicators of habitat suitability and species establishment.

Spatial molecular heterogeneity of POM during decomposition at different soil depths resolved by VNIR hyperspectral imaging

Soil organic matter is composed of fractions with different functions and reactivity. Among these, particulate organic matter (POM) is the main educt of new inputs of organic matter in soils and its chemical fate corresponds to the first stages of the SOM decomposition cascade ultimately leading to the association of organic and mineral phases. We aimed at investigating the POM molecular changes during decomposition at a sub-millimetre scale by combining direct measurements of POM elemental and molecular composition with laboratory imaging VNIR spectroscopy. For this, we set up an incubation experiment to compare the molecular composition of straw and composted green manure, materials greatly differing in their C/N ratio, during their decomposition in reconstituted topsoil or subsoil of a Luvisol, and recorded hyperspectral images at high spatial and spectral resolutions of complete soil cores at the start and end of the incubation. Hyperspectral imaging was successfully combined with machine learning ensembles to produce a precise mapping of POM alkyl/O-N alkyl ratio and C/N, revealing the spatial heterogeneity in the composition of both straw and green manure. We found that both types of organic amendment were more degraded in the reconstituted topsoil than in subsoil after the incubation. We also measured consistent trends in molecular changes undergone by straw, with the alkyl/O-N alkyl ratio slightly increasing from 0.06 to 0.07, and C/N dropping by about 40 units. The green manure material was very heterogeneous, with no clear molecular changes detected as a result of incubation. The visualisation approach presented here enables high-resolution mapping of the spatial distribution of the molecular characteristics of organic particles in soil cores, and offers opportunities to disentangle the roles of POM chemistry and morphology during the first steps of the decomposition cascade of organic matter in soils.