Evolutionary Machine Learning: A Survey

Evolutionary Computation (EC) approaches are inspired by nature and solve optimization problems in a stochastic manner. They can offer a reliable and effective approach to address complex problems in real-world applications. EC algorithms have recently been used to improve the performance of Machine Learning (ML) models and the quality of their results. Evolutionary approaches can be used in all three parts of ML: preprocessing (e.g., feature selection and resampling), learning (e.g., parameter setting, membership functions, and neural network topology), and postprocessing (e.g., rule optimization, decision tree/support vectors pruning, and ensemble learning). This article investigates the role of EC algorithms in solving different ML challenges. We do not provide a comprehensive review of evolutionary ML approaches here; instead, we discuss how EC algorithms can contribute to ML by addressing conventional challenges of the artificial intelligence and ML communities. We look at the contributions of EC to ML in nine sub-fields: feature selection, resampling, classifiers, neural networks, reinforcement learning, clustering, association rule mining, and ensemble methods. For each category, we discuss evolutionary machine learning in terms of three aspects: problem formulation, search mechanisms, and fitness value computation. We also consider open issues and challenges that should be addressed in future work.

Ensemble Machine Learning-Based Approach for Predicting of FRP–Concrete Interfacial Bonding

Developments in fiber-reinforced polymer (FRP) composite materials have created a huge impact on civil engineering techniques. Bonding properties of FRP led to its wide usage with concrete structures for interfacial bonding. FRP materials show great promise for rehabilitation of existing infrastructure by strengthening concrete structures. Existing machine learning-based models for predicting the FRP–concrete bond strength have not attained maximum performance in evaluating the bond strength. This paper presents an ensemble machine learning approach capable of predicting the FRP–concrete interfacial bond strength. In this work, a dataset holding details of 855 single-lap shear tests on FRP–concrete interfacial bonds extracted from the literature is used to build a bond strength prediction model. Test results hold data of different material properties and geometrical parameters influencing the FRP–concrete interfacial bond. This study employs CatBoost algorithm, an improved ensemble machine learning approach used to accurately predict bond strength of FRP–concrete interface. The algorithm performance is compared with those of other ensemble methods (i.e., histogram gradient boosting algorithm, extreme gradient boosting algorithm, and random forest). The CatBoost algorithm outperforms other ensemble methods with various performance metrics (i.e., lower root mean square error (2.310), lower covariance (21.8%), lower integral absolute error (8.8%), and higher R-square (96.1%)). A comparative study is performed between the proposed model and best performing bond strength prediction models in the literature. The results show that FRP–concrete interfacial bonding can be effectively predicted using proposed ensemble method.

Optimising Energy Management in Hybrid Microgrids

This article deals with the optimization of the operation of hybrid microgrids. Both the problem of controlling the management of load sharing between the different generators and energy storage and possible solutions for the integration of the microgrid into the electricity market will be discussed. Solar and wind energy as well as hybrid storage with hydrogen, as renewable sources, will be considered, which allows management of the energy balance on different time scales. The Machine Learning method of Decision Trees, combined with ensemble methods, will also be introduced to study the optimization of microgrids. The conclusions obtained indicate that the development of suitable controllers can facilitate a competitive participation of renewable energies and the integration of microgrids in the electricity system.

Ensemble based on Accuracy and Diversity Weighting for Evolving Data Streams

Ensemble classification is an actively researched paradigm that has received much attention due to increasing real-world applications. The crucial issue of ensemble learning is to construct a pool of base classifiers with accuracy and diversity. In this paper, unlike conventional data-streams oriented ensemble methods, we propose a novel Measure via both Accuracy and Diversity (MAD) instead of one of them to supervise ensemble learning. Based on MAD, a novel online ensemble method called Accuracy and Diversity weighted Ensemble (ADE) effectively handles concept drift in data streams. ADE mainly uses the following three steps to construct a concept-drift oriented ensemble: for the current data window, 1) a new base classifier is constructed based on the current concept when drift detect, 2) MAD is used to measure the performance of ensemble members, and 3) a newly built classifier replaces the worst base classifier. If the newly constructed classifier is the worst one, the replacement has not occurred. Comparing with the state-of-art algorithms, ADE exceeds the current best-related algorithm by 2.38% in average classification accuracy. Experimental results show that the proposed method can effectively adapt to different types of drifts.

Attack-Aware IoT Network Traffic Routing Leveraging Ensemble Learning

Network Intrusion Detection Systems (NIDSs) are indispensable defensive tools against various cyberattacks. Lightweight, multipurpose, and anomaly-based detection NIDSs employ several methods to build profiles for normal and malicious behaviors. In this paper, we design, implement, and evaluate the performance of machine-learning-based NIDS in IoT networks. Specifically, we study six supervised learning methods that belong to three different classes: (1) ensemble methods, (2) neural network methods, and (3) kernel methods. To evaluate the developed NIDSs, we use the distilled-Kitsune-2018 and NSL-KDD datasets, both consisting of a contemporary real-world IoT network traffic subjected to different network attacks. Standard performance evaluation metrics from the machine-learning literature are used to evaluate the identification accuracy, error rates, and inference speed. Our empirical analysis indicates that ensemble methods provide better accuracy and lower error rates compared with neural network and kernel methods. On the other hand, neural network methods provide the highest inference speed which proves their suitability for high-bandwidth networks. We also provide a comparison with state-of-the-art solutions and show that our best results are better than any prior art by 1~20%.

Detecting Web-Based Attacks with SHAP and Tree Ensemble Machine Learning Methods

Attacks using Uniform Resource Locators (URLs) and their JavaScript (JS) code content to perpetrate malicious activities on the Internet are rampant and continuously evolving. Methods such as blocklisting, client honeypots, domain reputation inspection, and heuristic and signature-based systems are used to detect these malicious activities. Recently, machine learning approaches have been proposed; however, challenges still exist. First, blocklist systems are easily evaded by new URLs and JS code content, obfuscation, fast-flux, cloaking, and URL shortening. Second, heuristic and signature-based systems do not generalize well to zero-day attacks. Third, the Domain Name System allows cybercriminals to easily migrate their malicious servers to hide their Internet protocol addresses behind domain names. Finally, crafting fully representative features is challenging, even for domain experts. This study proposes a feature selection and classification approach for malicious JS code content using Shapley additive explanations and tree ensemble methods. The JS code features are obtained from the Abstract Syntax Tree form of the JS code, sample JS attack codes, and association rule mining. The malicious and benign JS code datasets obtained from Hynek Petrak and the Majestic Million Service were used for performance evaluation. We compared the performance of the proposed method to those of other feature selection methods in the task of malicious JS code content detection. With a recall of 0.9989, our experimental results show that the proposed approach is a better prediction model.

A Hybrid Machine Learning Framework for Predicting Students’ Performance in Virtual Learning Environment

Virtual Learning Environments (VLE), such as Moodle and Blackboard, store vast data to help identify students' performance and engagement. As a result, researchers have been focusing their efforts on assisting educational institutions in providing machine learning models to predict at-risk students and improve their performance. However, it requires an efficient approach to construct a model that can ultimately provide accurate predictions. Consequently, this study proposes a hybrid machine learning framework to predict students' performance using eight classification algorithms and three ensemble methods (Bagging, Boosting, Voting) to determine the best-performing predictive model. In addition, this study used filter-based and wrapper-based feature selection techniques to select the best features of the dataset related to students' performance. The obtained results reveal that the ensemble methods recorded higher predictive accuracy when compared to single classifiers. Furthermore, the accuracy of the models improved due to the feature selection techniques utilized in this study.

Development of a method for identification of the state of computer systems based on bagging classifiers

The subject of the research is methods and means of identifying the state of a computer system . The purpose of the article is to improve the quality of computer system state identification by developing a method based on ensemble classifiers. Task: to investigate methods for constructing bagging classifiers based on decision trees, to configure them and develop a method for identifying the state of the computer system. Methods used: artificial intelligence methods, machine learning, ensemble methods. The following results were obtained: the use of bagging classifiers based on meta-algorithms were investigated: Pasting Ensemble, Bootstrap Ensemble, Random Subspace Ensemble, Random Patches Ensemble and Random Forest methods and their accuracy were assessed to identify the state of the computer system. The research of tuning parameters of individual decision trees was carried out and their optimal values were found, including: the maximum number of features used in the construction of the tree; the minimum number of branches when building a tree; minimum number of leaves and maximum tree depth. The optimal number of trees in the ensemble has been determined. A method for identifying the state of the computer system is proposed, which differs from the known ones by the choice of the classification meta-algorithm and the selection of the optimal parameters for its adjustment. An assessment of the accuracy of the developed method for identifying the state of a computer system is carried out. The developed method is implemented in software and investigated when solving the problem of identifying the abnormal state of the computer system functioning. Conclusions. The scientific novelty of the results obtained lies in the development of a method for identifying the state of the computer system by choosing a meta-algorithm for classification and determining the optimal parameters for its configuration.