scholarly journals Explainable Neural Network Ensembles

2021 ◽  
Author(s):  
Manomita Chakraborty ◽  
Saroj Kumar Biswas ◽  
Biswajit Purkayastha
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Performance ◽  
Decision Rules ◽  
Rule Extraction ◽  
Classification Performance ◽  
Classification Rules ◽  
Multiple Networks ◽  
Neural Network Ensembles ◽  
Network Ensembles

Abstract Neural networks are known for providing impressive classification performance, and the ensemble learning technique is further acting as a catalyst to enhance this performance by integrating multiple networks. But like neural networks, neural network ensembles are also considered as a black-box because they cannot explain their decision making process. So, despite having high classification performance, neural networks and their ensembles are not suited for some applications which require explainable decisions. However, the rule extraction technique can overcome this drawback by representing the knowledge learned by a neural network in the guise of interpretable decision rules. A rule extraction algorithm provides neural networks with the power to justify their classification responses through explainable classification rules. Several rule extraction algorithms exist to extract classification rules from neural networks, but only a few of them generates rules using neural network ensembles. So this paper proposes an algorithm named Rule Extraction using Ensemble of Neural Network Ensembles (RE-E-NNES) to demonstrate the high performance of neural network ensembles through rule extraction. RE-E-NNES extracts classification rules by ensembling several neural network ensembles. Results show the efficacy of the proposed RE-E-NNES algorithm compared to different existing rule extraction algorithms.

NEURAL NETWORK RULE EXTRACTION BY A NEW ENSEMBLE CONCEPT AND ITS THEORETICAL AND HISTORICAL BACKGROUND: A REVIEW

2013 ◽  
Vol 12 (04) ◽  
pp. 1340006 ◽  
Author(s):  
YOICHI HAYASHI
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Performance ◽  
Real Life ◽  
Rule Extraction ◽  
Historical Background ◽  
Neural Network Ensemble ◽  
Extraction Algorithm ◽  
Neural Network Ensembles ◽  
Network Ensembles

This paper presents theoretical and historical backgrounds related to neural network rule extraction. It also investigates approaches for neural network rule extraction by ensemble concepts. Bologna pointed out that although many authors had generated comprehensive models from individual networks, much less work had been done to explain ensembles of neural networks. This paper carefully surveyed the previous work on rule extraction from neural network ensembles since 1988. We are aware of three major research groups i.e., Bologna' group, Zhou' group and Hayashi' group. The reason of these situations is obvious. Since the structures of previous neural network ensembles were quite complicated, the research on the efficient rule extraction algorithm from neural network ensembles was few although their learning capability was extremely high. Thus, these issues make rule extraction algorithm for neural network ensemble difficult task. However, there is a practical need for new ideas for neural network ensembles in order to realize the extremely high-performance needs of various rule extraction problems in real life. This paper successively explain nature of artificial neural networks, origin of neural network rule extraction, incorporating fuzziness in neural network rule extraction, theoretical foundation of neural network rule extraction, computational complexity of neural network rule extraction, neuro-fuzzy hybridization, previous rule extraction from neural network ensembles and difficulties of previous neural network ensembles. Next, this paper address three principles of proposed neural network rule extraction: to increase recognition rates, to extract rules from neural network ensembles, and to minimize the use of computing resources. We also propose an ensemble-recursive-rule extraction (E-Re-RX) by two or three standard backpropagation to train multi-layer perceptrons (MLPs), which enabled extremely high recognition accuracy and the extraction of comprehensible rules. Furthermore, this enabled rule extraction that resulted in fewer rules than those in previously proposed methods. This paper summarizes experimental results of rule extraction using E-Re-RX by multiple standard backpropagation MLPs and provides deep discussions. The results make it possible for the output from a neural network ensemble to be in the form of rules, thus open the "black box" of trained neural networks ensembles. Finally, we provide valuable conclusions and as future work, three open questions on the E-Re-RX algorithm.

A COMPARATIVE STUDY OF DATA SAMPLING TECHNIQUES FOR CONSTRUCTING NEURAL NETWORK ENSEMBLES

2009 ◽  
Vol 19 (02) ◽  
pp. 67-89 ◽  
Author(s):  
M. A. H. AKHAND ◽  
MD. MONIRUL ISLAM ◽  
KAZUYUKI MURASE
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Decision Trees ◽  
Ensemble Methods ◽  
Sampling Techniques ◽  
Data Sampling ◽  
Classification Problems ◽  
Negative Correlation Learning ◽  
Neural Network Ensembles ◽  
Network Ensembles

Ensembles with several classifiers (such as neural networks or decision trees) are widely used to improve the generalization performance over a single classifier. Proper diversity among component classifiers is considered an important parameter for ensemble construction so that failure of one may be compensated by others. Among various approaches, data sampling, i.e., different data sets for different classifiers, is found more effective than other approaches. A number of ensemble methods have been proposed under the umbrella of data sampling in which some are constrained to neural networks or decision trees and others are commonly applicable to both types of classifiers. We studied prominent data sampling techniques for neural network ensembles, and then experimentally evaluated their effectiveness on a common test ground. Based on overlap and uncover, the relation between generalization and diversity is presented. Eight ensemble methods were tested on 30 benchmark classification problems. We found that bagging and boosting, the pioneer ensemble methods, are still better than most of the other proposed methods. However, negative correlation learning that implicitly encourages different networks to different training spaces is shown as better or at least comparable to bagging and boosting that explicitly create different training spaces.

Damage Identification Using Wavelet Packet Transform and Neural Network Ensembles

2018 ◽  
Vol 18 (12) ◽  
pp. 1850148 ◽  
Author(s):  
Xiang Zhang ◽  
Renwen Chen ◽  
Qinbang Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Damage Identification ◽  
Probabilistic Neural Network ◽  
Wavelet Packet ◽  
Rotor Blades ◽  
Identification Accuracy ◽  
Vibration Responses ◽  
Neural Network Ensembles ◽  
Network Ensembles

This study presents a damage identification method that combines wavelet packet transforms (WPTs) with neural network ensembles (NNEs). The WPT is used to extract damage features, which are taken as the input vectors in the NNEs used for damage identification. An experiment was performed on a helicopter rotor blades structure to verify the proposed method. First, the vibration responses collected by different sensors are decomposed using the WPT. Second, the relative band energy of each decomposed frequency band is calculated and fused as the damage feature vectors. Third, two types of the NNEs are designed. One is based on the backward propagation neural networks (BPNNs) for detecting the damage locations and severities and the other one is based on the probabilistic neural network (PNN) to detect the damage types. Finally, the trained NNEs are employed in damage identification. From the identification outcomes, it is concluded that damage information can be extracted effectively by the WPT and the identification accuracy of the NNEs is better than that of individual neural networks (INNs).

Modeling credit scoring using neural network ensembles

Kybernetes ◽  
2014 ◽  
Vol 43 (7) ◽  
pp. 1114-1123 ◽  
Author(s):  
Chih-Fong Tsai ◽  
Chihli Hung
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Credit Scoring ◽  
Data Sets ◽  
Content Type ◽  
Learning Techniques ◽  
Neural Network Ensembles ◽  
Hybrid Neural Networks ◽  
Network Ensembles

Purpose – Credit scoring is important for financial institutions in order to accurately predict the likelihood of business failure. Related studies have shown that machine learning techniques, such as neural networks, outperform many statistical approaches to solving this type of problem, and advanced machine learning techniques, such as classifier ensembles and hybrid classifiers, provide better prediction performance than single machine learning based classification techniques. However, it is not known which type of advanced classification technique performs better in terms of financial distress prediction. The paper aims to discuss these issues. Design/methodology/approach – This paper compares neural network ensembles and hybrid neural networks over three benchmarking credit scoring related data sets, which are Australian, German, and Japanese data sets. Findings – The experimental results show that hybrid neural networks and neural network ensembles outperform the single neural network. Although hybrid neural networks perform slightly better than neural network ensembles in terms of predication accuracy and errors with two of the data sets, there is no significant difference between the two types of prediction models. Originality/value – The originality of this paper is in comparing two types of advanced classification techniques, i.e. hybrid and ensemble learning techniques, in terms of financial distress prediction.

scholarly journals Classifcation of events in information security systems based on neural networks

2019 ◽  
Vol 23 (1) ◽  
pp. 57-63
Author(s):  
A. A. Mikryukov ◽  
A. V. Babash ◽  
V. A. Sizov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Information Security ◽  
Integrated Approach ◽  
Neural Network Ensemble ◽  
Security Systems ◽  
Event Classification ◽  
Security Event ◽  
Neural Network Ensembles ◽  
Network Ensembles

Purpose of the research.The aim of the study is to increase the effectiveness of information security and to enhance accuracy and promptness of the classification of security events, security incidents, and threats in information security systems. To respond to this challenge, neural network technologies were suggested as a classification tool for information security systems. These technologies allow accommodating incomplete, inaccurate and unidentified raw data, as well as utilizing previously accumulated information on security issues. To address the problem more effectively, collective methods based on collective neural ensembles aligned with an advanced complex approach were implemented.Materials and methods:When solving complex classification problems, often none of the classification algorithms provides the required accuracy. In such cases, it seems reasonable to build compositions of algorithms, mutually compensating errors of individual algorithms. The study also gives an insight into the application of neural network ensemble to address security issues in the corporate information system and provides a brief review of existing approaches to the construction of neural network ensembles and methods to shape problem solving with neural networks classifiers. An advanced integrated approach is proposed to tackle problems of security event classification based on neural network ensembles (neural network committees). The approach is based on a three-step procedure. The stages of the procedure implementation are described. It is shown that the use of this approach facilitates the efficiency of solving the problem.Results:An advanced integrated approach to addressing security event classification based on neural network ensembles (neural network committees) is proposed. This approach applies adaptive reduction of neural network ensemble (selection of the best classifiers is based on the assessment of the compliance degree of the competence area of the private neural network classifier and convergence of the results of private classifiers), as well as the selection and rationale of the voting method (composition or aggregation of outputs of private classifiers). The results of numerical experiments support the effectiveness of the proposed approach.Conclusion:Collectively used artificial neural networks in the form of neural network ensembles (committees of neural networks) will provide more accurate and reliable results of security event classification in the corporate information network. Moreover, an advanced integrated approach to the construction of a neural network ensemble is proposed to facilitate effectiveness of the classification process. The approach is based on the application of the adaptive reduction procedure for the results of private classifiers and the procedure for selecting the method of aggregation of the results of private classifiers. These outcomes will enable advancement of the system control over information security incidents. Finally, the paper defines tendencies and directions of the development of collective solution methods applying neural network ensembles (committees of neural networks).

scholarly journals Deep Convolutional Neural Network Ensembles Using ECOC

IEEE Access ◽  
2021 ◽  
pp. 1-1
Author(s):  
Sara Atito Ali Ahmed ◽  
Cemre Zor ◽  
Muhammad Awais ◽  
Berrin Yanikoglu ◽  
Josef Kittler
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Deep Convolutional Neural Network ◽  
Neural Network Ensembles ◽  
Network Ensembles

scholarly journals Communication Failure Resilient Distributed Neural Network for Edge Devices

Electronics ◽  
2021 ◽  
Vol 10 (14) ◽  
pp. 1614
Author(s):  
Jonghun Jeong ◽  
Jong Sung Park ◽  
Hoeseok Yang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Performance ◽  
State Of The Art ◽  
Wearable Devices ◽  
Communication Failure ◽  
Canadian Institute ◽  
Multiple Devices ◽  
Knowledge Distillation ◽  
Partitioning Technique

Recently, the necessity to run high-performance neural networks (NN) is increasing even in resource-constrained embedded systems such as wearable devices. However, due to the high computational and memory requirements of the NN applications, it is typically infeasible to execute them on a single device. Instead, it has been proposed to run a single NN application cooperatively on top of multiple devices, a so-called distributed neural network. In the distributed neural network, workloads of a single big NN application are distributed over multiple tiny devices. While the computation overhead could effectively be alleviated by this approach, the existing distributed NN techniques, such as MoDNN, still suffer from large traffics between the devices and vulnerability to communication failures. In order to get rid of such big communication overheads, a knowledge distillation based distributed NN, called Network of Neural Networks (NoNN), was proposed, which partitions the filters in the final convolutional layer of the original NN into multiple independent subsets and derives smaller NNs out of each subset. However, NoNN also has limitations in that the partitioning result may be unbalanced and it considerably compromises the correlation between filters in the original NN, which may result in an unacceptable accuracy degradation in case of communication failure. In this paper, in order to overcome these issues, we propose to enhance the partitioning strategy of NoNN in two aspects. First, we enhance the redundancy of the filters that are used to derive multiple smaller NNs by means of averaging to increase the immunity of the distributed NN to communication failure. Second, we propose a novel partitioning technique, modified from Eigenvector-based partitioning, to preserve the correlation between filters as much as possible while keeping the consistent number of filters distributed to each device. Throughout extensive experiments with the CIFAR-100 (Canadian Institute For Advanced Research-100) dataset, it has been observed that the proposed approach maintains high inference accuracy (over 70%, 1.53× improvement over the state-of-the-art approach), on average, even when a half of eight devices in a distributed NN fail to deliver their partial inference results.

Sign in / Sign up

Export Citation Format

Share Document