A COMPARATIVE STUDY OF DATA SAMPLING TECHNIQUES FOR CONSTRUCTING NEURAL NETWORK ENSEMBLES

2009 ◽  
Vol 19 (02) ◽  
pp. 67-89 ◽  
Author(s):  
M. A. H. AKHAND ◽  
MD. MONIRUL ISLAM ◽  
KAZUYUKI MURASE
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Decision Trees ◽  
Ensemble Methods ◽  
Sampling Techniques ◽  
Data Sampling ◽  
Classification Problems ◽  
Negative Correlation Learning ◽  
Neural Network Ensembles ◽  
Network Ensembles

Ensembles with several classifiers (such as neural networks or decision trees) are widely used to improve the generalization performance over a single classifier. Proper diversity among component classifiers is considered an important parameter for ensemble construction so that failure of one may be compensated by others. Among various approaches, data sampling, i.e., different data sets for different classifiers, is found more effective than other approaches. A number of ensemble methods have been proposed under the umbrella of data sampling in which some are constrained to neural networks or decision trees and others are commonly applicable to both types of classifiers. We studied prominent data sampling techniques for neural network ensembles, and then experimentally evaluated their effectiveness on a common test ground. Based on overlap and uncover, the relation between generalization and diversity is presented. Eight ensemble methods were tested on 30 benchmark classification problems. We found that bagging and boosting, the pioneer ensemble methods, are still better than most of the other proposed methods. However, negative correlation learning that implicitly encourages different networks to different training spaces is shown as better or at least comparable to bagging and boosting that explicitly create different training spaces.

Designing Neural Network Ensembles by Minimizing Mutual Information

2011 ◽  
pp. 1-21 ◽  
Author(s):  
Yong Liu ◽  
Xin Yao ◽  
Tetsuya Higuchi
Keyword(s):  
Neural Network ◽  
Mutual Information ◽  
Negative Correlation ◽  
Decision Boundary ◽  
Negative Correlation Learning ◽  
Response Sets ◽  
Correlation Learning ◽  
Neural Network Ensembles ◽  
Decision Boundaries ◽  
Network Ensembles

This chapter describes negative correlation learning for designing neural network ensembles. Negative correlation learning has been firstly analysed in terms of minimising mutual information on a regression task. By minimising the mutual information between variables extracted by two neural networks, they are forced to convey different information about some features of their input. Based on the decision boundaries and correct response sets, negative correlation learning has been further studied on two pattern classification problems. The purpose of examining the decision boundaries and the correct response sets is not only to illustrate the learning behavior of negative correlation learning, but also to cast light on how to design more effective neural network ensembles. The experimental results showed the decision boundary of the trained neural network ensemble by negative correlation learning is almost as good as the optimum decision boundary.

NEURAL NETWORK RULE EXTRACTION BY A NEW ENSEMBLE CONCEPT AND ITS THEORETICAL AND HISTORICAL BACKGROUND: A REVIEW

2013 ◽  
Vol 12 (04) ◽  
pp. 1340006 ◽  
Author(s):  
YOICHI HAYASHI
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Performance ◽  
Real Life ◽  
Rule Extraction ◽  
Historical Background ◽  
Neural Network Ensemble ◽  
Extraction Algorithm ◽  
Neural Network Ensembles ◽  
Network Ensembles

This paper presents theoretical and historical backgrounds related to neural network rule extraction. It also investigates approaches for neural network rule extraction by ensemble concepts. Bologna pointed out that although many authors had generated comprehensive models from individual networks, much less work had been done to explain ensembles of neural networks. This paper carefully surveyed the previous work on rule extraction from neural network ensembles since 1988. We are aware of three major research groups i.e., Bologna' group, Zhou' group and Hayashi' group. The reason of these situations is obvious. Since the structures of previous neural network ensembles were quite complicated, the research on the efficient rule extraction algorithm from neural network ensembles was few although their learning capability was extremely high. Thus, these issues make rule extraction algorithm for neural network ensemble difficult task. However, there is a practical need for new ideas for neural network ensembles in order to realize the extremely high-performance needs of various rule extraction problems in real life. This paper successively explain nature of artificial neural networks, origin of neural network rule extraction, incorporating fuzziness in neural network rule extraction, theoretical foundation of neural network rule extraction, computational complexity of neural network rule extraction, neuro-fuzzy hybridization, previous rule extraction from neural network ensembles and difficulties of previous neural network ensembles. Next, this paper address three principles of proposed neural network rule extraction: to increase recognition rates, to extract rules from neural network ensembles, and to minimize the use of computing resources. We also propose an ensemble-recursive-rule extraction (E-Re-RX) by two or three standard backpropagation to train multi-layer perceptrons (MLPs), which enabled extremely high recognition accuracy and the extraction of comprehensible rules. Furthermore, this enabled rule extraction that resulted in fewer rules than those in previously proposed methods. This paper summarizes experimental results of rule extraction using E-Re-RX by multiple standard backpropagation MLPs and provides deep discussions. The results make it possible for the output from a neural network ensemble to be in the form of rules, thus open the "black box" of trained neural networks ensembles. Finally, we provide valuable conclusions and as future work, three open questions on the E-Re-RX algorithm.

Damage Identification Using Wavelet Packet Transform and Neural Network Ensembles

2018 ◽  
Vol 18 (12) ◽  
pp. 1850148 ◽  
Author(s):  
Xiang Zhang ◽  
Renwen Chen ◽  
Qinbang Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Damage Identification ◽  
Probabilistic Neural Network ◽  
Wavelet Packet ◽  
Rotor Blades ◽  
Identification Accuracy ◽  
Vibration Responses ◽  
Neural Network Ensembles ◽  
Network Ensembles

This study presents a damage identification method that combines wavelet packet transforms (WPTs) with neural network ensembles (NNEs). The WPT is used to extract damage features, which are taken as the input vectors in the NNEs used for damage identification. An experiment was performed on a helicopter rotor blades structure to verify the proposed method. First, the vibration responses collected by different sensors are decomposed using the WPT. Second, the relative band energy of each decomposed frequency band is calculated and fused as the damage feature vectors. Third, two types of the NNEs are designed. One is based on the backward propagation neural networks (BPNNs) for detecting the damage locations and severities and the other one is based on the probabilistic neural network (PNN) to detect the damage types. Finally, the trained NNEs are employed in damage identification. From the identification outcomes, it is concluded that damage information can be extracted effectively by the WPT and the identification accuracy of the NNEs is better than that of individual neural networks (INNs).

scholarly journals Explainable Neural Network Ensembles

2021 ◽  
Author(s):  
Manomita Chakraborty ◽  
Saroj Kumar Biswas ◽  
Biswajit Purkayastha
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Performance ◽  
Decision Rules ◽  
Rule Extraction ◽  
Classification Performance ◽  
Classification Rules ◽  
Multiple Networks ◽  
Neural Network Ensembles ◽  
Network Ensembles

Abstract Neural networks are known for providing impressive classification performance, and the ensemble learning technique is further acting as a catalyst to enhance this performance by integrating multiple networks. But like neural networks, neural network ensembles are also considered as a black-box because they cannot explain their decision making process. So, despite having high classification performance, neural networks and their ensembles are not suited for some applications which require explainable decisions. However, the rule extraction technique can overcome this drawback by representing the knowledge learned by a neural network in the guise of interpretable decision rules. A rule extraction algorithm provides neural networks with the power to justify their classification responses through explainable classification rules. Several rule extraction algorithms exist to extract classification rules from neural networks, but only a few of them generates rules using neural network ensembles. So this paper proposes an algorithm named Rule Extraction using Ensemble of Neural Network Ensembles (RE-E-NNES) to demonstrate the high performance of neural network ensembles through rule extraction. RE-E-NNES extracts classification rules by ensembling several neural network ensembles. Results show the efficacy of the proposed RE-E-NNES algorithm compared to different existing rule extraction algorithms.

Modeling credit scoring using neural network ensembles

Kybernetes ◽  
2014 ◽  
Vol 43 (7) ◽  
pp. 1114-1123 ◽  
Author(s):  
Chih-Fong Tsai ◽  
Chihli Hung
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Credit Scoring ◽  
Data Sets ◽  
Content Type ◽  
Learning Techniques ◽  
Neural Network Ensembles ◽  
Hybrid Neural Networks ◽  
Network Ensembles

Purpose – Credit scoring is important for financial institutions in order to accurately predict the likelihood of business failure. Related studies have shown that machine learning techniques, such as neural networks, outperform many statistical approaches to solving this type of problem, and advanced machine learning techniques, such as classifier ensembles and hybrid classifiers, provide better prediction performance than single machine learning based classification techniques. However, it is not known which type of advanced classification technique performs better in terms of financial distress prediction. The paper aims to discuss these issues. Design/methodology/approach – This paper compares neural network ensembles and hybrid neural networks over three benchmarking credit scoring related data sets, which are Australian, German, and Japanese data sets. Findings – The experimental results show that hybrid neural networks and neural network ensembles outperform the single neural network. Although hybrid neural networks perform slightly better than neural network ensembles in terms of predication accuracy and errors with two of the data sets, there is no significant difference between the two types of prediction models. Originality/value – The originality of this paper is in comparing two types of advanced classification techniques, i.e. hybrid and ensemble learning techniques, in terms of financial distress prediction.

Sign in / Sign up

Export Citation Format

Share Document