Abstract
Not only common man but also intelligent machine always merge all available decisions to solve problems. However, given an amount of learned classifiers, how to select and combine diverse classifiers for machine is still a grand challenge in the literature for decades of history. In this paper, we introduce a novel approach for classifier ensemble, learning to diversify, which learns to adaptively combine classifiers by considering both accuracy and diversity. Specifically, our approach, learning to diversify via weighted kernels, performs classifier combination by optimizing a direct but simple criterion: maximizing ensemble accuracy and adaptive diversity simultaneously by minimizing a loss function. Given a measure formulation, the diversity is calculated with weighted kernels, i.e., the diversity is measured on the component classifiers’ outputs which are kernelized and weighted. Moreover, we propose an iterative training algorithm for weights optimization, where this loss function is iteratively minimized by estimating the kernel weights in conjunction with the classifier weights. Extensive experiments on a variety of classification benchmark datasets show that the proposed approach consistently outperforms state-of-the-art ensembles.
Learning to Diversify for Ensembling of An Amount of Classifiers
