Learning to Diversify for Ensembling of An Amount of Classifiers

Not only common man but also intelligent machine always merge all available decisions to solve problems. However, given an amount of learned classifiers, how to select and combine diverse classifiers for machine is still a grand challenge in the literature for decades of history. In this paper, we introduce a novel approach for classifier ensemble, learning to diversify, which learns to adaptively combine classifiers by considering both accuracy and diversity. Specifically, our approach, learning to diversify via weighted kernels, performs classifier combination by optimizing a direct but simple criterion: maximizing ensemble accuracy and adaptive diversity simultaneously by minimizing a loss function. Given a measure formulation, the diversity is calculated with weighted kernels, i.e., the diversity is measured on the component classifiers’ outputs which are kernelized and weighted. Moreover, we propose an iterative training algorithm for weights optimization, where this loss function is iteratively minimized by estimating the kernel weights in conjunction with the classifier weights. Extensive experiments on a variety of classification benchmark datasets show that the proposed approach consistently outperforms state-of-the-art ensembles.

Weighted p-regular kernels for reproducing kernel Hilbert spaces and Mercer Theorem

Let [Formula: see text] be a finite measure space and consider a Banach function space [Formula: see text]. Motivated by some previous papers and current applications, we provide a general framework for representing reproducing kernel Hilbert spaces as subsets of Köthe–Bochner (vector-valued) function spaces. We analyze operator-valued kernels [Formula: see text] that define integration maps [Formula: see text] between Köthe–Bochner spaces of Hilbert-valued functions [Formula: see text] We show a reduction procedure which allows to find a factorization of the corresponding kernel operator through weighted Bochner spaces [Formula: see text] and [Formula: see text] — where [Formula: see text] — under the assumption of [Formula: see text]-concavity of [Formula: see text] Equivalently, a new kernel obtained by multiplying [Formula: see text] by scalar functions can be given in such a way that the kernel operator is defined from [Formula: see text] to [Formula: see text] in a natural way. As an application, we prove a new version of Mercer Theorem for matrix-valued weighted kernels.