Multiple Kernel Learning via Low-Rank Nonnegative Matrix Factorization for Classification of Hyperspectral Imagery

Symmetric nonnegative matrix factorization (SNMF) approximates a symmetric nonnegative matrix by the product of a nonnegative low-rank matrix and its transpose. SNMF has been successfully used in many real-world applications such as clustering. In this paper, we propose an accelerated variant of the multiplicative update (MU) algorithm of He et al. designed to solve the SNMF problem. The accelerated algorithm is derived by using the extrapolation scheme of Nesterov and a restart strategy. The extrapolation scheme plays a leading role in accelerating the MU algorithm of He et al. and the restart strategy ensures that the objective function of SNMF is monotonically decreasing. We apply the accelerated algorithm to clustering problems and symmetric nonnegative tensor factorization (SNTF). The experiment results on both synthetic and real-world data show that it is more than four times faster than the MU algorithm of He et al. and performs favorably compared to recent state-of-the-art algorithms.

Download Full-text

A hybrid algorithm for low-rank approximation of nonnegative matrix factorization

Neurocomputing ◽

10.1016/j.neucom.2019.07.059 ◽

2019 ◽

Vol 364 ◽

pp. 129-137

Author(s):

Peitao Wang ◽

Zhaoshui He ◽

Kan Xie ◽

Junbin Gao ◽

Michael Antolovich ◽

...

Keyword(s):

Matrix Factorization ◽

Hybrid Algorithm ◽

Nonnegative Matrix Factorization ◽

Nonnegative Matrix ◽

Low Rank ◽

Low Rank Approximation ◽

Rank Approximation

Download Full-text

Efficient Multiple Kernel Learning Algorithms Using Low-Rank Representation

Computational Intelligence and Neuroscience ◽

10.1155/2017/3678487 ◽

2017 ◽

Vol 2017 ◽

pp. 1-9 ◽

Cited By ~ 2

Author(s):

Wenjia Niu ◽

Kewen Xia ◽

Baokai Zu ◽

Jianchuan Bai

Keyword(s):

Recognition Accuracy ◽

Multiple Kernel Learning ◽

Low Rank ◽

Kernel Learning ◽

Support Vector ◽

Distribution Characteristics ◽

Kernel Weights ◽

Multiple Kernel ◽

Kernel Approximation ◽

Low Rank Representation

Unlike Support Vector Machine (SVM), Multiple Kernel Learning (MKL) allows datasets to be free to choose the useful kernels based on their distribution characteristics rather than a precise one. It has been shown in the literature that MKL holds superior recognition accuracy compared with SVM, however, at the expense of time consuming computations. This creates analytical and computational difficulties in solving MKL algorithms. To overcome this issue, we first develop a novel kernel approximation approach for MKL and then propose an efficient Low-Rank MKL (LR-MKL) algorithm by using the Low-Rank Representation (LRR). It is well-acknowledged that LRR can reduce dimension while retaining the data features under a global low-rank constraint. Furthermore, we redesign the binary-class MKL as the multiclass MKL based on pairwise strategy. Finally, the recognition effect and efficiency of LR-MKL are verified on the datasets Yale, ORL, LSVT, and Digit. Experimental results show that the proposed LR-MKL algorithm is an efficient kernel weights allocation method in MKL and boosts the performance of MKL largely.

Download Full-text