Image Classification with Nonnegative Matrix Factorization Based on Spectral Projected Gradient

2015 ◽  
pp. 31-50 ◽  
Author(s):  
Rafał Zdunek ◽  
Anh Huy Phan ◽  
Andrzej Cichocki
Keyword(s):  
Image Classification ◽  
Matrix Factorization ◽  
Nonnegative Matrix Factorization ◽  
Nonnegative Matrix ◽  
Projected Gradient ◽  
Spectral Projected Gradient

scholarly journals Projected Gradient Methods for Nonnegative Matrix Factorization

2007 ◽  
Vol 19 (10) ◽  
pp. 2756-2779 ◽  
Author(s):  
Chih-Jen Lin
Keyword(s):  
Matrix Factorization ◽  
Nonnegative Matrix Factorization ◽  
Gradient Methods ◽  
Nonnegative Matrix ◽  
Theory And Practice ◽  
Bound Constraints ◽  
Projected Gradient ◽  
Matlab Code ◽  
Projected Gradient Methods ◽  
Multiplicative Update

Nonnegative matrix factorization (NMF) can be formulated as a minimization problem with bound constraints. Although bound-constrained optimization has been studied extensively in both theory and practice, so far no study has formally applied its techniques to NMF. In this letter, we propose two projected gradient methods for NMF, both of which exhibit strong optimization properties. We discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach. A simple Matlab code is also provided.

MULTILAYER NONNEGATIVE MATRIX FACTORIZATION USING PROJECTED GRADIENT APPROACHES

2007 ◽  
Vol 17 (06) ◽  
pp. 431-446 ◽  
Author(s):  
ANDRZEJ CICHOCKI ◽  
RAFAL ZDUNEK
Keyword(s):  
Blind Source Separation ◽  
Matrix Factorization ◽  
Nonnegative Matrix Factorization ◽  
Nonnegative Matrix ◽  
Low Complexity ◽  
Local Minima ◽  
Projected Gradient ◽  
Slow Convergence ◽  
Gradient Algorithms ◽  
Gradient Approach

The most popular algorithms for Nonnegative Matrix Factorization (NMF) belong to a class of multiplicative Lee-Seung algorithms which have usually relative low complexity but are characterized by slow-convergence and the risk of getting stuck to in local minima. In this paper, we present and compare the performance of additive algorithms based on three different variations of a projected gradient approach. Additionally, we discuss a novel multilayer approach to NMF algorithms combined with multi-start initializations procedure, which in general, considerably improves the performance of all the NMF algorithms. We demonstrate that this approach (the multilayer system with projected gradient algorithms) can usually give much better performance than standard multiplicative algorithms, especially, if data are ill-conditioned, badly-scaled, and/or a number of observations is only slightly greater than a number of nonnegative hidden components. Our new implementations of NMF are demonstrated with the simulations performed for Blind Source Separation (BSS) data.

scholarly journals Fast Nonnegative Matrix Factorization Algorithms Using Projected Gradient Approaches for Large-Scale Problems

2008 ◽  
Vol 2008 ◽  
pp. 1-13 ◽  
Author(s):  
Rafal Zdunek ◽  
Andrzej Cichocki
Keyword(s):  
Matrix Factorization ◽  
Large Scale ◽  
Nonnegative Matrix Factorization ◽  
High Efficiency ◽  
Nonnegative Matrix ◽  
Linear Constraints ◽  
Projected Gradient ◽  
Minimization Problems ◽  
Subspace Optimization ◽  
Factorization Algorithms

Recently, a considerable growth of interest in projected gradient (PG) methods has been observed due to their high efficiency in solving large-scale convex minimization problems subject to linear constraints. Since the minimization problems underlying nonnegative matrix factorization (NMF) of large matrices well matches this class of minimization problems, we investigate and test some recent PG methods in the context of their applicability to NMF. In particular, the paper focuses on the following modified methods: projected Landweber, Barzilai-Borwein gradient projection, projected sequential subspace optimization (PSESOP), interior-point Newton (IPN), and sequential coordinate-wise. The proposed and implemented NMF PG algorithms are compared with respect to their performance in terms of signal-to-interference ratio (SIR) and elapsed time, using a simple benchmark of mixed partially dependent nonnegative signals.

Sign in / Sign up

Export Citation Format

Share Document