Orthogonal Nonnegative Matrix Factorization: Multiplicative Updates on Stiefel Manifolds

Nonnegative matrix factorization (NMF) is a data analysis technique used in a great variety of applications such as text mining, image processing, hyperspectral data analysis, computational biology, and clustering. In this letter, we consider two well-known algorithms designed to solve NMF problems: the multiplicative updates of Lee and Seung and the hierarchical alternating least squares of Cichocki et al. We propose a simple way to significantly accelerate these schemes, based on a careful analysis of the computational cost needed at each iteration, while preserving their convergence properties. This acceleration technique can also be applied to other algorithms, which we illustrate on the projected gradient method of Lin. The efficiency of the accelerated algorithms is empirically demonstrated on image and text data sets and compares favorably with a state-of-the-art alternating nonnegative least squares algorithm.

Download Full-text

Adaptive Multiplicative Updates for Projective Nonnegative Matrix Factorization

Neural Information Processing - Lecture Notes in Computer Science ◽

10.1007/978-3-642-34487-9_34 ◽

2012 ◽

pp. 277-284 ◽

Cited By ~ 5

Author(s):

He Zhang ◽

Zhirong Yang ◽

Erkki Oja

Keyword(s):

Matrix Factorization ◽

Nonnegative Matrix Factorization ◽

Nonnegative Matrix ◽

Multiplicative Updates

Download Full-text

Algorithms for Nonnegative Matrix Factorization with the β-Divergence

Neural Computation ◽

10.1162/neco_a_00168 ◽

2011 ◽

Vol 23 (9) ◽

pp. 2421-2456 ◽

Cited By ~ 243

Author(s):

Cédric Févotte ◽

Jérôme Idier

Keyword(s):

Matrix Factorization ◽

Nonnegative Matrix Factorization ◽

Nonnegative Matrix ◽

Real Data ◽

Auxiliary Function ◽

Power Exponent ◽

Criterion Function ◽

Multiplicative Updates ◽

Special Cases ◽

Leibler Divergence

This letter describes algorithms for nonnegative matrix factorization (NMF) with the β-divergence (β-NMF). The β-divergence is a family of cost functions parameterized by a single shape parameter β that takes the Euclidean distance, the Kullback-Leibler divergence, and the Itakura-Saito divergence as special cases (β = 2, 1, 0 respectively). The proposed algorithms are based on a surrogate auxiliary function (a local majorization of the criterion function). We first describe a majorization-minimization algorithm that leads to multiplicative updates, which differ from standard heuristic multiplicative updates by a β-dependent power exponent. The monotonicity of the heuristic algorithm can, however, be proven for β ∈ (0, 1) using the proposed auxiliary function. Then we introduce the concept of the majorization-equalization (ME) algorithm, which produces updates that move along constant level sets of the auxiliary function and lead to larger steps than MM. Simulations on synthetic and real data illustrate the faster convergence of the ME approach. The letter also describes how the proposed algorithms can be adapted to two common variants of NMF: penalized NMF (when a penalty function of the factors is added to the criterion function) and convex NMF (when the dictionary is assumed to belong to a known subspace).

Download Full-text

On Nonnegative Matrix Factorization Algorithms for Signal-Dependent Noise with Application to Electromyography Data

Neural Computation ◽

10.1162/neco_a_00576 ◽

2014 ◽

Vol 26 (6) ◽

pp. 1128-1168 ◽

Cited By ~ 19

Author(s):

Karthik Devarajan ◽

Vincent C. K. Cheung

Keyword(s):

Language Processing ◽

Matrix Factorization ◽

Large Scale ◽

Goodness Of Fit ◽

Nonnegative Matrix Factorization ◽

Gaussian Model ◽

Nonnegative Matrix ◽

Theoretic Approach ◽

Multiplicative Updates ◽

Nonnegativity Constraints

Nonnegative matrix factorization (NMF) by the multiplicative updates algorithm is a powerful machine learning method for decomposing a high-dimensional nonnegative matrix V into two nonnegative matrices, W and H, where [Formula: see text]. It has been successfully applied in the analysis and interpretation of large-scale data arising in neuroscience, computational biology, and natural language processing, among other areas. A distinctive feature of NMF is its nonnegativity constraints that allow only additive linear combinations of the data, thus enabling it to learn parts that have distinct physical representations in reality. In this letter, we describe an information-theoretic approach to NMF for signal-dependent noise based on the generalized inverse gaussian model. Specifically, we propose three novel algorithms in this setting, each based on multiplicative updates, and prove monotonicity of updates using the EM algorithm. In addition, we develop algorithm-specific measures to evaluate their goodness of fit on data. Our methods are demonstrated using experimental data from electromyography studies, as well as simulated data in the extraction of muscle synergies, and compared with existing algorithms for signal-dependent noise.

Download Full-text