scholarly journals A Quantum-inspired Classical Algorithm for Separable Non-negative Matrix Factorization

2019 ◽  
Author(s):  
Zhihuai Chen ◽  
Yinan Li ◽  
Xiaoming Sun ◽  
Pei Yuan ◽  
Jialin Zhang
Keyword(s):  
Image Processing ◽  
Polynomial Time ◽  
Matrix Factorization ◽  
Text Analysis ◽  
Low Rank ◽  
Nonnegative Matrices ◽  
Classical Algorithm ◽  
Data Points ◽  
Conical Hull ◽  
Non Negative Matrix Factorization

Non-negative Matrix Factorization (NMF) asks to decompose a (entry-wise) non-negative matrix into the product of two smaller-sized nonnegative matrices, which has been shown intractable in general. In order to overcome this issue, separability assumption is introduced which assumes all data points are in a conical hull. This assumption makes NMF tractable and widely used in text analysis and image processing, but still impractical for huge-scale datasets. In this paper, inspired by recent development on dequantizing techniques, we propose a new classical algorithm for separable NMF problem. Our new algorithm runs in polynomial time in the rank and logarithmic in the size of input matrices, which achieves an exponential speedup in the low-rank setting.

Swarm Intelligence for Non-Negative Matrix Factorization

2011 ◽  
Vol 2 (4) ◽  
pp. 12-34 ◽  
Author(s):  
Andreas Janecek ◽  
Ying Tan
Keyword(s):  
Swarm Intelligence ◽  
Matrix Factorization ◽  
Classification Accuracy ◽  
Approximation Error ◽  
Synthetic Data ◽  
Low Rank ◽  
Low Rank Approximation ◽  
Approximation Quality ◽  
Initialization Strategy ◽  
Non Negative Matrix Factorization

The Non-negative Matrix Factorization (NMF) is a special low-rank approximation which allows for an additive parts-based and interpretable representation of the data. This article presents efforts to improve the convergence, approximation quality, and classification accuracy of NMF using five different meta-heuristics based on swarm intelligence. Several properties of the NMF objective function motivate the utilization of meta-heuristics: this function is non-convex, discontinuous, and may possess many local minima. The proposed optimization strategies are two-fold: On the one hand, a new initialization strategy for NMF is presented in order to initialize the NMF factors prior to the factorization; on the other hand, an iterative update strategy is proposed, which improves the accuracy per runtime for the multiplicative update NMF algorithm. The success of the proposed optimization strategies are shown by applying them on synthetic data and data sets coming from the areas of spam filtering/email classification, and evaluate them also in their application context. Experimental results show that both optimization strategies are able to improve NMF in terms of faster convergence, lower approximation error, and better classification accuracy. Especially the initialization strategy leads to significant reductions of the runtime per accuracy ratio for both, the NMF approximation as well as the classification results achieved with NMF.

scholarly journals A factor graph based genetic algorithm

2014 ◽  
Vol 24 (3) ◽  
pp. 621-633 ◽  
Author(s):  
B. Hoda Helmi ◽  
Adel T. Rahmani ◽  
Martin Pelikan
Keyword(s):  
Genetic Algorithm ◽  
Polynomial Time ◽  
Mathematical Analysis ◽  
Matrix Factorization ◽  
Optimization Problems ◽  
Factor Graph ◽  
The Matrix ◽  
Separable Problems ◽  
Linkage Learning ◽  
Non Negative Matrix Factorization

Abstract We propose a new linkage learning genetic algorithm called the Factor Graph based Genetic Algorithm (FGGA). In the FGGA, a factor graph is used to encode the underlying dependencies between variables of the problem. In order to learn the factor graph from a population of potential solutions, a symmetric non-negative matrix factorization is employed to factorize the matrix of pair-wise dependencies. To show the performance of the FGGA, encouraging experimental results on different separable problems are provided as support for the mathematical analysis of the approach. The experiments show that FGGA is capable of learning linkages and solving the optimization problems in polynomial time with a polynomial number of evaluations.

scholarly journals Probabilistic Non-Negative Matrix Factorization with Binary Components

Mathematics ◽  
2021 ◽  
Vol 9 (11) ◽  
pp. 1189
Author(s):  
Xindi Ma ◽  
Jie Gao ◽  
Xiaoyu Liu ◽  
Taiping Zhang ◽  
Yuanyan Tang
Keyword(s):  
Matrix Factorization ◽  
Fixed Number ◽  
Weight Matrix ◽  
Low Rank ◽  
Factor Matrix ◽  
Binary Features ◽  
Rank Decomposition ◽  
Real World Datasets ◽  
Low Rank Decomposition ◽  
Non Negative Matrix Factorization

Non-negative matrix factorization is used to find a basic matrix and a weight matrix to approximate the non-negative matrix. It has proven to be a powerful low-rank decomposition technique for non-negative multivariate data. However, its performance largely depends on the assumption of a fixed number of features. This work proposes a new probabilistic non-negative matrix factorization which factorizes a non-negative matrix into a low-rank factor matrix with constraints and a non-negative weight matrix. In order to automatically learn the potential binary features and feature number, a deterministic Indian buffet process variational inference is introduced to obtain the binary factor matrix. Further, the weight matrix is set to satisfy the exponential prior. To obtain the real posterior distribution of the two factor matrices, a variational Bayesian exponential Gaussian inference model is established. The comparative experiments on the synthetic and real-world datasets show the efficacy of the proposed method.

Instantaneous Versus Convolutive Non-Negative Matrix Factorization

2010 ◽  
pp. 353-370 ◽  
Author(s):  
Wenwu Wang
Keyword(s):  
Machine Learning ◽  
Performance Evaluation ◽  
Data Analysis ◽  
Matrix Factorization ◽  
Temporal Structure ◽  
Low Rank ◽  
Basis Matrix ◽  
And Performance ◽  
Separation Problems ◽  
Non Negative Matrix Factorization

Non-negative matrix factorization (NMF) is an emerging technique for data analysis and machine learning, which aims to find low-rank representations for non-negative data. Early works in NMF are mainly based on the instantaneous model, i.e. using a single basis matrix to represent the data. Recent works have shown that the instantaneous model may not be satisfactory for many audio application tasks. The convolutive NMF model, which has an advantage of revealing the temporal structure possessed by many signals, has been proposed. This chapter intends to provide a brief overview of the models and algorithms for both the instantaneous and the convolutive NMF, with a focus on the theoretical analysis and performance evaluation of the convolutive NMF algorithms, and their applications to audio pattern separation problems.

scholarly journals PROJECTIVE NON-NEGATIVE MATRIX FACTORIZATION WITH APPLICATIONS TO FACIAL IMAGE PROCESSING

2007 ◽  
Vol 21 (08) ◽  
pp. 1353-1362 ◽  
Author(s):  
ZHIRONG YANG ◽  
ZHIJIAN YUAN ◽  
JORMA LAAKSONEN
Keyword(s):  
Image Processing ◽  
Matrix Factorization ◽  
Recognition Rate ◽  
Objective Functions ◽  
Facial Image ◽  
Semantic Classes ◽  
Straight Line ◽  
New Variant ◽  
Facial Images ◽  
Non Negative Matrix Factorization

We propose a new variant of Non-negative Matrix Factorization (NMF), including its model and two optimization rules. Our method is based on positively constrained projections and is related to the conventional SVD or PCA decomposition. The new model can potentially be applied to image compression and feature extraction problems. Of the latter, we consider processing of facial images, where each image consists of several parts and for each part the observations with different lighting mainly distribute along a straight line through the origin. No regularization terms are required in the objective functions and both suggested optimization rules can easily be implemented by matrix manipulations. The experiments show that the derived base vectors are spatially more localized than those of NMF. In turn, the better part-based representations improve the recognition rate of semantic classes such as the gender or existence of mustache in the facial images.

Sign in / Sign up

Export Citation Format

Share Document