scholarly journals A Block Coordinate Descent-Based Projected Gradient Algorithm for Orthogonal Non-Negative Matrix Factorization

Mathematics ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. 540
Author(s):  
Soodabeh Asadi ◽  
Janez Povh
Keyword(s):  
Matrix Factorization ◽  
Synthetic Data ◽  
Coordinate Descent ◽  
The Other ◽  
Gradient Algorithm ◽  
Block Coordinate Descent ◽  
Projected Gradient Method ◽  
Projected Gradient ◽  
Factorization Problem ◽  
Non Negative Matrix Factorization

This article uses the projected gradient method (PG) for a non-negative matrix factorization problem (NMF), where one or both matrix factors must have orthonormal columns or rows. We penalize the orthonormality constraints and apply the PG method via a block coordinate descent approach. This means that at a certain time one matrix factor is fixed and the other is updated by moving along the steepest descent direction computed from the penalized objective function and projecting onto the space of non-negative matrices. Our method is tested on two sets of synthetic data for various values of penalty parameters. The performance is compared to the well-known multiplicative update (MU) method from Ding (2006), and with a modified global convergent variant of the MU algorithm recently proposed by Mirzal (2014). We provide extensive numerical results coupled with appropriate visualizations, which demonstrate that our method is very competitive and usually outperforms the other two methods.

scholarly journals A Two-Phase Algorithm for Robust Symmetric Non-Negative Matrix Factorization

Symmetry ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1757
Author(s):  
Bingjie Li ◽  
Xi Shi ◽  
Zhenyue Zhang
Keyword(s):  
Gradient Method ◽  
Matrix Factorization ◽  
Initial Point ◽  
Linear Structure ◽  
Second Phase ◽  
Projected Gradient Method ◽  
Two Phase ◽  
Negative Part ◽  
Projected Gradient ◽  
Non Negative Matrix Factorization

As a special class of non-negative matrix factorization, symmetric non-negative matrix factorization (SymNMF) has been widely used in the machine learning field to mine the hidden non-linear structure of data. Due to the non-negative constraint and non-convexity of SymNMF, the efficiency of existing methods is generally unsatisfactory. To tackle this issue, we propose a two-phase algorithm to solve the SymNMF problem efficiently. In the first phase, we drop the non-negative constraint of SymNMF and propose a new model with penalty terms, in order to control the negative component of the factor. Unlike previous methods, the factor sequence in this phase is not required to be non-negative, allowing fast unconstrained optimization algorithms, such as the conjugate gradient method, to be used. In the second phase, we revisit the SymNMF problem, taking the non-negative part of the solution in the first phase as the initial point. To achieve faster convergence, we propose an interpolation projected gradient (IPG) method for SymNMF, which is much more efficient than the classical projected gradient method. Our two-phase algorithm is easy to implement, with convergence guaranteed for both phases. Numerical experiments show that our algorithm performs better than others on synthetic data and unsupervised clustering tasks.

scholarly journals Integrating hypertension phenotype and genotype with hybrid non-negative matrix factorization

2018 ◽  
Vol 35 (8) ◽  
pp. 1395-1403 ◽  
Author(s):  
Yuan Luo ◽  
Chengsheng Mao ◽  
Yiben Yang ◽  
Fei Wang ◽  
Faraz S Ahmad ◽  
...  
Keyword(s):  
Matrix Factorization ◽  
Cardiac Mechanics ◽  
Approximation Problem ◽  
Supplementary Information ◽  
Joint Analysis ◽  
Learning Approaches ◽  
Patient Stratification ◽  
Projected Gradient Method ◽  
Genotype Information ◽  
Non Negative Matrix Factorization

Abstract Motivation Hypertension is a heterogeneous syndrome in need of improved subtyping using phenotypic and genetic measurements with the goal of identifying subtypes of patients who share similar pathophysiologic mechanisms and may respond more uniformly to targeted treatments. Existing machine learning approaches often face challenges in integrating phenotype and genotype information and presenting to clinicians an interpretable model. We aim to provide informed patient stratification based on phenotype and genotype features. Results In this article, we present a hybrid non-negative matrix factorization (HNMF) method to integrate phenotype and genotype information for patient stratification. HNMF simultaneously approximates the phenotypic and genetic feature matrices using different appropriate loss functions, and generates patient subtypes, phenotypic groups and genetic groups. Unlike previous methods, HNMF approximates phenotypic matrix under Frobenius loss, and genetic matrix under Kullback-Leibler (KL) loss. We propose an alternating projected gradient method to solve the approximation problem. Simulation shows HNMF converges fast and accurately to the true factor matrices. On a real-world clinical dataset, we used the patient factor matrix as features and examined the association of these features with indices of cardiac mechanics. We compared HNMF with six different models using phenotype or genotype features alone, with or without NMF, or using joint NMF with only one type of loss We also compared HNMF with 3 recently published methods for integrative clustering analysis, including iClusterBayes, Bayesian joint analysis and JIVE. HNMF significantly outperforms all comparison models. HNMF also reveals intuitive phenotype–genotype interactions that characterize cardiac abnormalities. Availability and implementation Our code is publicly available on github at https://github.com/yuanluo/hnmf. Supplementary information Supplementary data are available at Bioinformatics online.

PROBABILISTIC NON-NEGATIVE MATRIX FACTORIZATION: THEORY AND APPLICATION TO MICROARRAY DATA ANALYSIS

2014 ◽  
Vol 12 (01) ◽  
pp. 1450001 ◽  
Author(s):  
BELHASSEN BAYAR ◽  
NIDHAL BOUAYNAYA ◽  
ROMAN SHTERENBERG
Keyword(s):  
Physical Interpretation ◽  
Matrix Factorization ◽  
Dna Microarrays ◽  
Optimal Solution ◽  
Microarray Data Analysis ◽  
Factorization Problem ◽  
The Stability ◽  
Update Rules ◽  
Negative Factors ◽  
Non Negative Matrix Factorization

Non-negative matrix factorization (NMF) has proven to be a useful decomposition technique for multivariate data, where the non-negativity constraint is necessary to have a meaningful physical interpretation. NMF reduces the dimensionality of non-negative data by decomposing it into two smaller non-negative factors with physical interpretation for class discovery. The NMF algorithm, however, assumes a deterministic framework. In particular, the effect of the data noise on the stability of the factorization and the convergence of the algorithm are unknown. Collected data, on the other hand, is stochastic in nature due to measurement noise and sometimes inherent variability in the physical process. This paper presents new theoretical and applied developments to the problem of non-negative matrix factorization (NMF). First, we generalize the deterministic NMF algorithm to include a general class of update rules that converges towards an optimal non-negative factorization. Second, we extend the NMF framework to the probabilistic case (PNMF). We show that the Maximum a posteriori (MAP) estimate of the non-negative factors is the solution to a weighted regularized non-negative matrix factorization problem. We subsequently derive update rules that converge towards an optimal solution. Third, we apply the PNMF to cluster and classify DNA microarrays data. The proposed PNMF is shown to outperform the deterministic NMF and the sparse NMF algorithms in clustering stability and classification accuracy.

Swarm Intelligence for Non-Negative Matrix Factorization

2011 ◽  
Vol 2 (4) ◽  
pp. 12-34 ◽  
Author(s):  
Andreas Janecek ◽  
Ying Tan
Keyword(s):  
Swarm Intelligence ◽  
Matrix Factorization ◽  
Classification Accuracy ◽  
Approximation Error ◽  
Synthetic Data ◽  
Low Rank ◽  
Low Rank Approximation ◽  
Approximation Quality ◽  
Initialization Strategy ◽  
Non Negative Matrix Factorization

The Non-negative Matrix Factorization (NMF) is a special low-rank approximation which allows for an additive parts-based and interpretable representation of the data. This article presents efforts to improve the convergence, approximation quality, and classification accuracy of NMF using five different meta-heuristics based on swarm intelligence. Several properties of the NMF objective function motivate the utilization of meta-heuristics: this function is non-convex, discontinuous, and may possess many local minima. The proposed optimization strategies are two-fold: On the one hand, a new initialization strategy for NMF is presented in order to initialize the NMF factors prior to the factorization; on the other hand, an iterative update strategy is proposed, which improves the accuracy per runtime for the multiplicative update NMF algorithm. The success of the proposed optimization strategies are shown by applying them on synthetic data and data sets coming from the areas of spam filtering/email classification, and evaluate them also in their application context. Experimental results show that both optimization strategies are able to improve NMF in terms of faster convergence, lower approximation error, and better classification accuracy. Especially the initialization strategy leads to significant reductions of the runtime per accuracy ratio for both, the NMF approximation as well as the classification results achieved with NMF.

Sign in / Sign up

Export Citation Format

Share Document