Incremental Matrix Factorization: A Linear Feature Transformation Perspective

Matrix Factorization (MF) is among the most widely used techniques for collaborative filtering based recommendation. Along this line, a critical demand is to incrementally refine the MF models when new ratings come in an online scenario. However, most of existing incremental MF algorithms are limited by specific MF models or strict use restrictions. In this paper, we propose a general incremental MF framework by designing a linear transformation of user and item latent vectors over time. This framework shows a relatively high accuracy with a computation and space efficient training process in an online scenario. Meanwhile, we explain the framework with a low-rank approximation perspective, and give an upper bound on the training error when this framework is used for incremental learning in some special cases. Finally, extensive experimental results on two real-world datasets clearly validate the effectiveness, efficiency and storage performance of the proposed framework.

Download Full-text

A PSO Based Approach for Producing Optimized Latent Factor in Special Reference to Big Data

International Journal of Service Science Management Engineering and Technology ◽

10.4018/ijssmet.2016070104 ◽

2016 ◽

Vol 7 (3) ◽

pp. 55-70 ◽

Cited By ~ 9

Author(s):

Bharat Singh ◽

Om Prakash Vyas

Keyword(s):

Big Data ◽

Matrix Factorization ◽

Nonnegative Matrix ◽

Low Rank ◽

Data Matrix ◽

Experimental Result ◽

Low Rank Approximation ◽

Latent Factor ◽

New Approach ◽

Rank Approximation

Now a day's application deal with Big Data has tremendously been used in the popular areas. To tackle with such kind of data various approaches have been developed by researchers in the last few decades. A recent investigated techniques to factored the data matrix through a known latent factor in a lower size space is the so called matrix factorization. In addition, one of the problems with the NMF approaches, its randomized valued could not provide absolute optimization in limited iteration, but having local optimization. Due to this, the authors have proposed a new approach that considers the initial values of the decomposition to tackle the issues of computationally expensive. They have devised an algorithm for initializing the values of the decomposed matrix based on the PSO. In this paper, the auhtors have intended a genetic algorithm based technique while incorporating the nonnegative matrix factorization. Through the experimental result, they will show the proposed method converse very fast in comparison to other low rank approximation like simple NMF multiplicative, and ACLS technique.

Download Full-text

A hybrid algorithm for low-rank approximation of nonnegative matrix factorization

Neurocomputing ◽

10.1016/j.neucom.2019.07.059 ◽

2019 ◽

Vol 364 ◽

pp. 129-137

Author(s):

Peitao Wang ◽

Zhaoshui He ◽

Kan Xie ◽

Junbin Gao ◽

Michael Antolovich ◽

...

Keyword(s):

Matrix Factorization ◽

Hybrid Algorithm ◽

Nonnegative Matrix Factorization ◽

Nonnegative Matrix ◽

Low Rank ◽

Low Rank Approximation ◽

Rank Approximation

Download Full-text

A Bi-level Formulation for Label Noise Learning with Spectral Cluster Discovery

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/361 ◽

2020 ◽

Author(s):

Yijing Luo ◽

Bo Han ◽

Chen Gong

Keyword(s):

Ground Truth ◽

Spectral Cluster ◽

Low Rank ◽

Low Rank Approximation ◽

Label Noise ◽

Learning Stage ◽

Cluster Membership ◽

Rank Approximation ◽

Real World Datasets ◽

Classifier Learning

Practically, we often face the dilemma that some of the examples for training a classifier are incorrectly labeled due to various subjective and objective factors. Although intensive efforts have been put to design classifiers that are robust to label noise, most of the previous methods have not fully utilized data distribution information. To address this issue, this paper introduces a bi-level learning paradigm termed “Spectral Cluster Discovery'' (SCD) for combating with noisy labels. Namely, we simultaneously learn a robust classifier (Learning stage) by discovering the low-rank approximation to the ground-truth label matrix and learn an ideal affinity graph (Clustering stage). Specifically, we use the learned classifier to assign the examples with similar label to a mutual cluster. Based on the cluster membership, we utilize the learned affinity graph to explore the noisy examples based on the cluster membership. Both stages will reinforce each other iteratively. Experimental results on typical benchmark and real-world datasets verify the superiority of SCD to other label noise learning methods.

Download Full-text

A projective approach to nonnegative matrix factorization

Electronic Journal of Linear Algebra ◽

10.13001/ela.2021.5067 ◽

2021 ◽

Vol 37 ◽

pp. 583-597

Author(s):

Patrick Groetzner

Keyword(s):

Matrix Factorization ◽

Data Science ◽

Nonnegative Matrix Factorization ◽

Heuristic Method ◽

Nonnegative Matrix ◽

Type A ◽

Low Rank ◽

Feasibility Problem ◽

Low Rank Approximation ◽

Rank Approximation

In data science and machine learning, the method of nonnegative matrix factorization (NMF) is a powerful tool that enjoys great popularity. Depending on the concrete application, there exist several subclasses each of which performs a NMF under certain constraints. Consider a given square matrix $A$. The symmetric NMF aims for a nonnegative low-rank approximation $A\approx XX^T$ to $A$, where $X$ is entrywise nonnegative and of given order. Considering a rectangular input matrix $A$, the general NMF again aims for a nonnegative low-rank approximation to $A$ which is now of the type $A\approx XY$ for entrywise nonnegative matrices $X,Y$ of given order. In this paper, we introduce a new heuristic method to tackle the exact nonnegative matrix factorization problem (of type $A=XY$), based on projection approaches to solve a certain feasibility problem.

Download Full-text

HPOAnnotator: improving large-scale prediction of HPO annotations by low-rank approximation with HPO semantic similarities and multiple PPI networks

BMC Medical Genomics ◽

10.1186/s12920-019-0625-1 ◽

2019 ◽

Vol 12 (S10) ◽

Cited By ~ 2

Author(s):

Junning Gao ◽

Lizhi Liu ◽

Shuwei Yao ◽

Xiaodi Huang ◽

Hiroshi Mamitsuka ◽

...

Keyword(s):

Hierarchical Structure ◽

Matrix Factorization ◽

Large Scale ◽

Low Rank ◽

Superior Performance ◽

Low Rank Approximation ◽

Human Phenotype ◽

Ppi Networks ◽

The Hierarchical Structure ◽

Rank Approximation

Abstract Background As a standardized vocabulary of phenotypic abnormalities associated with human diseases, the Human Phenotype Ontology (HPO) has been widely used by researchers to annotate phenotypes of genes/proteins. For saving the cost and time spent on experiments, many computational approaches have been proposed. They are able to alleviate the problem to some extent, but their performances are still far from satisfactory. Method For inferring large-scale protein-phenotype associations, we propose HPOAnnotator that incorporates multiple Protein-Protein Interaction (PPI) information and the hierarchical structure of HPO. Specifically, we use a dual graph to regularize Non-negative Matrix Factorization (NMF) in a way that the information from different sources can be seamlessly integrated. In essence, HPOAnnotator solves the sparsity problem of a protein-phenotype association matrix by using a low-rank approximation. Results By combining the hierarchical structure of HPO and co-annotations of proteins, our model can well capture the HPO semantic similarities. Moreover, graph Laplacian regularizations are imposed in the latent space so as to utilize multiple PPI networks. The performance of HPOAnnotator has been validated under cross-validation and independent test. Experimental results have shown that HPOAnnotator outperforms the competing methods significantly. Conclusions Through extensive comparisons with the state-of-the-art methods, we conclude that the proposed HPOAnnotator is able to achieve the superior performance as a result of using a low-rank approximation with a graph regularization. It is promising in that our approach can be considered as a starting point to study more efficient matrix factorization-based algorithms.

Download Full-text

Integrating Low-rank Approximation and Word Embedding for Feature Transformation in the High-dimensional Text Classification

Procedia Computer Science ◽

10.1016/j.procs.2017.08.058 ◽

2017 ◽

Vol 112 ◽

pp. 437-446

Author(s):

Le Nguyen Hoai Nam ◽

Ho Bao Quoc

Keyword(s):

Text Classification ◽

Word Embedding ◽

Low Rank ◽

High Dimensional ◽

Feature Transformation ◽

Low Rank Approximation ◽

Rank Approximation

Download Full-text

Pseudoinverse graph convolutional networks

Data Mining and Knowledge Discovery ◽

10.1007/s10618-021-00752-w ◽

2021 ◽

Author(s):

Dominik Alfke ◽

Martin Stoll

Keyword(s):

Real World ◽

Point Clouds ◽

Graph Laplacian ◽

Low Rank ◽

Low Rank Approximation ◽

Convolutional Networks ◽

3D Point Clouds ◽

Non Local ◽

Rank Approximation ◽

Real World Datasets

AbstractGraph Convolutional Networks (GCNs) have proven to be successful tools for semi-supervised classification on graph-based datasets. We propose a new GCN variant whose three-part filter space is targeted at dense graphs. Our examples include graphs generated from 3D point clouds with an increased focus on non-local information, as well as hypergraphs based on categorical data of real-world problems. These graphs differ from the common sparse benchmark graphs in terms of the spectral properties of their graph Laplacian. Most notably we observe large eigengaps, which are unfavorable for popular existing GCN architectures. Our method overcomes these issues by utilizing the pseudoinverse of the Laplacian. Another key ingredient is a low-rank approximation of the convolutional matrix, ensuring computational efficiency and increasing accuracy at the same time. We outline how the necessary eigeninformation can be computed efficiently in each applications and discuss the appropriate choice of the only metaparameter, the approximation rank. We finally showcase our method’s performance regarding runtime and accuracy in various experiments with real-world datasets.

Download Full-text