scholarly journals A Boosting Framework of Factorization Machine

2021 ◽  
pp. 2159036
Author(s):  
Jun Zhou ◽  
Longfei Li ◽  
Ziqi Liu ◽  
Chaochao Chen
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Experimental Results ◽  
Low Rank ◽  
Inner Product ◽  
Adaptive Boosting ◽  
Rank Matrix ◽  
Factorization Machine ◽  
Fixed Rank ◽  
Low Rank Matrix

Recently, Factorization Machine (FM) has become more and more popular for recommendation systems due to its effectiveness in finding informative interactions between features. Usually, the weights for the interactions are learned as a low rank weight matrix, which is formulated as an inner product of two low rank matrices. This low rank matrix can help improve the generalization ability of Factorization Machine. However, to choose the rank properly, it usually needs to run the algorithm for many times using different ranks, which clearly is inefficient for some large-scale datasets. To alleviate this issue, we propose an Adaptive Boosting framework of Factorization Machine (AdaFM), which can adaptively search for proper ranks for different datasets without re-training. Instead of using a fixed rank for FM, the proposed algorithm will gradually increase its rank according to its performance until the performance does not grow. Extensive experiments are conducted to validate the proposed method on multiple large-scale datasets. The experimental results demonstrate that the proposed method can be more effective than the state-of-the-art Factorization Machines.

scholarly journals Low-Rank Approximation of Difference between Correlation Matrices Using Inner Product

2021 ◽  
Vol 11 (10) ◽  
pp. 4582
Author(s):  
Kensuke Tanioka ◽  
Satoru Hiwa
Keyword(s):  
Dimensional Reduction ◽  
Low Rank ◽  
Inner Product ◽  
Low Rank Approximation ◽  
Correlation Matrices ◽  
Rank Matrix ◽  
Clustering Approach ◽  
Rank Approximation ◽  
The Difference ◽  
Low Rank Matrix

In the domain of functional magnetic resonance imaging (fMRI) data analysis, given two correlation matrices between regions of interest (ROIs) for the same subject, it is important to reveal relatively large differences to ensure accurate interpretation. However, clustering results based only on differences tend to be unsatisfactory and interpreting the features tends to be difficult because the differences likely suffer from noise. Therefore, to overcome these problems, we propose a new approach for dimensional reduction clustering. Methods: Our proposed dimensional reduction clustering approach consists of low-rank approximation and a clustering algorithm. The low-rank matrix, which reflects the difference, is estimated from the inner product of the difference matrix, not only from the difference. In addition, the low-rank matrix is calculated based on the majorize–minimization (MM) algorithm such that the difference is bounded within the range −1 to 1. For the clustering process, ordinal k-means is applied to the estimated low-rank matrix, which emphasizes the clustering structure. Results: Numerical simulations show that, compared with other approaches that are based only on differences, the proposed method provides superior performance in recovering the true clustering structure. Moreover, as demonstrated through a real-data example of brain activity measured via fMRI during the performance of a working memory task, the proposed method can visually provide interpretable community structures consisting of well-known brain functional networks, which can be associated with the human working memory system. Conclusions: The proposed dimensional reduction clustering approach is a very useful tool for revealing and interpreting the differences between correlation matrices, even when the true differences tend to be relatively small.

scholarly journals A non-convex optimization framework for large-scale low-rank matrix factorization

2020 ◽  
Author(s):  
Sajad Fathi Hafshejani ◽  
Saeed Vahidian ◽  
Zahra Moaberfard ◽  
Reza Alikhani ◽  
Bill Lin
Keyword(s):  
Matrix Factorization ◽  
Large Scale ◽  
Significant Loss ◽  
Low Rank ◽  
Function Evaluation ◽  
Dimensional Manifold ◽  
Step Size ◽  
Rank Matrix ◽  
Real World Datasets ◽  
Low Rank Matrix

Low-rank matrix factorization problems such as non negative matrix factorization (NMF) can be categorized as a clustering or dimension reduction technique. The latter denotes techniques designed to find representations of some high dimensional dataset in a lower dimensional manifold without a significant loss of information. If such a representation exists, the features ought to contain the most relevant features of the dataset. Many linear dimensionality reduction techniques can be formulated as a matrix factorization. In this paper, we combine the conjugate gradient (CG) method with the Barzilai and Borwein (BB) gradient method, and propose a BB scaling CG method for NMF problems. The new method does not require to compute and store matrices associated with Hessian of the objective functions. Moreover, adopting a suitable BB step size along with a proper nonmonotone strategy which comes by the size convex parameter $\eta_k$, results in a new algorithm that can significantly improve the CPU time, efficiency, the number of function evaluation. Convergence result is established and numerical comparisons of methods on both synthetic and real-world datasets show that the proposed method is efficient in comparison with existing methods and demonstrate the superiority of our algorithms.

scholarly journals A Non-Convex Optimization Framework for Large-Scale Low-rank Matrix Factorization

2021 ◽  
Author(s):  
Sajad Fathi Hafshejani ◽  
Saeed Vahidian ◽  
Zahra Moaberfard ◽  
Bill Lin
Keyword(s):  
Matrix Factorization ◽  
Large Scale ◽  
Significant Loss ◽  
Low Rank ◽  
Function Evaluation ◽  
Dimensional Manifold ◽  
Step Size ◽  
Rank Matrix ◽  
Real World Datasets ◽  
Low Rank Matrix

Low-rank matrix factorization problems such as non negative matrix factorization (NMF) can be categorized as a clustering or dimension reduction technique. The latter denotes techniques designed to find representations of some high dimensional dataset in a lower dimensional manifold without a significant loss of information. If such a representation exists, the features ought to contain the most relevant features of the dataset. Many linear dimensionality reduction techniques can be formulated as a matrix factorization. In this paper, we combine the conjugate gradient (CG) method with the Barzilai and Borwein (BB) gradient method, and propose a BB scaling CG method for NMF problems. The new method does not require to compute and store matrices associated with Hessian of the objective functions. Moreover, adopting a suitable BB step size along with a proper nonmonotone strategy which comes by the size convex parameter $\eta_k$, results in a new algorithm that can significantly improve the CPU time, efficiency, the number of function evaluation. Convergence result is established and numerical comparisons of methods on both synthetic and real-world datasets show that the proposed method is efficient in comparison with existing methods and demonstrate the superiority of our algorithms.

Fast and unique Tucker decompositions via multiway blind source separation

2012 ◽  
Vol 60 (3) ◽  
pp. 389-405 ◽  
Author(s):  
G. Zhou ◽  
A. Cichocki
Keyword(s):  
Dimensionality Reduction ◽  
Blind Source Separation ◽  
Large Scale ◽  
High Efficiency ◽  
Source Separation ◽  
Low Rank ◽  
Large Scale Problems ◽  
Rank Matrix ◽  
Reduction Methods ◽  
Low Rank Matrix

Abstract A multiway blind source separation (MBSS) method is developed to decompose large-scale tensor (multiway array) data. Benefitting from all kinds of well-established constrained low-rank matrix factorization methods, MBSS is quite flexible and able to extract unique and interpretable components with physical meaning. The multilinear structure of Tucker and the essential uniqueness of BSS methods allow MBSS to estimate each component matrix separately from an unfolding matrix in each mode. Consequently, alternating least squares (ALS) iterations, which are considered as the workhorse for tensor decompositions, can be avoided and various robust and efficient dimensionality reduction methods can be easily incorporated to pre-process the data, which makes MBSS extremely fast, especially for large-scale problems. Identification and uniqueness conditions are also discussed. Two practical issues dimensionality reduction and estimation of number of components are also addressed based on sparse and random fibers sampling. Extensive simulations confirmed the validity, flexibility, and high efficiency of the proposed method. We also demonstrated by simulations that the MBSS approach can successfully extract desired components while most existing algorithms may fail for ill-conditioned and large-scale problems.

scholarly journals Robust Asymmetric Bayesian Adaptive Matrix Factorization

2017 ◽  
Author(s):  
Xin Guo ◽  
Boyuan Pan ◽  
Deng Cai ◽  
Xiaofei He
Keyword(s):  
Real World ◽  
Matrix Factorization ◽  
State Of The Art ◽  
Low Rank ◽  
Data Sets ◽  
Real World Application ◽  
Factorization Model ◽  
Rank Matrix ◽  
Wide Range ◽  
Low Rank Matrix

Low rank matrix factorizations(LRMF) have attracted much attention due to its wide range of applications in computer vision, such as image impainting and video denoising. Most of the existing methods assume that the loss between an observed measurement matrix and its bilinear factorization follows symmetric distribution, like gaussian or gamma families. However, in real-world situations, this assumption is often found too idealized, because pictures under various illumination and angles may suffer from multi-peaks, asymmetric and irregular noises. To address these problems, this paper assumes that the loss follows a mixture of Asymmetric Laplace distributions and proposes robust Asymmetric Laplace Adaptive Matrix Factorization model(ALAMF) under bayesian matrix factorization framework. The assumption of Laplace distribution makes our model more robust and the asymmetric attribute makes our model more flexible and adaptable to real-world noise. A variational method is then devised for model inference. We compare ALAMF with other state-of-the-art matrix factorization methods both on data sets ranging from synthetic and real-world application. The experimental results demonstrate the effectiveness of our proposed approach.

scholarly journals A Generalized Robust Minimization Framework for Low-Rank Matrix Recovery

2014 ◽  
Vol 2014 ◽  
pp. 1-8 ◽  
Author(s):  
Wen-Ze Shao ◽  
Qi Ge ◽  
Zong-Liang Gan ◽  
Hai-Song Deng ◽  
Hai-Bo Li
Keyword(s):  
Lagrange Multiplier ◽  
State Of The Art ◽  
Numerical Algorithms ◽  
Low Rank ◽  
Nonconvex Minimization ◽  
Matrix Recovery ◽  
Rank Matrix ◽  
Augmented Lagrange Multiplier ◽  
Low Rank Matrix Recovery ◽  
Low Rank Matrix

This paper considers the problem of recovering low-rank matrices which are heavily corrupted by outliers or large errors. To improve the robustness of existing recovery methods, the problem is solved by formulating it as a generalized nonsmooth nonconvex minimization functional via exploiting the Schattenp-norm(0 < p ≤1)andLq(0 < q ≤1)seminorm. Two numerical algorithms are provided based on the augmented Lagrange multiplier (ALM) and accelerated proximal gradient (APG) methods as well as efficient root-finder strategies. Experimental results demonstrate that the proposed generalized approach is more inclusive and effective compared with state-of-the-art methods, either convex or nonconvex.

Matrix completion via minimizing an approximate rank

2019 ◽  
Vol 17 (05) ◽  
pp. 689-713
Author(s):  
Xueying Zeng ◽  
Lixin Shen ◽  
Yuesheng Xu ◽  
Jian Lu
Keyword(s):  
State Of The Art ◽  
Matrix Completion ◽  
Relaxation Parameter ◽  
Rank Function ◽  
Low Rank ◽  
Completion Problem ◽  
Rank Matrix ◽  
Matrix Completion Problem ◽  
Low Rank Matrix Completion ◽  
Low Rank Matrix

The low rank matrix completion problem which aims to recover a matrix from that having missing entries has received much attention in many fields such as image processing and machine learning. The rank of a matrix may be measured by the [Formula: see text] norm of the vector of its singular values. Due to the nonconvexity and discontinuity of the [Formula: see text] norm, solving the low rank matrix completion problem which is clearly NP hard suffers from computational challenges. In this paper, we propose a constrained matrix completion model in which a novel nonconvex continuous rank surrogate is used to approximate the rank function of a matrix, promote low rank of the recovered matrix and address the computational challenges. The proposed rank surrogate differs from the convex nuclear norm and other existing state-of-the-art nonconvex surrogates in a way that it alleviates the discontinuity and nonconvexity of the rank function through a local [Formula: see text]-relaxation of the [Formula: see text] norm so that it possesses several desirable properties. These properties ensure that it accurately approximates the rank function by choosing an appropriate relaxation parameter. We moreover develop an efficient iterative algorithm to solve the resulting model. We also propose strategies of automatically updating the relaxation parameter to practically ensure the global convergence and speed up the algorithm. We establish theoretical convergence results for the proposed algorithm. Experimental results are presented to demonstrate significant performance improvements of the proposed model and the associated algorithm as compared to state-of-the-art methods in both recoverability and computational efficiency.

scholarly journals Low rank approximation of difference between correlation matrices by using inner product

2021 ◽  
Author(s):  
Kensuke Tanioka ◽  
Satoru Hiwa
Keyword(s):  
Dimensional Reduction ◽  
Low Rank ◽  
Inner Product ◽  
Low Rank Approximation ◽  
Correlation Matrices ◽  
Rank Matrix ◽  
Clustering Approach ◽  
Rank Approximation ◽  
The Difference ◽  
Low Rank Matrix

ABSTRACTIntroductionIn the domain of functional magnetic resonance imaging (fMRI) data analysis, given two correlation matrices between regions of interest (ROIs) for the same subject, it is important to reveal relatively large differences to ensure accurate interpretations. However, clustering results based only on difference tend to be unsatisfactory, and interpreting features is difficult because the difference suffers from noise. Therefore, to overcome these problems, we propose a new approach for dimensional reduction clustering.MethodsOur proposed dimensional reduction clustering approach consists of low rank approximation and a clustering algorithm. The low rank matrix, which reflects the difference, is estimated from the inner product of the difference matrix, not only the difference. In addition, the low rank matrix is calculated based on the majorize-minimization (MM) algorithm such that the difference is bounded from 1 to 1. For the clustering process, ordinal k-means is applied to the estimated low rank matrix, which emphasizes the clustering structure.ResultsNumerical simulations show that, compared with other approaches that are based only on difference, the proposed method provides superior performance in recovering the true clustering structure. Moreover, as demonstrated through a real data example of brain activity while performing a working memory task measured by fMRI, the proposed method can visually provide interpretable community structures consisted of well-known brain functional networks which can be associated with human working memory system.ConclusionsThe proposed dimensional reduction clustering approach is a very useful tool for revealing and interpreting the differences between correlation matrices, even if the true difference tends to be relatively small.

scholarly journals Multifocus Image Fusion Using a Sparse and Low-Rank Matrix Decomposition for Aviator’s Night Vision Goggle

2020 ◽  
Vol 10 (6) ◽  
pp. 2178 ◽  
Author(s):  
Bo-Lin Jian ◽  
Wen-Lin Chu ◽  
Yu-Chung Li ◽  
Her-Terng Yau
Keyword(s):  
Image Fusion ◽  
Search Algorithm ◽  
Matrix Decomposition ◽  
Night Vision ◽  
Experimental Results ◽  
Low Rank ◽  
Image Capture ◽  
Rank Matrix ◽  
Low Rank Matrix ◽  
Consistency Verification

This study proposed the concept of sparse and low-rank matrix decomposition to address the need for aviator’s night vision goggles (NVG) automated inspection processes when inspecting equipment availability. First, the automation requirements include machinery and motor-driven focus knob of NVGs and image capture using cameras to achieve autofocus. Traditionally, passive autofocus involves first computing of sharpness of each frame and then use of a search algorithm to quickly find the sharpest focus. In this study, the concept of sparse and low-rank matrix decomposition was adopted to achieve autofocus calculation and image fusion. Image fusion can solve the multifocus problem caused by mechanism errors. Experimental results showed that the sharpest image frame and its nearby frame can be image-fused to resolve minor errors possibly arising from the image-capture mechanism. In this study, seven samples and 12 image-fusing indicators were employed to verify the image fusion based on variance calculated in a discrete cosine transform domain without consistency verification, with consistency verification, structure-aware image fusion, and the proposed image fusion method. Experimental results showed that the proposed method was superior to other methods and compared the autofocus put forth in this paper and the normalized gray-level variance sharpness results in the documents to verify accuracy.

Sign in / Sign up

Export Citation Format

Share Document