scholarly journals On critical points of quadratic low-rank matrix optimization problems

2020 ◽  
Vol 40 (4) ◽  
pp. 2626-2651
Author(s):  
André Uschmajew ◽  
Bart Vandereycken
Keyword(s):  
Critical Points ◽  
Optimization Problems ◽  
Quadratic Function ◽  
Singular Values ◽  
Positive Semidefinite ◽  
Optimization Methods ◽  
Low Rank ◽  
Local Minima ◽  
Rank Matrix ◽  
Low Rank Matrix

Abstract The absence of spurious local minima in certain nonconvex low-rank matrix recovery problems has been of recent interest in computer science, machine learning and compressed sensing since it explains the convergence of some low-rank optimization methods to global optima. One such example is low-rank matrix sensing under restricted isometry properties (RIPs). It can be formulated as a minimization problem for a quadratic function on the Riemannian manifold of low-rank matrices, with a positive semidefinite Riemannian Hessian that acts almost like an identity on low-rank matrices. In this work new estimates for singular values of local minima for such problems are given, which lead to improved bounds on RIP constants to ensure absence of nonoptimal local minima and sufficiently negative curvature at all other critical points. A geometric viewpoint is taken, which is inspired by the fact that the Euclidean distance function to a rank-$k$ matrix possesses no critical points on the corresponding embedded submanifold of rank-$k$ matrices except for the single global minimum.

scholarly journals s-Goodness for Low-Rank Matrix Recovery

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Lingchen Kong ◽  
Levent Tunçel ◽  
Naihua Xiu
Keyword(s):  
Linear Transformation ◽  
Optimization Problems ◽  
Sufficient Conditions ◽  
Nuclear Norm ◽  
Low Rank ◽  
Matrix Recovery ◽  
Rank Matrix ◽  
Necessary And Sufficient ◽  
Low Rank Matrix Recovery ◽  
Low Rank Matrix

Low-rank matrix recovery (LMR) is a rank minimization problem subject to linear equality constraints, and it arises in many fields such as signal and image processing, statistics, computer vision, and system identification and control. This class of optimization problems is generally𝒩𝒫hard. A popular approach replaces the rank function with the nuclear norm of the matrix variable. In this paper, we extend and characterize the concept ofs-goodness for a sensing matrix in sparse signal recovery (proposed by Juditsky and Nemirovski (Math Program, 2011)) to linear transformations in LMR. Using the two characteristics-goodness constants,γsandγ^s, of a linear transformation, we derive necessary and sufficient conditions for a linear transformation to bes-good. Moreover, we establish the equivalence ofs-goodness and the null space properties. Therefore,s-goodness is a necessary and sufficient condition for exacts-rank matrix recovery via the nuclear norm minimization.

An exact penalty method for semidefinite-box-constrained low-rank matrix optimization problems

2018 ◽  
Vol 40 (1) ◽  
pp. 563-586
Author(s):  
Tianxiang Liu ◽  
Zhaosong Lu ◽  
Xiaojun Chen ◽  
Yu-Hong Dai
Keyword(s):  
Penalty Method ◽  
Original Problem ◽  
Optimization Problems ◽  
Rank Correlation ◽  
Low Rank ◽  
Matrix Optimization ◽  
Rank Constraint ◽  
Rank Matrix ◽  
Adaptive Penalty ◽  
Low Rank Matrix

Abstract This paper considers a matrix optimization problem where the objective function is continuously differentiable and the constraints involve a semidefinite-box constraint and a rank constraint. We first replace the rank constraint by adding a non-Lipschitz penalty function in the objective and prove that this penalty problem is exact with respect to the original problem. Next, for the penalty problem we present a nonmonotone proximal gradient (NPG) algorithm whose subproblem can be solved by Newton’s method with globally quadratic convergence. We also prove the convergence of the NPG algorithm to a first-order stationary point of the penalty problem. Furthermore, based on the NPG algorithm, we propose an adaptive penalty method (APM) for solving the original problem. Finally, the efficiency of an APM is shown via numerical experiments for the sensor network localization problem and the nearest low-rank correlation matrix problem.

scholarly journals Fabric defect detection based on deep-handcrafted feature and weighted low-rank matrix representation

2021 ◽  
Vol 16 ◽  
pp. 155892502110084
Author(s):  
Chunlei Li ◽  
Ban Jiang ◽  
Zhoufeng Liu ◽  
Yan Dong ◽  
Shuili Tang ◽  
...  
Keyword(s):  
Defect Detection ◽  
Matrix Representation ◽  
Singular Values ◽  
Low Rank ◽  
Fabric Defect Detection ◽  
Rank Matrix ◽  
Fabric Defect ◽  
The Matrix ◽  
Low Rank Representation ◽  
Low Rank Matrix

In the process of textile production, automatic defect detection plays a key role in controlling product quality. Due to the complex texture features of fabric image, the traditional detection methods have poor adaptability, and low detection accuracy. The low rank representation model can divide the image into the low rank background and sparse object, and has proven suitable for fabric defect detection. However, how to further effectively characterize the fabric texture is still problematic in this kind of method. Moreover, most of them adopt nuclear norm optimization algorithm to solve the low rank model, which treat every singular value in the matrix equally. However, in the task of fabric defect detection, different singular values of feature matrix represent different information. In this paper, we proposed a novel fabric defect detection method based on the deep-handcrafted feature and weighted low-rank matrix representation. The feature characterization ability is effectively improved by fusing the global deep feature extracted by VGG network and the handcrafted low-level feature. Moreover, a weighted low-rank representation model is constructed to treat the matrix singular values differently by different weights, thus the most distinguishing feature of fabric texture can be preserved, which can efficiently outstand the defect and suppress the background. Qualitative and quantitative experiments on two public datasets show that our proposed method outperforms the state-of-the-art methods.

Active Subspace: Toward Scalable Low-Rank Learning

2012 ◽  
Vol 24 (12) ◽  
pp. 3371-3394 ◽  
Author(s):  
Guangcan Liu ◽  
Shuicheng Yan
Keyword(s):  
Large Scale ◽  
Optimization Problems ◽  
Optimal Solution ◽  
Low Rank ◽  
Small Scale ◽  
Large Solution ◽  
Rank Matrix ◽  
Low Rank Matrix ◽  
Active Subspace ◽  
Solution Matrix

We address the scalability issues in low-rank matrix learning problems. Usually these problems resort to solving nuclear norm regularized optimization problems (NNROPs), which often suffer from high computational complexities if based on existing solvers, especially in large-scale settings. Based on the fact that the optimal solution matrix to an NNROP is often low rank, we revisit the classic mechanism of low-rank matrix factorization, based on which we present an active subspace algorithm for efficiently solving NNROPs by transforming large-scale NNROPs into small-scale problems. The transformation is achieved by factorizing the large solution matrix into the product of a small orthonormal matrix (active subspace) and another small matrix. Although such a transformation generally leads to nonconvex problems, we show that a suboptimal solution can be found by the augmented Lagrange alternating direction method. For the robust PCA (RPCA) (Candès, Li, Ma, & Wright, 2009 ) problem, a typical example of NNROPs, theoretical results verify the suboptimality of the solution produced by our algorithm. For the general NNROPs, we empirically show that our algorithm significantly reduces the computational complexity without loss of optimality.

Optimized quantum singular value thresholding algorithm based on a hybrid quantum computer

2021 ◽  
Author(s):  
Yangyang Ge ◽  
Zhimin Wang ◽  
Wen Zheng ◽  
Yu Zhang ◽  
Xiangmin Yu ◽  
...  
Keyword(s):  
Quantum Computer ◽  
Singular Values ◽  
Singular Value ◽  
Low Rank ◽  
Continuous Variables ◽  
Rank Matrix ◽  
Near Term ◽  
Thresholding Algorithm ◽  
Low Rank Matrix ◽  
Singular Value Thresholding

Abstract Quantum singular value thresholding (QSVT) algorithm, as a core module of many mathematical models, seeks the singular values of a sparse and low rank matrix exceeding a threshold and their associated singular vectors. The existing all-qubit QSVT algorithm demands lots of ancillary qubits, remaining a huge challenge for realization on near-term intermediate-scale quantum computers. In this paper, we propose a hybrid QSVT (HQSVT) algorithm utilizing both discrete variables (DVs) and continuous variables (CVs). In our algorithm, raw data vectors are encoded into a qubit system and the following data processing is fulfilled by hybrid quantum operations. Our algorithm requires O[log(MN)] qubits with O(1) qumodes and totally performs O(1) operations, which significantly reduces the space and runtime consumption.

scholarly journals Image Denoising Using Low Rank Matrix Approximation in Singular Value Decomposition

2021 ◽  
Vol 11 (2) ◽  
pp. 1430-1446
Author(s):  
Satyanarayana Tallapragada V.V.
Keyword(s):  
Singular Value Decomposition ◽  
Image Denoising ◽  
Singular Values ◽  
Singular Value ◽  
Low Rank ◽  
Lower Rank ◽  
Rank Matrix ◽  
Wide Range ◽  
Value Decomposition ◽  
Low Rank Matrix

The factorization of a matrix into lower rank matrices give solutions to a wide range of computer vision and image processing tasks. The inherent patches or the atomic patches can completely describe the whole image. The lower rank matrices are obtained using different tools including Singular Value Decomposition (SVD), which is typically found in minimization problems of nuclear norms. The singular values obtained will generally be a thresholder to realize the nuclear norm minimization. However, soft-thresholding is performed uniformly on all the singular values that lead to a similar importance to all the patches whether it is principal/useful or not. Our observation is that the decision on a patch (to be principal/useful or not) can be taken only when the application of this minimization is taken into consideration. Thus, in this paper, we propose a new method for image denoising by choosing variable weights to different singular values with a deep noise effect. Experimental results illustrate that the proposed weighted scheme performs better than the state-of-the-art methods.

Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks

2018 ◽  
Author(s):  
Daniel Povey ◽  
Gaofeng Cheng ◽  
Yiming Wang ◽  
Ke Li ◽  
Hainan Xu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Matrix Factorization ◽  
Deep Neural Networks ◽  
Low Rank ◽  
Rank Matrix ◽  
Low Rank Matrix
Sign in / Sign up

Export Citation Format

Share Document