scholarly journals Projective Low-rank Subspace Clustering via Learning Deep Encoder

2017 ◽  
Author(s):  
Jun Li ◽  
Liu Hongfu ◽  
Handong Zhao ◽  
Yun Fu
Keyword(s):  
Spectral Clustering ◽  
Large Scale ◽  
Computational Cost ◽  
Subspace Clustering ◽  
The State ◽  
Low Rank ◽  
Clustering Problem ◽  
Rank Decomposition ◽  
Row Space ◽  
Small Dataset

Low-rank subspace clustering (LRSC) has been considered as the state-of-the-art method on small datasets. LRSC constructs a desired similarity graph by low-rank representation (LRR), and employs a spectral clustering to segment the data samples. However, effectively applying LRSC into clustering big data becomes a challenge because both LRR and spectral clustering suffer from high computational cost. To address this challenge, we create a projective low-rank subspace clustering (PLrSC) scheme for large scale clustering problem. First, a small dataset is randomly sampled from big dataset. Second, our proposed predictive low-rank decomposition (PLD) is applied to train a deep encoder by using the small dataset, and the deep encoder is used to fast compute the low-rank representations of all data samples. Third, fast spectral clustering is employed to segment the representations. As a non-trivial contribution, we theoretically prove the deep encoder can universally approximate to the exact (or bounded) recovery of the row space. Experiments verify that our scheme outperforms the related methods on large scale datasets in a small amount of time. We achieve the state-of-art clustering accuracy by 95.8% on MNIST using scattering convolution features.

scholarly journals Large-scale Subspace Clustering by Fast Regression Coding

2017 ◽  
Author(s):  
Jun Li ◽  
Handong Zhao ◽  
Zhiqiang Tao ◽  
Yun Fu
Keyword(s):  
Linear Function ◽  
Spectral Clustering ◽  
Large Scale ◽  
Order Approximation ◽  
Subspace Clustering ◽  
Low Rank ◽  
First Order ◽  
Non Linear ◽  
Data Points ◽  
First Order Approximation

Large-Scale Subspace Clustering (LSSC) is an interesting and important problem in big data era. However, most existing methods (i.e., sparse or low-rank subspace clustering) cannot be directly used for solving LSSC because they suffer from the high time complexity-quadratic or cubic in n (the number of data points). To overcome this limitation, we propose a Fast Regression Coding (FRC) to optimize regression codes, and simultaneously train a non-linear function to approximate the codes. By using FRC, we develop an efficient Regression Coding Clustering (RCC) framework to solve the LSSC problem. It consists of sampling, FRC and clustering. RCC randomly samples a small number of data points, quickly calculates the codes of all data points by using the non-linear function learned from FRC, and employs a large-scale spectral clustering method to cluster the codes. Besides, we provide a theorem guarantee that the non-linear function has a first-order approximation ability and a group effect. The theorem manifests that the codes are easily used to construct a dividable similarity graph. Compared with the state-of-the-art LSSC methods, our model achieves better clustering results in large-scale datasets.

scholarly journals Orthogonal and Nonnegative Graph Reconstruction for Large Scale Clustering

2017 ◽  
Author(s):  
Junwei Han ◽  
Kai Xiong ◽  
Feiping Nie
Keyword(s):  
Spectral Clustering ◽  
Large Scale ◽  
Computational Cost ◽  
Post Processing ◽  
Normalized Cut ◽  
Clustering Problem ◽  
Novel Approach ◽  
Graph Reconstruction ◽  
Final Cluster ◽  
High Computational Cost

Spectral clustering has been widely used due to its simplicity for solving graph clustering problem in recent years. However, it suffers from the high computational cost as data grow in scale, and is limited by the performance of post-processing. To address these two problems simultaneously, in this paper, we propose a novel approach denoted by orthogonal and nonnegative graph reconstruction (ONGR) that scales linearly with the data size. For the relaxation of Normalized Cut, we add nonnegative constraint to the objective. Due to the nonnegativity, ONGR offers interpretability that the final cluster labels can be directly obtained without post-processing. Extensive experiments on clustering tasks demonstrate the effectiveness of the proposed method.

scholarly journals Sparse and Low-Rank Subspace Data Clustering with Manifold Regularization Learned by Local Linear Embedding

2018 ◽  
Vol 8 (11) ◽  
pp. 2175 ◽  
Author(s):  
Ye Yang ◽  
Yongli Hu ◽  
Fei Wu
Keyword(s):  
Data Clustering ◽  
Spectral Clustering ◽  
Subspace Clustering ◽  
Low Rank ◽  
Clustering Methods ◽  
Local Linear Embedding ◽  
Local Linear ◽  
Non Linear ◽  
Laplacian Graph ◽  
Linear Embedding

Data clustering is an important research topic in data mining and signal processing communications. In all the data clustering methods, the subspace spectral clustering methods based on self expression model, e.g., the Sparse Subspace Clustering (SSC) and the Low Rank Representation (LRR) methods, have attracted a lot of attention and shown good performance. The key step of SSC and LRR is to construct a proper affinity or similarity matrix of data for spectral clustering. Recently, Laplacian graph constraint was introduced into the basic SSC and LRR and obtained considerable improvement. However, the current graph construction methods do not well exploit and reveal the non-linear properties of the clustering data, which is common for high dimensional data. In this paper, we introduce the classic manifold learning method, the Local Linear Embedding (LLE), to learn the non-linear structure underlying the data and use the learned local geometry of manifold as a regularization for SSC and LRR, which results the proposed LLE-SSC and LLE-LRR clustering methods. Additionally, to solve the complex optimization problem involved in the proposed models, an efficient algorithm is also proposed. We test the proposed data clustering methods on several types of public databases. The experimental results show that our methods outperform typical subspace clustering methods with Laplacian graph constraint.

scholarly journals Image Inpainting Algorithm Based on Low-Rank Approximation and Texture Direction

2014 ◽  
Vol 2014 ◽  
pp. 1-11
Author(s):  
Jinjiang Li ◽  
Mengjun Li ◽  
Hui Fan
Keyword(s):  
Level Set ◽  
Large Scale ◽  
Image Inpainting ◽  
Low Rank ◽  
Low Rank Approximation ◽  
Rank Matrix ◽  
Rank Approximation ◽  
Rank Decomposition ◽  
The Impact ◽  
Low Rank Decomposition

Existing image inpainting algorithm based on low-rank matrix approximation cannot be suitable for complex, large-scale, damaged texture image. An inpainting algorithm based on low-rank approximation and texture direction is proposed in the paper. At first, we decompose the image using low-rank approximation method. Then the area to be repaired is interpolated by level set algorithm, and we can reconstruct a new image by the boundary values of level set. In order to obtain a better restoration effect, we make iteration for low-rank decomposition and level set interpolation. Taking into account the impact of texture direction, we segment the texture and make low-rank decomposition at texture direction. Experimental results show that the new algorithm is suitable for texture recovery and maintaining the overall consistency of the structure, which can be used to repair large-scale damaged image.

scholarly journals Efficient Techniques for Solving the Periodic Projected Lyapunov Equations and Model Reduction of Periodic Systems

2017 ◽  
Vol 2017 ◽  
pp. 1-11 ◽  
Author(s):  
Mohammad-Sahadet Hossain ◽  
M. Monir Uddin
Keyword(s):  
Model Reduction ◽  
Discrete Time ◽  
Large Scale ◽  
Computational Cost ◽  
Descriptor Systems ◽  
Low Rank ◽  
Lyapunov Equations ◽  
Adi Method ◽  
Smith Method ◽  
Alternating Direction

We have presented the efficient techniques for the solutions of large-scale sparse projected periodic discrete-time Lyapunov equations in lifted form. These types of problems arise in model reduction and state feedback problems of periodic descriptor systems. Two most popular techniques to solve such Lyapunov equations iteratively are the low-rank alternating direction implicit (LR-ADI) method and the low-rank Smith method. The main contribution of this paper is to update the LR-ADI method by exploiting the ideas of the adaptive shift parameters computation and the efficient handling of complex shift parameters. These approaches efficiently reduce the computational cost with respect to time and memory. We also apply these iterative Lyapunov solvers in balanced truncation model reduction of periodic discrete-time descriptor systems. We illustrate numerical results to show the performance and accuracy of the proposed methods.

Robust Structured Low-Rank Representation for Image Segmentation

2018 ◽  
Vol 27 (05) ◽  
pp. 1850020 ◽  
Author(s):  
Cong-Zhe You ◽  
Vasile Palade ◽  
Xiao-Jun Wu
Keyword(s):  
Clustering Analysis ◽  
Spectral Clustering ◽  
Optimization Problem ◽  
Subspace Clustering ◽  
Low Rank ◽  
Joint Optimization ◽  
High Dimensional ◽  
Great Success ◽  
Subspace Segmentation ◽  
Low Rank Representation

Subspace clustering analysis algorithms are often employed when dealing with high-dimensional data. As a representative approach, Low-Rank Representation (LRR) of data has achieved great success for subspace segmentation tasks in applications such as image processing. The traditional LRR-related methods consist of two separate tasks: first, the affinity graph construction by using lowrank minimization techniques, and then the spectral clustering, which is done on the affinity graph to get the final segmentation. Since these two steps are independent of each other, this method does not guarantee that the results obtained by the algorithm are globally optimal. In this paper, a method called Robust Structured Low-Rank Representation (RSLRR) is proposed, by integrating the two above mentioned tasks and solve a joint optimization problem. This paper also puts forward a method to solve the joint optimization problem, which can efficiently get both the segmentation and the structured low-rank representation. Experiments on several standard datasets show that, compared with other algorithms, the algorithm proposed in this paper can achieve better clustering results.

scholarly journals A Random Algorithm for Low-Rank Decomposition of Large-Scale Matrices With Missing Entries

2015 ◽  
Vol 24 (11) ◽  
pp. 4502-4511 ◽  
Author(s):  
Yiguang Liu ◽  
Yinjie Lei ◽  
Chunguang Li ◽  
Wenzheng Xu ◽  
Yifei Pu
Keyword(s):  
Large Scale ◽  
Low Rank ◽  
Random Algorithm ◽  
Rank Decomposition ◽  
Low Rank Decomposition

scholarly journals LogDet Rank Minimization with Application to Subspace Clustering

2015 ◽  
Vol 2015 ◽  
pp. 1-10 ◽  
Author(s):  
Zhao Kang ◽  
Chong Peng ◽  
Jie Cheng ◽  
Qiang Cheng
Keyword(s):  
Large Scale ◽  
Clustering Algorithms ◽  
Subspace Clustering ◽  
Singular Values ◽  
Nuclear Norm ◽  
Low Rank ◽  
Face Clustering ◽  
Rank Matrix ◽  
Low Rank Representation ◽  
Clustering Data

Low-rank matrix is desired in many machine learning and computer vision problems. Most of the recent studies use the nuclear norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and thus the rank may not be well approximated in practical problems. In this paper, we propose using a log-determinant (LogDet) function as a smooth and closer, though nonconvex, approximation to rank for obtaining a low-rank representation in subspace clustering. Augmented Lagrange multipliers strategy is applied to iteratively optimize the LogDet-based nonconvex objective function on potentially large-scale data. By making use of the angular information of principal directions of the resultant low-rank representation, an affinity graph matrix is constructed for spectral clustering. Experimental results on motion segmentation and face clustering data demonstrate that the proposed method often outperforms state-of-the-art subspace clustering algorithms.

Sign in / Sign up

Export Citation Format

Share Document