scholarly journals Scaled Coupled Norms and Coupled Higher-Order Tensor Completion

2020 ◽  
Vol 32 (2) ◽  
pp. 447-484
Author(s):  
Kishan Wimalawarne ◽  
Makoto Yamada ◽  
Hiroshi Mamitsuka
Keyword(s):  
Real Data ◽  
Higher Order ◽  
Low Rank ◽  
Tensor Completion ◽  
Coupled Modes ◽  
Coupled Mode ◽  
The Matrix ◽  
Tensor Norms ◽  
Higher Order Tensors ◽  
Risk Bounds

Recently, a set of tensor norms known as coupled norms has been proposed as a convex solution to coupled tensor completion. Coupled norms have been designed by combining low-rank inducing tensor norms with the matrix trace norm. Though coupled norms have shown good performances, they have two major limitations: they do not have a method to control the regularization of coupled modes and uncoupled modes, and they are not optimal for couplings among higher-order tensors. In this letter, we propose a method that scales the regularization of coupled components against uncoupled components to properly induce the low-rankness on the coupled mode. We also propose coupled norms for higher-order tensors by combining the square norm to coupled norms. Using the excess risk-bound analysis, we demonstrate that our proposed methods lead to lower risk bounds compared to existing coupled norms. We demonstrate the robustness of our methods through simulation and real-data experiments.

scholarly journals Convex Coupled Matrix and Tensor Completion

2018 ◽  
Vol 30 (11) ◽  
pp. 3095-3127 ◽  
Author(s):  
Kishan Wimalawarne ◽  
Makoto Yamada ◽  
Hiroshi Mamitsuka
Keyword(s):  
Optimal Solution ◽  
Real Data ◽  
Excess Risk ◽  
Low Rank ◽  
Trace Norm ◽  
Tensor Completion ◽  
The Matrix ◽  
Risk Bounds ◽  
Matrix Trace

We propose a set of convex low-rank inducing norms for coupled matrices and tensors (hereafter referred to as coupled tensors), in which information is shared between the matrices and tensors through common modes. More specifically, we first propose a mixture of the overlapped trace norm and the latent norms with the matrix trace norm, and then, propose a completion model regularized using these norms to impute coupled tensors. A key advantage of the proposed norms is that they are convex and can be used to find a globally optimal solution, whereas existing methods for coupled learning are nonconvex. We also analyze the excess risk bounds of the completion model regularized using our proposed norms and show that they can exploit the low-rankness of coupled tensors, leading to better bounds compared to those obtained using uncoupled norms. Through synthetic and real-data experiments, we show that the proposed completion model compares favorably with existing ones.

scholarly journals Tensor theta norms and low rank recovery

2020 ◽  
Author(s):  
Holger Rauhut ◽  
Željka Stojanac
Keyword(s):  
Convex Relaxation ◽  
Nuclear Norm ◽  
Low Rank ◽  
Nuclear Norm Minimization ◽  
Computational Algebraic Geometry ◽  
Matrix Recovery ◽  
Norm Minimization ◽  
The Matrix ◽  
Higher Order Tensors ◽  
Tensor Nuclear Norm

AbstractWe study extensions of compressive sensing and low rank matrix recovery to the recovery of tensors of low rank from incomplete linear information. While the reconstruction of low rank matrices via nuclear norm minimization is rather well-understand by now, almost no theory is available so far for the extension to higher order tensors due to various theoretical and computational difficulties arising for tensor decompositions. In fact, nuclear norm minimization for matrix recovery is a tractable convex relaxation approach, but the extension of the nuclear norm to tensors is in general NP-hard to compute. In this article, we introduce convex relaxations of the tensor nuclear norm which are computable in polynomial time via semidefinite programming. Our approach is based on theta bodies, a concept from real computational algebraic geometry which is similar to the one of the better known Lasserre relaxations. We introduce polynomial ideals which are generated by the second-order minors corresponding to different matricizations of the tensor (where the tensor entries are treated as variables) such that the nuclear norm ball is the convex hull of the algebraic variety of the ideal. The theta body of order k for such an ideal generates a new norm which we call the θk-norm. We show that in the matrix case, these norms reduce to the standard nuclear norm. For tensors of order three or higher however, we indeed obtain new norms. The sequence of the corresponding unit-θk-norm balls converges asymptotically to the unit tensor nuclear norm ball. By providing the Gröbner basis for the ideals, we explicitly give semidefinite programs for the computation of the θk-norm and for the minimization of the θk-norm under an affine constraint. Finally, numerical experiments for order-three tensor recovery via θ1-norm minimization suggest that our approach successfully reconstructs tensors of low rank from incomplete linear (random) measurements.

scholarly journals Near-optimal sample complexity for convex tensor completion

2018 ◽  
Vol 8 (3) ◽  
pp. 577-619 ◽  
Author(s):  
Navid Ghadermarzy ◽  
Yaniv Plan ◽  
Özgür Yilmaz
Keyword(s):  
Low Rank ◽  
Sample Complexity ◽  
Tensor Completion ◽  
Norm Minimization ◽  
Optimal Sample ◽  
Rank Tensor ◽  
Atomic Norm ◽  
The Matrix ◽  
Tensor Nuclear Norm ◽  
Minimax Rate

Abstract We study the problem of estimating a low-rank tensor when we have noisy observations of a subset of its entries. A rank-$r$, order-$d$, $N \times N \times \cdots \times N$ tensor, where $r=O(1)$, has $O(dN)$ free variables. On the other hand, prior to our work, the best sample complexity that was achieved in the literature is $O\left(N^{\frac{d}{2}}\right)$, obtained by solving a tensor nuclear-norm minimization problem. In this paper, we consider the ‘M-norm’, an atomic norm whose atoms are rank-1 sign tensors. We also consider a generalization of the matrix max-norm to tensors, which results in a quasi-norm that we call ‘max-qnorm’. We prove that solving an M-norm constrained least squares (LS) problem results in nearly optimal sample complexity for low-rank tensor completion (TC). A similar result holds for max-qnorm as well. Furthermore, we show that these bounds are nearly minimax rate-optimal. We also provide promising numerical results for max-qnorm constrained TC, showing improved recovery compared to matricization and alternating LS.

scholarly journals An Introduction to Hierarchical (H-) Rank and TT-Rank of Tensors with Examples

2011 ◽  
Vol 11 (3) ◽  
pp. 291-304 ◽  
Author(s):  
Lars Grasedyck ◽  
Wolfgang Hackbusch
Keyword(s):  
Sparse Representation ◽  
Higher Order ◽  
The Other ◽  
Matrix Rank ◽  
The Matrix ◽  
Rank One ◽  
Higher Order Tensors

Abstract We review two similar concepts of hierarchical rank of tensors (which extend the matrix rank to higher order tensors): the TT-rank and the H-rank (hierarchical or H-Tucker rank). Based on this notion of rank, one can define a data-sparse representation of tensors involving O(dnk + dk^3) data for order d tensors with mode sizes n and rank k. Simple examples underline the differences and similarities between the different formats and ranks. Finally, we derive rank bounds for tensors in one of the formats based on the ranks in the other format.

scholarly journals An Efficient Tensor Completion Method Combining Matrix Factorization and Smoothness

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Leiming Tang ◽  
Xunjie Cao ◽  
Weiyang Chen ◽  
Changbo Ye
Keyword(s):  
Matrix Factorization ◽  
Low Complexity ◽  
Low Rank ◽  
Total Variation Regularization ◽  
Tensor Completion ◽  
Alternating Direction ◽  
Factorization Model ◽  
Efficiency And Effectiveness ◽  
The Matrix ◽  
Accelerate Convergence

In this paper, the low-complexity tensor completion (LTC) scheme is proposed to improve the efficiency of tensor completion. On one hand, the matrix factorization model is established for complexity reduction, which adopts the matrix factorization into the model of low-rank tensor completion. On the other hand, we introduce the smoothness by total variation regularization and framelet regularization to guarantee the completion performance. Accordingly, given the proposed smooth matrix factorization (SMF) model, an alternating direction method of multiple- (ADMM-) based solution is further proposed to realize the efficient and effective tensor completion. Additionally, we employ a novel tensor initialization approach to accelerate convergence speed. Finally, simulation results are presented to confirm the system gain of the proposed LTC scheme in both efficiency and effectiveness.

scholarly journals Outlier-Robust Multi-Aspect Streaming Tensor Completion and Factorization

2019 ◽  
Author(s):  
Mehrnaz Najafi ◽  
Lifang He ◽  
Philip S. Yu
Keyword(s):  
Real World ◽  
Missing Values ◽  
Real Data ◽  
Low Rank ◽  
Tensor Factorization ◽  
Tensor Completion ◽  
Smooth Optimization ◽  
Novel Method ◽  
Real World Datasets ◽  
Tensor Data

With the increasing popularity of streaming tensor data such as videos and audios, tensor factorization and completion have attracted much attention recently in this area. Existing work usually assume that streaming tensors only grow in one mode. However, in many real-world scenarios, tensors may grow in multiple modes (or dimensions), i.e., multi-aspect streaming tensors. Standard streaming methods cannot directly handle this type of data elegantly. Moreover, due to inevitable system errors, data may be contaminated by outliers, which cause significant deviations from real data values and make such research particularly challenging. In this paper, we propose a novel method for Outlier-Robust Multi-Aspect Streaming Tensor Completion and Factorization (OR-MSTC), which is a technique capable of dealing with missing values and outliers in multi-aspect streaming tensor data. The key idea is to decompose the tensor structure into an underlying low-rank clean tensor and a structured-sparse error (outlier) tensor, along with a weighting tensor to mask missing data. We also develop an efficient algorithm to solve the non-convex and non-smooth optimization problem of OR-MSTC. Experimental results on various real-world datasets show the superiority of the proposed method over the baselines and its robustness against outliers.

scholarly journals Supervised dimensionality reduction for big data

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Joshua T. Vogelstein ◽  
Eric W. Bridgeford ◽  
Minh Tang ◽  
Da Zheng ◽  
Christopher Douville ◽  
...  
Keyword(s):  
Dimensionality Reduction ◽  
Data Science ◽  
Real Data ◽  
Low Rank ◽  
Conditional Moment ◽  
Desktop Computer ◽  
Reduction Techniques ◽  
Reduction Methods ◽  
The Individual ◽  
Low Dimensional

AbstractTo solve key biomedical problems, experimentalists now routinely measure millions or billions of features (dimensions) per sample, with the hope that data science techniques will be able to build accurate data-driven inferences. Because sample sizes are typically orders of magnitude smaller than the dimensionality of these data, valid inferences require finding a low-dimensional representation that preserves the discriminating information (e.g., whether the individual suffers from a particular disease). There is a lack of interpretable supervised dimensionality reduction methods that scale to millions of dimensions with strong statistical theoretical guarantees. We introduce an approach to extending principal components analysis by incorporating class-conditional moment estimates into the low-dimensional projection. The simplest version, Linear Optimal Low-rank projection, incorporates the class-conditional means. We prove, and substantiate with both synthetic and real data benchmarks, that Linear Optimal Low-Rank Projection and its generalizations lead to improved data representations for subsequent classification, while maintaining computational efficiency and scalability. Using multiple brain imaging datasets consisting of more than 150 million features, and several genomics datasets with more than 500,000 features, Linear Optimal Low-Rank Projection outperforms other scalable linear dimensionality reduction techniques in terms of accuracy, while only requiring a few minutes on a standard desktop computer.

scholarly journals Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity

Biometrika ◽  
2020 ◽  
Author(s):  
S Na ◽  
M Kolar ◽  
O Koyejo
Keyword(s):  
Graphical Models ◽  
Latent Variable ◽  
Graphical Model ◽  
Brain Connectivity ◽  
Statistical Error ◽  
Real Data ◽  
Low Rank ◽  
Conditional Dependence ◽  
Gradient Descent Algorithm ◽  
The Difference

Abstract Differential graphical models are designed to represent the difference between the conditional dependence structures of two groups, thus are of particular interest for scientific investigation. Motivated by modern applications, this manuscript considers an extended setting where each group is generated by a latent variable Gaussian graphical model. Due to the existence of latent factors, the differential network is decomposed into sparse and low-rank components, both of which are symmetric indefinite matrices. We estimate these two components simultaneously using a two-stage procedure: (i) an initialization stage, which computes a simple, consistent estimator, and (ii) a convergence stage, implemented using a projected alternating gradient descent algorithm applied to a nonconvex objective, initialized using the output of the first stage. We prove that given the initialization, the estimator converges linearly with a nontrivial, minimax optimal statistical error. Experiments on synthetic and real data illustrate that the proposed nonconvex procedure outperforms existing methods.

Sign in / Sign up

Export Citation Format

Share Document