Projection Methods for Dynamical Low-Rank Approximation of High-Dimensional Problems

2019 ◽  
Vol 19 (1) ◽  
pp. 73-92 ◽  
Author(s):  
Emil Kieri ◽  
Bart Vandereycken
Keyword(s):  
Time Integration ◽  
Singular Values ◽  
Projection Methods ◽  
Euler Method ◽  
Low Rank ◽  
High Dimensional ◽  
Matrix Product ◽  
Low Rank Approximation ◽  
Rank Approximation ◽  
Fixed Rank

AbstractWe consider dynamical low-rank approximation on the manifold of fixed-rank matrices and tensor trains (also called matrix product states), and analyse projection methods for the time integration of such problems. First, under suitable approximability assumptions, we prove error estimates for the explicit Euler method equipped with quasi-optimal projections to the manifold. Then we discuss the possibilities and difficulties with higher-order explicit methods. In particular, we discuss ways for limiting rank growth in the increments, and robustness with respect to small singular values.

scholarly journals An unconventional robust integrator for dynamical low-rank approximation

2021 ◽  
Author(s):  
Gianluca Ceruti ◽  
Christian Lubich
Keyword(s):  
Differential Equation ◽  
Time Integration ◽  
Singular Values ◽  
Time Dependent ◽  
Low Rank ◽  
Low Rank Approximation ◽  
Numerical Integrator ◽  
The Matrix ◽  
Rank Approximation ◽  
Matrix Differential

AbstractWe propose and analyse a numerical integrator that computes a low-rank approximation to large time-dependent matrices that are either given explicitly via their increments or are the unknown solution to a matrix differential equation. Furthermore, the integrator is extended to the approximation of time-dependent tensors by Tucker tensors of fixed multilinear rank. The proposed low-rank integrator is different from the known projector-splitting integrator for dynamical low-rank approximation, but it retains the important robustness to small singular values that has so far been known only for the projector-splitting integrator. The new integrator also offers some potential advantages over the projector-splitting integrator: It avoids the backward time integration substep of the projector-splitting integrator, which is a potentially unstable substep for dissipative problems. It offers more parallelism, and it preserves symmetry or anti-symmetry of the matrix or tensor when the differential equation does. Numerical experiments illustrate the behaviour of the proposed integrator.

scholarly journals Discretized Dynamical Low-Rank Approximation in the Presence of Small Singular Values

2016 ◽  
Vol 54 (2) ◽  
pp. 1020-1038 ◽  
Author(s):  
Emil Kieri ◽  
Christian Lubich ◽  
Hanna Walach
Keyword(s):  
Singular Values ◽  
Low Rank ◽  
Low Rank Approximation ◽  
Rank Approximation

On Low Rank Approximation of Linear Operators in p-Norms and Some Algorithms

2016 ◽  
Vol 33 (04) ◽  
pp. 1650023
Author(s):  
Yang Liu
Keyword(s):  
Linear Operator ◽  
Sparse Matrix ◽  
Matrix Completion ◽  
Singular Values ◽  
Linear Operators ◽  
Low Rank ◽  
Minkowski Distance ◽  
Low Rank Approximation ◽  
Rank Approximation ◽  
Generalized Singular Values

In this paper, we study the optimal or best approximation of any linear operator by low rank linear operators, especially, any linear operator on the [Formula: see text]-space, [Formula: see text], under [Formula: see text] norm, or in Minkowski distance. Considering generalized singular values and using techniques from differential geometry, we extend the classical Schmidt–Mirsky theorem in the direction of the [Formula: see text]-norm of linear operators for some [Formula: see text] values. Also, we develop and provide algorithms for finding the solution to the low rank approximation problems in some nontrivial scenarios. The results can be applied to, in particular, matrix completion and sparse matrix recovery.

scholarly journals Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation

Entropy ◽  
2021 ◽  
Vol 23 (8) ◽  
pp. 990
Author(s):  
Théo Galy-Fajou ◽  
Valerio Perrone ◽  
Manfred Opper
Keyword(s):  
Gaussian Approximation ◽  
Approximation Problem ◽  
Low Rank ◽  
High Dimensional ◽  
Variational Approximation ◽  
Gradient Flows ◽  
Low Rank Approximation ◽  
Rank Approximation ◽  
Non Gaussian ◽  
Number Of Particles

Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets.

Sign in / Sign up

Export Citation Format

Share Document