scholarly journals The first negative eigenvalue of Yoshida lifts

2019 ◽  
Vol 5 (3) ◽  
Author(s):  
Soumya Das ◽  
Ritwik Pal
Keyword(s):  
Negative Eigenvalue

scholarly journals Gradient Descent with Identity Initialization Efficiently Learns Positive-Definite Linear Transformations by Deep Residual Networks

2019 ◽  
Vol 31 (3) ◽  
pp. 477-502 ◽  
Author(s):  
Peter L. Bartlett ◽  
David P. Helmbold ◽  
Philip M. Long
Keyword(s):  
Least Squares ◽  
Gradient Descent ◽  
Singular Values ◽  
Negative Eigenvalue ◽  
Positive Definite ◽  
Quadratic Loss ◽  
Linear Transformations ◽  
Excess Loss ◽  
Initial Hypothesis ◽  
Matrix Formula

We analyze algorithms for approximating a function [Formula: see text] mapping [Formula: see text] to [Formula: see text] using deep linear neural networks, that is, that learn a function [Formula: see text] parameterized by matrices [Formula: see text] and defined by [Formula: see text]. We focus on algorithms that learn through gradient descent on the population quadratic loss in the case that the distribution over the inputs is isotropic. We provide polynomial bounds on the number of iterations for gradient descent to approximate the least-squares matrix [Formula: see text], in the case where the initial hypothesis [Formula: see text] has excess loss bounded by a small enough constant. We also show that gradient descent fails to converge for [Formula: see text] whose distance from the identity is a larger constant, and we show that some forms of regularization toward the identity in each layer do not help. If [Formula: see text] is symmetric positive definite, we show that an algorithm that initializes [Formula: see text] learns an [Formula: see text]-approximation of [Formula: see text] using a number of updates polynomial in [Formula: see text], the condition number of [Formula: see text], and [Formula: see text]. In contrast, we show that if the least-squares matrix [Formula: see text] is symmetric and has a negative eigenvalue, then all members of a class of algorithms that perform gradient descent with identity initialization, and optionally regularize toward the identity in each layer, fail to converge. We analyze an algorithm for the case that [Formula: see text] satisfies [Formula: see text] for all [Formula: see text] but may not be symmetric. This algorithm uses two regularizers: one that maintains the invariant [Formula: see text] for all [Formula: see text] and the other that “balances” [Formula: see text] so that they have the same singular values.

scholarly journals The non-convex geometry of low-rank matrix optimization

2018 ◽  
Vol 8 (1) ◽  
pp. 51-96 ◽  
Author(s):  
Qiuwei Li ◽  
Zhihui Zhu ◽  
Gongguo Tang
Keyword(s):  
Convex Function ◽  
Optimization Problems ◽  
Hessian Matrix ◽  
Convex Geometry ◽  
Negative Eigenvalue ◽  
Global Optimum ◽  
Nuclear Norm ◽  
Low Rank ◽  
Scalable Algorithms ◽  
General Convex

Abstract This work considers two popular minimization problems: (i) the minimization of a general convex function f(X) with the domain being positive semi-definite matrices, and (ii) the minimization of a general convex function f(X) regularized by the matrix nuclear norm $\|X\|_{*}$ with the domain being general matrices. Despite their optimal statistical performance in the literature, these two optimization problems have a high computational complexity even when solved using tailored fast convex solvers. To develop faster and more scalable algorithms, we follow the proposal of Burer and Monteiro to factor the low-rank variable $X = UU^{\top } $ (for semi-definite matrices) or $X=UV^{\top } $ (for general matrices) and also replace the nuclear norm $\|X\|_{*}$ with $\big(\|U\|_{F}^{2}+\|V\|_{F}^{2}\big)/2$. In spite of the non-convexity of the resulting factored formulations, we prove that each critical point either corresponds to the global optimum of the original convex problems or is a strict saddle where the Hessian matrix has a strictly negative eigenvalue. Such a nice geometric structure of the factored formulations allows many local-search algorithms to find a global optimizer even with random initializations.

scholarly journals Locating and Characterizing the Stationary Points of the Extended Rosenbrock Function

2009 ◽  
Vol 17 (3) ◽  
pp. 437-453 ◽  
Author(s):  
Schalk Kok ◽  
Carl Sandrock
Keyword(s):  
Random Search ◽  
Saddle Points ◽  
Relative Magnitude ◽  
Negative Eigenvalue ◽  
High Dimensionality ◽  
Stationary Points ◽  
Particle Swarm Algorithm ◽  
Initial Population ◽  
Arbitrary Precision ◽  
Newtonian Method

Two variants of the extended Rosenbrock function are analyzed in order to find the stationary points. The first variant is shown to possess a single stationary point, the global minimum. The second variant has numerous stationary points for high dimensionality. A previously proposed method is shown to be numerically intractable, requiring arbitrary precision computation in many cases to enumerate candidate solutions. Instead, a standard Newtonian method with multi-start is applied to locate stationary points. The relative magnitude of the negative and positive eigenvalues of the Hessian is also computed, in order to characterize the saddle points. For dimensions up to 100, only two local minimizers are found, but many saddle points exist. Two saddle points with a single negative eigenvalue exist for high dimensionality, which may appear as “near” local minima. The remaining saddle points we found have a predictable form, and a method is proposed to estimate their number. Monte Carlo simulation indicates that it is unlikely to escape these saddle points using uniform random search. A standard particle swarm algorithm also struggles to improve upon a saddle point contained within the initial population.

SLOW MANIFOLDS OF SOME CHAOTIC SYSTEMS WITH APPLICATIONS TO LASER SYSTEMS

2000 ◽  
Vol 10 (12) ◽  
pp. 2729-2744 ◽  
Author(s):  
SOFIANE RAMDANI ◽  
BRUNO ROSSETTO ◽  
LEON O. CHUA ◽  
RENÉ LOZI
Keyword(s):  
Negative Eigenvalue ◽  
Slow Manifold ◽  
Qualitative Description ◽  
Analytic Equation ◽  
Geometrical Characterization ◽  
Fast System ◽  
Laser Systems ◽  
The Lorenz System ◽  
Autonomous Dynamical Systems

In this work we deal with slow–fast autonomous dynamical systems. We initially define them as being modeled by systems of differential equations having a small parameter multiplying one of their velocity components. In order to analyze their solutions, some being chaotic, we have proposed a mathematical analytic method based on an iterative approach [Rossetto et al., 1998]. Under some conditions, this method allows us to give an analytic equation of the slow manifold. This equation is obtained by considering that the slow manifold is locally defined by a plane orthogonal to the tangent system's left fast eigenvector. In this paper, we give another method to compute the slow manifold equation by using the tangent system's slow eigenvectors. This method allows us to give a geometrical characterization of the attractor and a global qualitative description of its dynamics. The method used to compute the equation of the slow manifold has been extended to systems having a real and negative eigenvalue in a large domain of the phase space, as it is the case with the Lorenz system. Indeed, we give the Lorenz slow manifold equation and this allows us to make a qualitative study comparing this model and Chua's model. Finally, we apply our results to derive the slow manifold equations of a nonlinear optical slow–fast system, namely, the optical parametric oscillator model.

Effect of the eigenvalues of the velocity gradient tensor on particle collisions

2016 ◽  
Vol 792 ◽  
pp. 36-49 ◽  
Author(s):  
Vincent E. Perrin ◽  
Harmen J. J. Jonker
Keyword(s):  
Turbulent Flows ◽  
Velocity Gradient ◽  
Stokes Number ◽  
Negative Eigenvalue ◽  
Collision Kernel ◽  
Flow Structures ◽  
Particle Collisions ◽  
Gradient Tensor ◽  
Local Flow ◽  
Velocity Gradient Tensor

This study uses the eigenvalues of the local velocity gradient tensor to categorize the local flow structures in incompressible turbulent flows into different types of saddle nodes and vortices and investigates their effect on the local collision kernel of heavy particles. Direct numerical simulation (DNS) results show that most of the collisions occur in converging regions with real and negative eigenvalues. Those regions are associated not only with a stronger preferential clustering of particles, but also with a relatively higher collision kernel. To better understand the DNS results, a conceptual framework is developed to compute the collision kernel of individual flow structures. Converging regions, where two out of three eigenvalues are negative, posses a very high collision kernel, as long as a critical amount of rotation is not exceeded. Diverging regions, where two out of three eigenvalues are positive, have a very low collision kernel, which is governed by the third and negative eigenvalue. This model is not suited for particles with Stokes number $St\gg 1$, where the contribution of particle collisions from caustics is dominant.

Unsupervised classification based on non‐negative eigenvalue decomposition and Wishart classifier

2014 ◽  
Vol 8 (8) ◽  
pp. 957-964 ◽  
Author(s):  
Chunle Wang ◽  
Weidong Yu ◽  
Robert Wang ◽  
Yunkai Deng ◽  
Fengjun Zhao ◽  
...  
Keyword(s):  
Unsupervised Classification ◽  
Negative Eigenvalue ◽  
Eigenvalue Decomposition

STABILITY OR METASTABILITY AND EIGENVALUES OF THE EQUATION OF SMALL FLUCTUATIONS

1990 ◽  
Vol 05 (07) ◽  
pp. 1319-1339 ◽  
Author(s):  
H.J.W. MÜLLER-KIRSTEN ◽  
ZHANG JIAN-ZU ◽  
D.H. TCHRAKIAN
Keyword(s):  
Negative Eigenvalue ◽  
Variational Derivative ◽  
Euclidean Action ◽  
Double Well Potential

A theory with an asymmetric double-well potential is discussed and shown to possess a nontopological classical configuration like a bounce. It is then shown that the second variational derivative of the Euclidean action at this bounce-like configuration does not possess a negative eigenvalue. The significance of this observation is discussed in relation to more familiar models. Then a theory with a cubic potential is discussed and shown to possess a bounce with one negative eigenvalue of the second variational derivative which is indicative of metastability.

Sign in / Sign up

Export Citation Format

Share Document