Role of copositivity in optimality criteria for nonconvex optimization problems

1992 ◽  
Vol 75 (3) ◽  
pp. 535-558 ◽  
Author(s):  
G. Danninger
Keyword(s):  
Nonconvex Optimization ◽  
Optimization Problems ◽  
Optimality Criteria ◽  
Nonconvex Optimization Problems

scholarly journals A Diffusion Approximation Theory of Momentum Stochastic Gradient Descent in Nonconvex Optimization

2021 ◽  
Author(s):  
Tianyi Liu ◽  
Zhehui Chen ◽  
Enlu Zhou ◽  
Tuo Zhao
Keyword(s):  
Neural Networks ◽  
Nonconvex Optimization ◽  
Gradient Descent ◽  
Deep Neural Networks ◽  
Optimization Problems ◽  
Saddle Points ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Nonconvex Optimization Problems ◽  
Empirical Success

Momentum stochastic gradient descent (MSGD) algorithm has been widely applied to many nonconvex optimization problems in machine learning (e.g., training deep neural networks, variational Bayesian inference, etc.). Despite its empirical success, there is still a lack of theoretical understanding of convergence properties of MSGD. To fill this gap, we propose to analyze the algorithmic behavior of MSGD by diffusion approximations for nonconvex optimization problems with strict saddle points and isolated local optima. Our study shows that the momentum helps escape from saddle points but hurts the convergence within the neighborhood of optima (if without the step size annealing or momentum annealing). Our theoretical discovery partially corroborates the empirical success of MSGD in training deep neural networks.

scholarly journals MATHEMATICAL PROGRAMMING APPLICATIONS PECULIARITIES IN SHAKEDOWN PROBLEM/MATEMATINIO PROGRAMAVIMO TAIKYMO KONSTRUKCIJŲ PRISITAIKYMO UŽDAVINIUOSE YPATUMAI

2001 ◽  
Vol 7 (2) ◽  
pp. 106-114
Author(s):  
Ela Chraptovič ◽  
Juozas Atkočiūnas
Keyword(s):  
Mathematical Programming ◽  
Optimization Problems ◽  
Optimality Criteria ◽  
Point Of View ◽  
Design Matrix ◽  
Problem Solution ◽  
Strain Compatibility ◽  
Complete Set ◽  
Mechanical Interpretation

The theory of mathematical programming widely spread as a method of a solution of extreme problems. It accompanies the study of plastic theory problem from its posing up to final solution. However, here again from our point of view not all possibilities are realized. Unfortunately, the use of mathematical programming as an instrument of a numerical solution for structural analysis frequently is also restricted by that. The possibilities of mechanical interpretation of optimality criteria of applied algorithms are not uncovered. The global solution of the problem of mathematical programming exists, if Kuhn-Tucker conditions are satisfied. These conditions do not depend on the applied algorithm of a problem solution. The identity of Kuhn-Tucker conditions with a optimality criteria of Rosen algorithm is finding out in this research. The role of a design matrix for the creating of strain compatibility equations is clarified. The Kuhn-Tucker conditions mean the residual strain compatibility equations in analysis of elastic-plastic systems. It is proved in the article that for problems of limiting equilibrium the Kuhn-Tucker conditions include the dependences of the associated law of plastic flow. The Kuhn-Tucker conditions together with limitations of a source problem of account represent a complete set of dependences of the theory of shakedown. The correct mathematical and mechanical interpretation of the Kuhn-Tucker conditions allows to refuse a direct solution of a dual problem of mathematical programming. It makes easier the solution of optimization problems of structures at shakedown.

A new finite-time varying-parameter convergent-differential neural-network for solving nonlinear and nonconvex optimization problems

2018 ◽  
Vol 319 ◽  
pp. 74-83 ◽  
Author(s):  
Zhijun Zhang ◽  
Lunan Zheng ◽  
Lingao Li ◽  
Xiaoyan Deng ◽  
Lin Xiao ◽  
...  
Keyword(s):  
Neural Network ◽  
Nonconvex Optimization ◽  
Finite Time ◽  
Optimization Problems ◽  
Time Varying ◽  
Nonconvex Optimization Problems ◽  
Time Varying Parameter

A Novel Neural Network for Solving Nonsmooth Nonconvex Optimization Problems

2020 ◽  
Vol 31 (5) ◽  
pp. 1475-1488
Author(s):  
Xin Yu ◽  
Lingzhen Wu ◽  
Chenhua Xu ◽  
Yue Hu ◽  
Chong Ma
Keyword(s):  
Neural Network ◽  
Nonconvex Optimization ◽  
Optimization Problems ◽  
Nonconvex Optimization Problems ◽  
Nonsmooth Nonconvex Optimization
Sign in / Sign up

Export Citation Format

Share Document