Examining the effect of second-order terms in mathematical programming approaches to the classification problem

1996 ◽  
Vol 93 (3) ◽  
pp. 582-601 ◽  
Author(s):  
Pradit Wanarat ◽  
Robert Pavur
Keyword(s):  
Mathematical Programming ◽  
Classification Problem ◽  
Second Order

scholarly journals Accelerating Symmetric Rank-1 Quasi-Newton Method with Nesterov’s Gradient for Training Neural Networks

2021 ◽  
Author(s):  
S. Indrapriyadarsini ◽  
Shahrzad Mahboubi ◽  
Hiroshi Ninomiya ◽  
Takeshi Kamio ◽  
Hideki Asai
Keyword(s):  
Neural Networks ◽  
Newton Method ◽  
Trust Region ◽  
Nonlinear Problems ◽  
Classification Problem ◽  
Second Order ◽  
Highly Nonlinear ◽  
Gradient Based ◽  
Quasi Newton ◽  
Second Order Methods

Gradient based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method though less commonly used in training neural networks, are known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks and briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

scholarly journals Accelerating Symmetric Rank-1 Quasi-Newton Method with Nesterov’s Gradient for Training Neural Networks

2021 ◽  
Author(s):  
S. Indrapriyadarsini ◽  
Shahrzad Mahboubi ◽  
Hiroshi Ninomiya ◽  
Takeshi Kamio ◽  
Hideki Asai
Keyword(s):  
Neural Networks ◽  
Newton Method ◽  
Trust Region ◽  
Nonlinear Problems ◽  
Classification Problem ◽  
Second Order ◽  
Highly Nonlinear ◽  
Gradient Based ◽  
Quasi Newton ◽  
Second Order Methods

Gradient based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method though less commonly used in training neural networks, are known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks and briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

scholarly journals Symmetric duality in complex spaces over cones

2021 ◽  
pp. 4-4
Author(s):  
Izhar Ahmad ◽  
Divya Agarwal ◽  
Kumar Gupta
Keyword(s):  
Mathematical Programming ◽  
Duality Theory ◽  
Optimization Theory ◽  
Second Order ◽  
Symmetric Duality ◽  
Polyhedral Cones ◽  
Complex Spaces ◽  
Duality Results ◽  
Duality Relations

Duality theory plays an important role in optimization theory. It has been extensively used for many theoretical and computational problems in mathematical programming. In this paper duality results are established for first and second order Wolfe and Mond-Weir type symmetric dual programs over general polyhedral cones in complex spaces. Corresponding duality relations for nondifferentiable case are also stated. This work will also remove inconsistencies in the earlier work from the literature.

On the uniformization of plane direction fields, and of second-order partial differential operators

1973 ◽  
Vol 73 (1) ◽  
pp. 87-109 ◽  
Author(s):  
Robert Finn
Keyword(s):  
Elliptic Equations ◽  
Differential Operators ◽  
Classification Problem ◽  
Second Order ◽  
Generalized Solutions ◽  
Partial Differential ◽  
Uniformization Theorem ◽  
Partial Differential Operators ◽  
Recent Developments ◽  
Direction Fields

One of the topics covered by most textbooks on partial differential equations is the classification problem for second-order equations in the plane. In a typical treatment, it is shown that under some smoothness conditions on the coefficients, hyperbolic and parabolic equations can be reduced locally to normal form (uniformized) by an elementary procedure. For elliptic equations, the procedure fails unless an extraneous hypothesis (analyticity of the coefficients) is introduced. It is then pointed out that a different and much deeper method (essentially the general uniformization theorem) is effective for the elliptic case and even yields a global result. It is striking, but not surprising in view of recent developments on generalized solutions, that the alternate (global) procedure for elliptic equations requires much less smoothness of the coefficients than is needed for a sensible local result in the other cases.

Sign in / Sign up

Export Citation Format

Share Document