A Gradient-Based Globalization Strategy for the Newton Method

2020 ◽  
pp. 177-185
Author(s):  
Daniela di Serafino ◽  
Gerardo Toraldo ◽  
Marco Viola
Keyword(s):  
Newton Method ◽  
Gradient Based ◽  
Globalization Strategy

A non-smooth Newton method for the solution of magnetostatic field problems with hysteresis

2019 ◽  
Vol 38 (5) ◽  
pp. 1584-1594 ◽  
Author(s):  
Stephan Willerich ◽  
Hans-Georg Herzog
Keyword(s):  
Finite Element ◽  
Newton Method ◽  
Linear Equations ◽  
Element Formulation ◽  
Magnetostatic Field ◽  
Generalized Jacobian ◽  
Content Type ◽  
Non Linear ◽  
Generalized Framework ◽  
Gradient Based

Purpose The use of gradient-based methods in finite element schemes can be prevented by undefined derivatives, which are encountered when modeling hysteresis in constitutive material laws. This paper aims to present a method to deal with this problem. Design/methodology/approach Non-smooth Newton methods provide a generalized framework for the treatment of minimization problems with undefined derivatives. Within this paper, a magnetostatic finite element formulation that includes hysteresis is presented. The non-linear equations are solved using a non-smooth Newton method. Findings The non-smooth Newton method shows promising convergence behavior when applied to a model problem. The numbers of iterations for magnetization curves with and without hysteresis are within the same range. Originality/value Mathematical tools like Clarke's generalized Jacobian are applied to magnetostatic field problems with hysteresis. The relation between the non-smooth Newton method and other methods for solving non-linear systems with hysteresis like the M(B)-iteration is established.

scholarly journals Accelerating Symmetric Rank-1 Quasi-Newton Method with Nesterov’s Gradient for Training Neural Networks

2021 ◽  
Author(s):  
S. Indrapriyadarsini ◽  
Shahrzad Mahboubi ◽  
Hiroshi Ninomiya ◽  
Takeshi Kamio ◽  
Hideki Asai
Keyword(s):  
Neural Networks ◽  
Newton Method ◽  
Trust Region ◽  
Nonlinear Problems ◽  
Classification Problem ◽  
Second Order ◽  
Highly Nonlinear ◽  
Gradient Based ◽  
Quasi Newton ◽  
Second Order Methods

Gradient based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method though less commonly used in training neural networks, are known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks and briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

scholarly journals Accelerating Symmetric Rank-1 Quasi-Newton Method with Nesterov’s Gradient for Training Neural Networks

2021 ◽  
Author(s):  
S. Indrapriyadarsini ◽  
Shahrzad Mahboubi ◽  
Hiroshi Ninomiya ◽  
Takeshi Kamio ◽  
Hideki Asai
Keyword(s):  
Neural Networks ◽  
Newton Method ◽  
Trust Region ◽  
Nonlinear Problems ◽  
Classification Problem ◽  
Second Order ◽  
Highly Nonlinear ◽  
Gradient Based ◽  
Quasi Newton ◽  
Second Order Methods

Gradient based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method though less commonly used in training neural networks, are known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks and briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

scholarly journals Accelerating Symmetric Rank-1 Quasi-Newton Method with Nesterov’s Gradient for Training Neural Networks

Algorithms ◽  
2021 ◽  
Vol 15 (1) ◽  
pp. 6
Author(s):  
S. Indrapriyadarsini ◽  
Shahrzad Mahboubi ◽  
Hiroshi Ninomiya ◽  
Takeshi Kamio ◽  
Hideki Asai
Keyword(s):  
Neural Networks ◽  
Newton Method ◽  
Trust Region ◽  
Nonlinear Problems ◽  
Classification Problem ◽  
Second Order ◽  
Highly Nonlinear ◽  
Gradient Based ◽  
Quasi Newton ◽  
Second Order Methods

Gradient-based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have been shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method, though less commonly used in training neural networks, is known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks, and to briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

scholarly journals A skew gradient-based Newton method for traffic assignment with side constraints

2007 ◽  
Vol 12 (2) ◽  
pp. 184-191
Author(s):  
Lin Cheng ◽  
Wei Wang ◽  
Zhijian Zhu ◽  
Chunqing Yu
Keyword(s):  
Newton Method ◽  
Traffic Assignment ◽  
Side Constraints ◽  
Gradient Based

Gradient based system-level diagnosis

2007 ◽  
Vol 51 (1-2) ◽  
pp. 43
Author(s):  
Balázs Polgár ◽  
Endre Selényi
Keyword(s):  
System Level ◽  
Gradient Based

An Efficient Multiple Description Coding for Multi-View Video Based on the Correlation of Spatial Polyphase Transformed Subsequences

2019 ◽  
Vol 63 (5) ◽  
pp. 50401-1-50401-7 ◽  
Author(s):  
Jing Chen ◽  
Jie Liao ◽  
Huanqiang Zeng ◽  
Canhui Cai ◽  
Kai-Kuang Ma
Keyword(s):  
Video Transmission ◽  
Video Sequence ◽  
Three Dimensional ◽  
The Other ◽  
Multiple Description Coding ◽  
Multiple Description ◽  
Prediction Mode ◽  
Gradient Based ◽  
Directional Interpolation ◽  
Description Coding

Abstract For a robust three-dimensional video transmission through error prone channels, an efficient multiple description coding for multi-view video based on the correlation of spatial polyphase transformed subsequences (CSPT_MDC_MVC) is proposed in this article. The input multi-view video sequence is first separated into four subsequences by spatial polyphase transform and then grouped into two descriptions. With the correlation of macroblocks in corresponding subsequence positions, these subsequences should not be coded in completely the same way. In each description, one subsequence is directly coded by the Joint Multi-view Video Coding (JMVC) encoder and the other subsequence is classified into four sets. According to the classification, the indirectly coding subsequence selectively employed the prediction mode and the prediction vector of the counter directly coding subsequence, which reduces the bitrate consumption and the coding complexity of multiple description coding for multi-view video. On the decoder side, the gradient-based directional interpolation is employed to improve the side reconstructed quality. The effectiveness and robustness of the proposed algorithm is verified by experiments in the JMVC coding platform.

Sign in / Sign up

Export Citation Format

Share Document