scholarly journals An algorithm for LC1 optimization

2005 ◽  
Vol 15 (2) ◽  
pp. 301-306 ◽  
Author(s):  
Nada Djuranovic-Milicic
Keyword(s):  
Rate Of Convergence ◽  
Unconstrained Optimization ◽  
Optimization Problems ◽  
Directional Derivative ◽  
Second Order ◽  
General Algorithm ◽  
Unconstrained Optimization Problems ◽  
Convergence Proof

In this paper an algorithm for LC1 unconstrained optimization problems, which uses the second order Dini upper directional derivative is considered. The purpose of the paper is to establish general algorithm hypotheses under which convergence occurs to optimal points. A convergence proof is given, as well as an estimate of the rate of convergence.

First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method

1992 ◽  
Vol 4 (2) ◽  
pp. 141-166 ◽  
Author(s):  
Roberto Battiti
Keyword(s):  
Large Scale ◽  
Feedforward Neural Networks ◽  
Optimization Methods ◽  
Second Order ◽  
Optimization Techniques ◽  
Neural Nets ◽  
Classification Problems ◽  
On Line ◽  
Mutual Relations ◽  
Safety Procedures

On-line first-order backpropagation is sufficiently fast and effective for many large-scale classification problems but for very high precision mappings, batch processing may be the method of choice. This paper reviews first- and second-order optimization methods for learning in feedforward neural networks. The viewpoint is that of optimization: many methods can be cast in the language of optimization techniques, allowing the transfer to neural nets of detailed results about computational complexity and safety procedures to ensure convergence and to avoid numerical problems. The review is not intended to deliver detailed prescriptions for the most appropriate methods in specific applications, but to illustrate the main characteristics of the different methods and their mutual relations.

Sign in / Sign up

Export Citation Format

Share Document