scholarly journals A New Hybrid Three-Term Conjugate Gradient Algorithm for Large-Scale Unconstrained Problems

Mathematics ◽  
2021 ◽  
Vol 9 (12) ◽  
pp. 1353
Author(s):  
Qi Tian ◽  
Xiaoliang Wang ◽  
Liping Pang ◽  
Mingkun Zhang ◽  
Fanyun Meng
Keyword(s):  
Global Convergence ◽  
Conjugate Gradient ◽  
Large Scale ◽  
Gradient Methods ◽  
Conjugate Gradient Algorithm ◽  
Gradient Algorithm ◽  
Descent Property ◽  
Secant Equation ◽  
Practical Factors ◽  
Unconstrained Problems

Three-term conjugate gradient methods have attracted much attention for large-scale unconstrained problems in recent years, since they have attractive practical factors such as simple computation, low memory requirement, better descent property and strong global convergence property. In this paper, a hybrid three-term conjugate gradient algorithm is proposed and it owns a sufficient descent property, independent of any line search technique. Under some mild conditions, the proposed method is globally convergent for uniformly convex objective functions. Meanwhile, by using the modified secant equation, the proposed method is also global convergence without convexity assumption on the objective function. Numerical results also indicate that the proposed algorithm is more efficient and reliable than the other methods for the testing problems.

scholarly journals Generalized Dai-Yuan conjugate gradient algorithm for training multi-layer feed-forward neural networks

2019 ◽  
Vol 24 (1) ◽  
pp. 115
Author(s):  
Hind H. Mohammed
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Global Convergence ◽  
Conjugate Gradient ◽  
Conjugate Gradient Algorithm ◽  
Gradient Algorithm ◽  
Conjugacy Condition ◽  
Feed Forward ◽  
Descent Property ◽  
Feed Forward Neural Networks

In this paper, we will present different type of CG algorithms depending on Peary conjugacy condition. The new conjugate gradient training (GDY) algorithm using to train MFNNs and prove it's descent property and global convergence for it and then we tested the behavior of this algorithm in the training of artificial neural networks and compared it with known algorithms in this field through two types of issues   http://dx.doi.org/10.25130/tjps.24.2019.020

scholarly journals A Modified Three-Term Type CD Conjugate Gradient Algorithm for Unconstrained Optimization Problems

2020 ◽  
Vol 2020 ◽  
pp. 1-14
Author(s):  
Zhan Wang ◽  
Pengyuan Li ◽  
Xiangrong Li ◽  
Hongtruong Pham
Keyword(s):  
Conjugate Gradient Method ◽  
Conjugate Gradient ◽  
Gradient Method ◽  
Optimization Problems ◽  
Gradient Methods ◽  
Trust Region ◽  
Conjugate Gradient Algorithm ◽  
Gradient Algorithm ◽  
General Function ◽  
Term Type

Conjugate gradient methods are well-known methods which are widely applied in many practical fields. CD conjugate gradient method is one of the classical types. In this paper, a modified three-term type CD conjugate gradient algorithm is proposed. Some good features are presented as follows: (i) A modified three-term type CD conjugate gradient formula is presented. (ii) The given algorithm possesses sufficient descent property and trust region property. (iii) The algorithm has global convergence with the modified weak Wolfe–Powell (MWWP) line search technique and projection technique for general function. The new algorithm has made great progress in numerical experiments. It shows that the modified three-term type CD conjugate gradient method is more competitive than the classical CD conjugate gradient method.

scholarly journals A New Modified Three-Term Hestenes–Stiefel Conjugate Gradient Method with Sufficient Descent Property and Its Global Convergence

2018 ◽  
Vol 2018 ◽  
pp. 1-13 ◽  
Author(s):  
Bakhtawar Baluch ◽  
Zabidin Salleh ◽  
Ahmad Alhawarat
Keyword(s):  
Global Convergence ◽  
Conjugate Gradient Method ◽  
Conjugate Gradient ◽  
Gradient Method ◽  
Line Search ◽  
Gradient Methods ◽  
Conjugate Gradient Methods ◽  
Sufficient Descent Property ◽  
Descent Property ◽  
Sufficient Descent

This paper describes a modified three-term Hestenes–Stiefel (HS) method. The original HS method is the earliest conjugate gradient method. Although the HS method achieves global convergence using an exact line search, this is not guaranteed in the case of an inexact line search. In addition, the HS method does not usually satisfy the descent property. Our modified three-term conjugate gradient method possesses a sufficient descent property regardless of the type of line search and guarantees global convergence using the inexact Wolfe–Powell line search. The numerical efficiency of the modified three-term HS method is checked using 75 standard test functions. It is known that three-term conjugate gradient methods are numerically more efficient than two-term conjugate gradient methods. Importantly, this paper quantifies how much better the three-term performance is compared with two-term methods. Thus, in the numerical results, we compare our new modification with an efficient two-term conjugate gradient method. We also compare our modification with a state-of-the-art three-term HS method. Finally, we conclude that our proposed modification is globally convergent and numerically efficient.

scholarly journals A conjugate gradient algorithm and its application in large-scale optimization problems and image restoration

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Gonglin Yuan ◽  
Tingting Li ◽  
Wujie Hu
Keyword(s):  
Image Restoration ◽  
Conjugate Gradient ◽  
Large Scale ◽  
Optimization Problems ◽  
Trust Region ◽  
Conjugate Gradient Algorithm ◽  
Gradient Algorithm ◽  
Smooth Functions ◽  
Nonsmooth Functions ◽  
Large Scale Unconstrained Optimization

Abstract To solve large-scale unconstrained optimization problems, a modified PRP conjugate gradient algorithm is proposed and is found to be interesting because it combines the steepest descent algorithm with the conjugate gradient method and successfully fully utilizes their excellent properties. For smooth functions, the objective algorithm sufficiently utilizes information about the gradient function and the previous direction to determine the next search direction. For nonsmooth functions, a Moreau–Yosida regularization is introduced into the proposed algorithm, which simplifies the process in addressing complex problems. The proposed algorithm has the following characteristics: (i) a sufficient descent feature as well as a trust region trait; (ii) the ability to achieve global convergence; (iii) numerical results for large-scale smooth/nonsmooth functions prove that the proposed algorithm is outstanding compared to other similar optimization methods; (iv) image restoration problems are done to turn out that the given algorithm is successful.

Sign in / Sign up

Export Citation Format

Share Document