scholarly journals Convergence Analysis of an Improved BFGS Method and Its Application in the Muskingum Model

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Tianshan Yang ◽  
Pengyuan Li ◽  
Xiaoliang Wang
Keyword(s):  
Convergence Analysis ◽  
Line Search ◽  
Optimization Problems ◽  
Test Problems ◽  
Bfgs Method ◽  
Muskingum Model ◽  
Convex Minimization Problems ◽  
Line Search Technique ◽  
Quasi Newton ◽  
Academic Test

The BFGS method is one of the most effective quasi-Newton algorithms for minimization-optimization problems. In this paper, an improved BFGS method with a modified weak Wolfe–Powell line search technique is used to solve convex minimization problems and its convergence analysis is established. Seventy-four academic test problems and the Muskingum model are implemented in the numerical experiment. The numerical results show that our algorithm is comparable to the usual BFGS algorithm in terms of the number of iterations and the time consumed, which indicates our algorithm is effective and reliable.

scholarly journals Global Convergence of a Modified Two-Parameter Scaled BFGS Method with Yuan-Wei-Lu Line Search for Unconstrained Optimization

2020 ◽  
Vol 2020 ◽  
pp. 1-15
Author(s):  
Pengyuan Li ◽  
Zhan Wang ◽  
Dan Luo ◽  
Hongtruong Pham
Keyword(s):  
Unconstrained Optimization ◽  
Line Search ◽  
Optimization Problems ◽  
Medium Size ◽  
Bfgs Method ◽  
Nonconvex Functions ◽  
Bfgs Update ◽  
Unconstrained Optimization Problems ◽  
Quasi Newton ◽  
Two Parameter

The BFGS method is one of the most efficient quasi-Newton methods for solving small- and medium-size unconstrained optimization problems. For the sake of exploring its more interesting properties, a modified two-parameter scaled BFGS method is stated in this paper. The intention of the modified scaled BFGS method is to improve the eigenvalues structure of the BFGS update. In this method, the first two terms and the last term of the standard BFGS update formula are scaled with two different positive parameters, and the new value of yk is given. Meanwhile, Yuan-Wei-Lu line search is also proposed. Under the mentioned line search, the modified two-parameter scaled BFGS method is globally convergent for nonconvex functions. The extensive numerical experiments show that this form of the scaled BFGS method outperforms the standard BFGS method or some similar scaled methods.

scholarly journals Comparative assessment of smooth and non-smooth optimization solvers in HANSO software

2021 ◽  
Vol 12 (1) ◽  
pp. 39-46
Author(s):  
Ali Hakan Tor
Keyword(s):  
Nonsmooth Optimization ◽  
Hybrid Algorithm ◽  
Optimization Problems ◽  
Optimization Methods ◽  
Comparative Assessment ◽  
Test Problems ◽  
Bfgs Method ◽  
Smooth Optimization ◽  
Optimization Solvers ◽  
Academic Test

The aim of this study is to compare the performance of smooth and nonsmooth optimization solvers from HANSO (Hybrid Algorithm for Nonsmooth Optimization) software. The smooth optimization solver is the implementation of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method and the nonsmooth optimization solver is the Hybrid Algorithm for Nonsmooth Optimization. More precisely, the nonsmooth optimization algorithm is the combination of the BFGS and the Gradient Sampling Algorithm (GSA). We use well-known collection of academic test problems for nonsmooth optimization containing both convex and nonconvex problems. The motivation for this research is the importance of the comparative assessment of smooth optimization methods for solving nonsmooth optimization problems. This assessment will demonstrate how successful is the BFGS method for solving nonsmooth optimization problems in comparison with the nonsmooth optimization solver from HANSO. Performance profiles using the number iterations, the number of function evaluations and the number of subgradient evaluations are used to compare solvers.

scholarly journals A Nonmonotone Weighting Self-Adaptive Trust Region Algorithm for Unconstrained Nonconvex Optimization

2015 ◽  
Vol 2015 ◽  
pp. 1-8
Author(s):  
Yunlong Lu ◽  
Weiwei Yang ◽  
Wenyu Li ◽  
Xiaowei Jiang ◽  
Yueting Yang
Keyword(s):  
Nonconvex Optimization ◽  
Line Search ◽  
Optimization Problems ◽  
Trust Region Method ◽  
Trust Region ◽  
Search Technique ◽  
Trust Region Algorithm ◽  
Unconstrained Optimization Problems ◽  
Line Search Technique ◽  
Self Adaptive

A new trust region method is presented, which combines nonmonotone line search technique, a self-adaptive update rule for the trust region radius, and the weighting technique for the ratio between the actual reduction and the predicted reduction. Under reasonable assumptions, the global convergence of the method is established for unconstrained nonconvex optimization. Numerical results show that the new method is efficient and robust for solving unconstrained optimization problems.

A new three-term spectral conjugate gradient algorithm with higher numerical performance for solving large scale optimization problems based on Quasi-Newton equation

2021 ◽  
pp. 2150053
Author(s):  
Jie Guo ◽  
Zhong Wan
Keyword(s):  
Conjugate Gradient ◽  
Large Scale ◽  
Line Search ◽  
Optimization Problems ◽  
Gradient Algorithm ◽  
Large Scale Optimization ◽  
Newton Equation ◽  
Numerical Performance ◽  
Scale Optimization ◽  
Quasi Newton

A new spectral three-term conjugate gradient algorithm in virtue of the Quasi-Newton equation is developed for solving large-scale unconstrained optimization problems. It is proved that the search directions in this algorithm always satisfy a sufficiently descent condition independent of any line search. Global convergence is established for general objective functions if the strong Wolfe line search is used. Numerical experiments are employed to show its high numerical performance in solving large-scale optimization problems. Particularly, the developed algorithm is implemented to solve the 100 benchmark test problems from CUTE with different sizes from 1000 to 10,000, in comparison with some similar ones in the literature. The numerical results demonstrate that our algorithm outperforms the state-of-the-art ones in terms of less CPU time, less number of iteration or less number of function evaluation.

scholarly journals A randomized adaptive trust region line search method

2020 ◽  
Vol 10 (2) ◽  
pp. 259-263
Author(s):  
Saman Babaie-Kafaki ◽  
Saeed Rezaee
Keyword(s):  
Simulated Annealing ◽  
Numerical Experiments ◽  
Line Search ◽  
Optimization Problems ◽  
Trust Region ◽  
Search Method ◽  
Test Problems ◽  
Unconstrained Optimization Problems ◽  
Line Search Method ◽  
Region Line

Hybridizing the trust region, line search and simulated annealing methods, we develop a heuristic algorithm for solving unconstrained optimization problems. We make some numerical experiments on a set of CUTEr test problems to investigate efficiency of the suggested algorithm. The results show that the algorithm is practically promising.

A Simulated Annealing-Based Barzilai–Borwein Gradient Method for Unconstrained Optimization Problems

2019 ◽  
Vol 36 (04) ◽  
pp. 1950017 ◽  
Author(s):  
Wen-Li Dong ◽  
Xing Li ◽  
Zheng Peng
Keyword(s):  
Simulated Annealing ◽  
Unconstrained Optimization ◽  
Gradient Method ◽  
Line Search ◽  
High Efficiency ◽  
Optimization Problems ◽  
Search Technique ◽  
Unconstrained Optimization Problems ◽  
Armijo Line Search ◽  
Line Search Technique

In this paper, we propose a simulated annealing-based Barzilai–Borwein (SABB) gradient method for unconstrained optimization problems. The SABB method accepts the Barzilai–Borwein (BB) step by a simulated annealing rule. If the BB step cannot be accepted, the Armijo line search is used. The global convergence of the SABB method is established under some mild conditions. Numerical experiments indicate that, compared to some existing BB methods using nonmonotone line search technique, the SABB method performs well with high efficiency.

scholarly journals A Truncated Descent HS Conjugate Gradient Method and Its Global Convergence

2009 ◽  
Vol 2009 ◽  
pp. 1-13
Author(s):  
Wanyou Cheng ◽  
Zongguo Zhang
Keyword(s):  
Global Convergence ◽  
Line Search ◽  
Optimization Problems ◽  
Test Problems ◽  
Uniformly Convex ◽  
Convex Problems ◽  
Unconstrained Optimization Problems ◽  
Globally Convergent ◽  
Armijo Line Search ◽  
Numerical Performance

Recently, Zhang (2006) proposed a three-term modified HS (TTHS) method for unconstrained optimization problems. An attractive property of the TTHS method is that the direction generated by the method is always descent. This property is independent of the line search used. In order to obtain the global convergence of the TTHS method, Zhang proposed a truncated TTHS method. A drawback is that the numerical performance of the truncated TTHS method is not ideal. In this paper, we prove that the TTHS method with standard Armijo line search is globally convergent for uniformly convex problems. Moreover, we propose a new truncated TTHS method. Under suitable conditions, global convergence is obtained for the proposed method. Extensive numerical experiment show that the proposed method is very efficient for the test problems from the CUTE Library.

A Nonmonotone Modified BFGS Method for Non-Convex Minimization

2014 ◽  
Vol 556-562 ◽  
pp. 4023-4026
Author(s):  
Ting Feng Li ◽  
Zhi Yuan Liu ◽  
Sheng Hui Yan
Keyword(s):  
Large Scale ◽  
Line Search ◽  
Optimization Problems ◽  
Convergence Property ◽  
Bfgs Method ◽  
Remarkable Feature ◽  
Convexity Assumption ◽  
Unconstrained Optimization Problems ◽  
Global Convergence Property ◽  
Large Scale Unconstrained Optimization

In this paper, a modification BFGS method with nonmonotone line-search for solving large-scale unconstrained optimization problems is proposed. A remarkable feature of the proposed method is that it possesses a global convergence property even without convexity assumption on the objective function. Some numerical results are reported which illustrate that the proposed method is efficient

scholarly journals A Line-Search-Based Partial Proximal Alternating Directions Method for Separable Convex Optimization

2014 ◽  
Vol 2014 ◽  
pp. 1-8
Author(s):  
Yu-hua Zeng ◽  
Yu-fei Yang ◽  
Zheng Peng
Keyword(s):  
Convex Optimization ◽  
Line Search ◽  
Optimization Problems ◽  
Proximal Point Method ◽  
Search Technique ◽  
Proximal Point ◽  
Separable Convex Optimization ◽  
Line Search Technique ◽  
Alternating Directions Method ◽  
Proximal Term

We propose an appealing line-search-based partial proximal alternating directions (LSPPAD) method for solving a class of separable convex optimization problems. These problems under consideration are common in practice. The proposed method solves two subproblems at each iteration: one is solved by a proximal point method, while the proximal term is absent from the other. Both subproblems admit inexact solutions. A line search technique is used to guarantee the convergence. The convergence of the LSPPAD method is established under some suitable conditions. The advantage of the proposed method is that it provides the tractability of the subproblem in which the proximal term is absent. Numerical tests show that the LSPPAD method has better performance compared with the existing alternating projection based prediction-correction (APBPC) method if both are employed to solve the described problem.

Nonmonotone conic trust region method with line search technique for bound constrained optimization

2019 ◽  
Vol 53 (3) ◽  
pp. 787-805
Author(s):  
Lijuan Zhao
Keyword(s):  
Constrained Optimization ◽  
Line Search ◽  
Optimization Problems ◽  
Trust Region Method ◽  
Trust Region ◽  
Affine Scaling ◽  
Search Technique ◽  
Region Method ◽  
Bound Constrained Optimization ◽  
Line Search Technique

In this paper, we propose a nonmonotone trust region method for bound constrained optimization problems, where the bounds are dealt with by affine scaling technique. Differing from the traditional trust region methods, the subproblem in our algorithm is based on a conic model. Moreover, when the trial point isn’t acceptable by the usual trust region criterion, a line search technique is used to find an acceptable point. This procedure avoids resolving the trust region subproblem, which may reduce the total computational cost. The global convergence and Q-superlinear convergence of the algorithm are established under some mild conditions. Numerical results on a series of standard test problems are reported to show the effectiveness of the new method.

Sign in / Sign up

Export Citation Format

Share Document