scholarly journals Hybridization rule applied on accelerated double step size optimization scheme

Filomat ◽  
2019 ◽  
Vol 33 (3) ◽  
pp. 655-665 ◽  
Author(s):  
Milena Petrovic
Keyword(s):  
Optimization Problems ◽  
Numerical Test ◽  
Step Length ◽  
Uniformly Convex ◽  
Test Results ◽  
Optimization Scheme ◽  
Step Size ◽  
Double Step ◽  
Accelerated Methods ◽  
Unconstrained Optimization Problems

A hybrid accelerated model with two step length parameters for solving unconstrained optimization problems is presented. Applied hybridization process involves an efficient three term hybrid method. The accelerated double step size model is taken as guiding operator in this hybridization process. Defined method is convergent on the set of uniformly convex functions as well as on the set on strictly convex quadratics. We display a Dolan Mor? performance profiles of derived iteration and of some other comparative hybrid and accelerated methods regarding the number of iterations and the number of function evaluations metrics. Displayed numerical test results confirm that derived model keeps a good properties of its forerunner method and outperform other comparative hybrid accelerated schemes.

scholarly journals On q-BFGS algorithm for unconstrained optimization problems

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Shashi Kant Mishra ◽  
Geetanjali Panda ◽  
Suvra Kanti Chakraborty ◽  
Mohammad Esmael Samei ◽  
Bhagwat Ram
Keyword(s):  
Global Convergence ◽  
Objective Function ◽  
Unconstrained Optimization ◽  
Optimization Problems ◽  
Objective Function Value ◽  
Numerical Test ◽  
Step Length ◽  
Test Problems ◽  
Bfgs Method ◽  
Unconstrained Optimization Problems

AbstractVariants of the Newton method are very popular for solving unconstrained optimization problems. The study on global convergence of the BFGS method has also made good progress. The q-gradient reduces to its classical version when q approaches 1. In this paper, we propose a quantum-Broyden–Fletcher–Goldfarb–Shanno algorithm where the Hessian is constructed using the q-gradient and descent direction is found at each iteration. The algorithm presented in this paper is implemented by applying the independent parameter q in the Armijo–Wolfe conditions to compute the step length which guarantees that the objective function value decreases. The global convergence is established without the convexity assumption on the objective function. Further, the proposed method is verified by the numerical test problems and the results are depicted through the performance profiles.

scholarly journals An Improved Modification of Accelerated Double Direction and Double Step-Size Optimization Schemes

Mathematics ◽  
2022 ◽  
Vol 10 (2) ◽  
pp. 259
Author(s):  
Milena J. Petrović ◽  
Dragana Valjarević ◽  
Dejan Ilić ◽  
Aleksandar Valjarević ◽  
Julija Mladenović
Keyword(s):  
Iterative Method ◽  
Numerical Test ◽  
Unconstrained Minimization ◽  
Test Results ◽  
Step Size ◽  
Double Step ◽  
Gradient Optimization ◽  
Accelerated Gradient ◽  
Optimization Schemes ◽  
Gradient Models

We propose an improved variant of the accelerated gradient optimization models for solving unconstrained minimization problems. Merging the positive features of either double direction, as well as double step size accelerated gradient models, we define an iterative method of a simpler form which is generally more effective. Performed convergence analysis shows that the defined iterative method is at least linearly convergent for uniformly convex and strictly convex functions. Numerical test results confirm the efficiency of the developed model regarding the CPU time, the number of iterations and the number of function evaluations metrics.

scholarly journals A Truncated Descent HS Conjugate Gradient Method and Its Global Convergence

2009 ◽  
Vol 2009 ◽  
pp. 1-13
Author(s):  
Wanyou Cheng ◽  
Zongguo Zhang
Keyword(s):  
Global Convergence ◽  
Line Search ◽  
Optimization Problems ◽  
Test Problems ◽  
Uniformly Convex ◽  
Convex Problems ◽  
Unconstrained Optimization Problems ◽  
Globally Convergent ◽  
Armijo Line Search ◽  
Numerical Performance

Recently, Zhang (2006) proposed a three-term modified HS (TTHS) method for unconstrained optimization problems. An attractive property of the TTHS method is that the direction generated by the method is always descent. This property is independent of the line search used. In order to obtain the global convergence of the TTHS method, Zhang proposed a truncated TTHS method. A drawback is that the numerical performance of the truncated TTHS method is not ideal. In this paper, we prove that the TTHS method with standard Armijo line search is globally convergent for uniformly convex problems. Moreover, we propose a new truncated TTHS method. Under suitable conditions, global convergence is obtained for the proposed method. Extensive numerical experiment show that the proposed method is very efficient for the test problems from the CUTE Library.

scholarly journals A Transformation of Accelerated Double Step Size Method for Unconstrained Optimization

2015 ◽  
Vol 2015 ◽  
pp. 1-8 ◽  
Author(s):  
Predrag S. Stanimirović ◽  
Gradimir V. Milovanović ◽  
Milena J. Petrović ◽  
Nataša Z. Kontrec
Keyword(s):  
Gradient Descent ◽  
Linear Convergence ◽  
Step Length ◽  
Descent Method ◽  
Single Step ◽  
Gradient Descent Method ◽  
Step Size ◽  
Double Step ◽  
Substantial Progress ◽  
Accelerated Gradient

A reduction of the originally double step size iteration into the single step length scheme is derived under the proposed condition that relates two step lengths in the accelerated double step size gradient descent scheme. The proposed transformation is numerically tested. Obtained results confirm the substantial progress in comparison with the single step size accelerated gradient descent method defined in a classical way regarding all analyzed characteristics: number of iterations, CPU time, and number of function evaluations. Linear convergence of derived method has been proved.

scholarly journals Concurrent Modified Constant Modulus Algorithm and Decision Directed Scheme with Barzilai-Borwein Method

2020 ◽  
Author(s):  
Tongtong Xu ◽  
Zheng Xiang
Keyword(s):  
Theoretical Analysis ◽  
Gradient Method ◽  
Optimization Problems ◽  
Constant Modulus ◽  
Constant Modulus Algorithm ◽  
Step Size ◽  
Unconstrained Optimization Problems ◽  
Communications Systems ◽  
Simulation Results ◽  
Wireless Communications Systems

Abstract In this work, concurrent modified constant modulus algorithm and decision-directed scheme with the Barzilai-Borwein method is proposed for blind equalization of wireless communications systems. The Barzilai-Borwein method, the two-step gradient method, is usually used to solve multidimensional unconstrained optimization problems. The proposed algorithm concurrently operates a modified constant modulus algorithm equalizer and a decision directed equalizer, it then adaptively adjusts the step size of the decision directed equalizer using Barzilai-Borwein method. Theoretical analysis is provided to illustrate that the proposed algorithm has a faster convergence speed, and better equalization performance than the original one. Simulation results support the new proposed technique.

scholarly journals IFA-EO: An Improved Firefly Algorithm Hybridized With Extremal Optimization for Continuous Unconstrained Optimization Problems

2021 ◽  
Author(s):  
Min-Rong Chen ◽  
Liu-Qing Yang ◽  
Guo-Qiang Zeng ◽  
Kang-Di Lu ◽  
Yi-Yuan Huang
Keyword(s):  
Firefly Algorithm ◽  
Optimization Problems ◽  
Local Optimum ◽  
Step Size ◽  
Extremal Optimization ◽  
Unconstrained Optimization Problems ◽  
Improved Firefly Algorithm ◽  
Attraction Model ◽  
Complex Optimization ◽  
Adaptive Step

Abstract As one of the evolutionary algorithms, firefly algorithm (FA) has been widely used to solve various complex optimization problems. However, FA has significant drawbacks in slow convergence rate and is easily trapped into local optimum. To tackle these defects, this paper proposes an improved FA combined with extremal optimization (EO), named IFA-EO, where three strategies are incorporated. First, to balance the tradeoff between exploration ability and exploitation ability, we adopt a new attraction model for FA operation, which combines the full attraction model and the single attraction model through the probability choice strategy. In the single attraction model, small probability accepts the worse solution to improve the diversity of the offspring. Second, the adaptive step size is proposed based on the number of iterations to dynamically adjust the attention to the exploration model or exploitation model. Third, we combine an EO algorithm with powerful ability in local-search into FA. Experiments are tested on two group popular benchmarks including complex unimodal and multimodal functions. Our experimental results demonstrate that the proposed IFA-EO algorithm can deal with various complex optimization problems and has similar or better performance than the other eight FA variants, three EO-based algorithms, and one advanced differential evolution variant in terms of accuracy and statistical results.

Accelerated multiple step-size methods for solving unconstrained optimization problems

2019 ◽  
pp. 1-32
Author(s):  
Branislav Ivanov ◽  
Predrag S. Stanimirović ◽  
Gradimir V. Milovanović ◽  
Snežana Djordjević ◽  
Ivona Brajević
Keyword(s):  
Unconstrained Optimization ◽  
Optimization Problems ◽  
Step Size ◽  
Unconstrained Optimization Problems ◽  
Multiple Step

scholarly journals Global Convergence of a Modified Spectral Conjugate Gradient Method

2012 ◽  
Vol 2012 ◽  
pp. 1-13 ◽  
Author(s):  
Huabin Jiang ◽  
Songhai Deng ◽  
Xiaodong Zheng ◽  
Zhong Wan
Keyword(s):  
Global Convergence ◽  
Conjugate Gradient Method ◽  
Conjugate Gradient ◽  
Gradient Method ◽  
Line Search ◽  
Optimization Problems ◽  
Step Length ◽  
Mild Conditions ◽  
Unconstrained Optimization Problems ◽  
Spectral Conjugate Gradient

A modified spectral PRP conjugate gradient method is presented for solving unconstrained optimization problems. The constructed search direction is proved to be a sufficiently descent direction of the objective function. With an Armijo-type line search to determinate the step length, a new spectral PRP conjugate algorithm is developed. Under some mild conditions, the theory of global convergence is established. Numerical results demonstrate that this algorithm is promising, particularly, compared with the existing similar ones.

scholarly journals Concurrent Modified Constant Modulus Algorithm and Decision Directed Scheme With Barzilai-Borwein Method

2021 ◽  
Vol 15 ◽  
Author(s):  
Tongtong Xu ◽  
Zheng Xiang ◽  
Hua Yang ◽  
Yun Chen ◽  
Jun Luo ◽  
...  
Keyword(s):  
Wireless Communication ◽  
Optimization Problems ◽  
Blind Equalization ◽  
Wireless Channel ◽  
Constant Modulus ◽  
Constant Modulus Algorithm ◽  
Step Size ◽  
Pilot Signal ◽  
Unconstrained Optimization Problems ◽  
Wireless Communications Systems

At present, in robot technology, remote control of robot is realized by wireless communication technology, and data anti-interference in wireless channel becomes a very important part. Any wireless communication system has an inherent multi-path propagation problem, which leads to the expansion of generated symbols on a time scale, resulting in symbol overlap and Inter-symbol Interference (ISI). ISI in the signal must be removed and the signal restores to its original state at the time of transmission or becomes as close to it as possible. Blind equalization is a popular equalization method for recovering transmitted symbols of superimposed noise without any pilot signal. In this work, we propose a concurrent modified constant modulus algorithm (MCMA) and the decision-directed scheme (DDS) with the Barzilai-Borwein (BB) method for the purpose of blind equalization of wireless communications systems (WCS). The BB method, which is two-step gradient method, has been widely employed to solve multidimensional unconstrained optimization problems. Considering the similarity of equalization process and optimization process, the proposed algorithm combines existing blind equalization algorithm and Barzilai-Borwein method, and concurrently operates a MCMA equalizer and a DD equalizer. After that, it modifies the DD equalizer's step size (SS) by the BB method. Theoretical investigation was involved and it demonstrated rapid convergence and improved equalization performance of the proposed algorithm compared with the original one. Additionally, the simulation results were consistent with the proposed technique.

Sign in / Sign up

Export Citation Format

Share Document