scholarly journals Spectral proximal method for solving large scale sparse optimization

2021 ◽  
Vol 36 ◽  
pp. 04007
Author(s):  
Gillian Yi Han Woo ◽  
Hong Seng Sim ◽  
Yong Kheng Goh ◽  
Wah June Leong
Keyword(s):  
Unconstrained Optimization ◽  
Gradient Method ◽  
Large Scale ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Sparse Optimization ◽  
Computational Time ◽  
Proximal Method ◽  
Unconstrained Optimization Problem ◽  
Spectral Gradient Method

In this paper, we propose to use spectral proximal method to solve sparse optimization problems. Sparse optimization refers to an optimization problem involving the ι0 -norm in objective or constraints. The previous research showed that the spectral gradient method is outperformed the other standard unconstrained optimization methods. This is due to spectral gradient method replaced the full rank matrix by a diagonal matrix and the memory decreased from Ο(n2) to Ο(n). Since ι0-norm term is nonconvex and non-smooth, it cannot be solved by standard optimization algorithm. We will solve the ι0 -norm problem with an underdetermined system as its constraint will be considered. Using Lagrange method, this problem is transformed into an unconstrained optimization problem. A new method called spectral proximal method is proposed, which is a combination of proximal method and spectral gradient method. The spectral proximal method is then applied to the ι0-norm unconstrained optimization problem. The programming code will be written in Python to compare the efficiency of the proposed method with some existing methods. The benchmarks of the comparison are based on number of iterations, number of functions call and the computational time. Theoretically, the proposed method requires less storage and less computational time.

scholarly journals An Efficient Modified AZPRP Conjugate Gradient Method for Large-Scale Unconstrained Optimization Problem

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Ahmad Alhawarat ◽  
Thoi Trung Nguyen ◽  
Ramadan Sabra ◽  
Zabidin Salleh
Keyword(s):  
Unconstrained Optimization ◽  
Conjugate Gradient ◽  
Large Scale ◽  
Line Search ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Unconstrained Optimization Problems ◽  
Unconstrained Optimization Problem ◽  
Descent Property ◽  
Large Scale Unconstrained Optimization

To find a solution of unconstrained optimization problems, we normally use a conjugate gradient (CG) method since it does not cost memory or storage of second derivative like Newton’s method or Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. Recently, a new modification of Polak and Ribiere method was proposed with new restart condition to give a so-call AZPRP method. In this paper, we propose a new modification of AZPRP CG method to solve large-scale unconstrained optimization problems based on a modification of restart condition. The new parameter satisfies the descent property and the global convergence analysis with the strong Wolfe-Powell line search. The numerical results prove that the new CG method is strongly aggressive compared with CG_Descent method. The comparisons are made under a set of more than 140 standard functions from the CUTEst library. The comparison includes number of iterations and CPU time.

scholarly journals Nonmonotone spectral gradient method based on memoryless symmetric rank-one update for large-scale unconstrained optimization

2021 ◽  
Vol 0 (0) ◽  
pp. 0
Author(s):  
Hong Seng Sim ◽  
Chuei Yee Chen ◽  
Wah June Leong ◽  
Jiao Li
Keyword(s):  
Unconstrained Optimization ◽  
Spectral Parameter ◽  
Gradient Method ◽  
Large Scale ◽  
Line Search ◽  
Optimization Problems ◽  
Rank One ◽  
Spectral Gradient Method ◽  
Symmetric Rank One ◽  
Large Scale Unconstrained Optimization

<p style='text-indent:20px;'>This paper proposes a nonmonotone spectral gradient method for solving large-scale unconstrained optimization problems. The spectral parameter is derived from the eigenvalues of an optimally sized memoryless symmetric rank-one matrix obtained under the measure defined as a ratio of the determinant of updating matrix over its largest eigenvalue. Coupled with a nonmonotone line search strategy where backtracking-type line search is applied selectively, the spectral parameter acts as a stepsize during iterations when no line search is performed and as a milder form of quasi-Newton update when backtracking line search is employed. Convergence properties of the proposed method are established for uniformly convex functions. Extensive numerical experiments are conducted and the results indicate that our proposed spectral gradient method outperforms some standard conjugate-gradient methods.</p>

scholarly journals GENO – Optimization for Classical Machine Learning Made Fast and Easy

2020 ◽  
Vol 34 (09) ◽  
pp. 13620-13621
Author(s):  
Sören Laue ◽  
Matthias Mitterreiter ◽  
Joachim Giesen
Keyword(s):  
Machine Learning ◽  
Unconstrained Optimization ◽  
Optimization Problem ◽  
Specific Problem ◽  
Optimization Problems ◽  
Modeling Language ◽  
Large Margin ◽  
Unconstrained Optimization Problem

Most problems from classical machine learning can be cast as an optimization problem. We introduce GENO (GENeric Optimization), a framework that lets the user specify a constrained or unconstrained optimization problem in an easy-to-read modeling language. GENO then generates a solver, i.e., Python code, that can solve this class of optimization problems. The generated solver is usually as fast as hand-written, problem-specific, and well-engineered solvers. Often the solvers generated by GENO are faster by a large margin compared to recently developed solvers that are tailored to a specific problem class.An online interface to our framework can be found at http://www.geno-project.org.

scholarly journals A modification of steepest descent method for solving large-scaled unconstrained optimization problems

2018 ◽  
Vol 7 (3.28) ◽  
pp. 72
Author(s):  
Siti Farhana Husin ◽  
Mustafa Mamat ◽  
Mohd Asrul Hery Ibrahim ◽  
Mohd Rivaie
Keyword(s):  
Unconstrained Optimization ◽  
Large Scale ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Descent Method ◽  
Steepest Descent ◽  
Steepest Descent Method ◽  
Search Direction ◽  
Exact Line Search ◽  
Scale Optimization

In this paper, we develop a new search direction for Steepest Descent (SD) method by replacing previous search direction from Conjugate Gradient (CG) method, , with gradient from the previous step,  for solving large-scale optimization problem. We also used one of the conjugate coefficient as a coefficient for matrix . Under some reasonable assumptions, we prove that the proposed method with exact line search satisfies descent property and possesses the globally convergent. Further, the numerical results on some unconstrained optimization problem show that the proposed algorithm is promising. 

scholarly journals A Modified Liu and Storey Conjugate Gradient Method for Large Scale Unconstrained Optimization Problems

Algorithms ◽  
2021 ◽  
Vol 14 (8) ◽  
pp. 227
Author(s):  
Zabidin Salleh ◽  
Ghaliah Alhamzi ◽  
Ibitsam Masmali ◽  
Ahmad Alhawarat
Keyword(s):  
Unconstrained Optimization ◽  
Conjugate Gradient Method ◽  
Conjugate Gradient ◽  
Gradient Method ◽  
Large Scale ◽  
Optimization Problems ◽  
Numerical Results ◽  
New Method ◽  
Unconstrained Optimization Problems ◽  
Large Scale Unconstrained Optimization

The conjugate gradient method is one of the most popular methods to solve large-scale unconstrained optimization problems since it does not require the second derivative, such as Newton’s method or approximations. Moreover, the conjugate gradient method can be applied in many fields such as neural networks, image restoration, etc. Many complicated methods are proposed to solve these optimization functions in two or three terms. In this paper, we propose a simple, easy, efficient, and robust conjugate gradient method. The new method is constructed based on the Liu and Storey method to overcome the convergence problem and descent property. The new modified method satisfies the convergence properties and the sufficient descent condition under some assumptions. The numerical results show that the new method outperforms famous CG methods such as CG-Descent5.3, Liu and Storey, and Dai and Liao. The numerical results include the number of iterations and CPU time.

scholarly journals Solving a large scale nonlinear unconstrained optimization problems by using new coefficient of conjugate gradient method with exact line search direction

2018 ◽  
Vol 7 (3.28) ◽  
pp. 92
Author(s):  
Talat Alkouli ◽  
Mustafa Mamat ◽  
Mohd Rivaie ◽  
Puspa Liza Ghazali
Keyword(s):  
Unconstrained Optimization ◽  
Conjugate Gradient Method ◽  
Conjugate Gradient ◽  
Gradient Method ◽  
Large Scale ◽  
Line Search ◽  
Optimization Problems ◽  
Exact Line Search ◽  
Unconstrained Optimization Problems ◽  
Large Scale Unconstrained Optimization

In this paper, an efficient modification of nonlinear conjugate gradient method and an associated implementation, based on an exact line search, are proposed and analyzed to solve large-scale unconstrained optimization problems. The method satisfies the sufficient descent property. Furthermore, global convergence result is proved. Computational results for a set of unconstrained optimization test problems, some of them from CUTE library, showed that this new conjugate gradient algorithm seems to converge more stable and outperforms the other similar methods in many situations.   

Sign in / Sign up

Export Citation Format

Share Document