scholarly journals Variable Smoothing for Convex Optimization Problems Using Stochastic Gradients

2020 ◽  
Vol 85 (2) ◽  
Author(s):  
Radu Ioan Boţ ◽  
Axel Böhm
Keyword(s):  
Convex Optimization ◽  
Linear Operator ◽  
Large Scale ◽  
Optimization Problems ◽  
Convex Optimization Problem ◽  
Nonsmooth Function ◽  
Large Scale Problems ◽  
Structured Convex Optimization ◽  
Primal Dual ◽  
Novel Algorithms

AbstractWe aim to solve a structured convex optimization problem, where a nonsmooth function is composed with a linear operator. When opting for full splitting schemes, usually, primal–dual type methods are employed as they are effective and also well studied. However, under the additional assumption of Lipschitz continuity of the nonsmooth function which is composed with the linear operator we can derive novel algorithms through regularization via the Moreau envelope. Furthermore, we tackle large scale problems by means of stochastic oracle calls, very similar to stochastic gradient techniques. Applications to total variational denoising and deblurring, and matrix factorization are provided.

scholarly journals Investigation of Improved Cooperative Coevolution for Large-Scale Global Optimization Problems

Algorithms ◽  
2021 ◽  
Vol 14 (5) ◽  
pp. 146
Author(s):  
Aleksei Vakhnin ◽  
Evgenii Sopov
Keyword(s):  
Global Optimization ◽  
Evolutionary Algorithms ◽  
Numerical Experiments ◽  
Large Scale ◽  
Optimization Problems ◽  
State Of The Art ◽  
Fixed Number ◽  
High Dimensional ◽  
Cooperative Coevolution ◽  
Large Scale Problems

Modern real-valued optimization problems are complex and high-dimensional, and they are known as “large-scale global optimization (LSGO)” problems. Classic evolutionary algorithms (EAs) perform poorly on this class of problems because of the curse of dimensionality. Cooperative Coevolution (CC) is a high-performed framework for performing the decomposition of large-scale problems into smaller and easier subproblems by grouping objective variables. The efficiency of CC strongly depends on the size of groups and the grouping approach. In this study, an improved CC (iCC) approach for solving LSGO problems has been proposed and investigated. iCC changes the number of variables in subcomponents dynamically during the optimization process. The SHADE algorithm is used as a subcomponent optimizer. We have investigated the performance of iCC-SHADE and CC-SHADE on fifteen problems from the LSGO CEC’13 benchmark set provided by the IEEE Congress of Evolutionary Computation. The results of numerical experiments have shown that iCC-SHADE outperforms, on average, CC-SHADE with a fixed number of subcomponents. Also, we have compared iCC-SHADE with some state-of-the-art LSGO metaheuristics. The experimental results have shown that the proposed algorithm is competitive with other efficient metaheuristics.

scholarly journals A GPU-Enabled Compact Genetic Algorithm for Very Large-Scale Optimization Problems

Mathematics ◽  
2020 ◽  
Vol 8 (5) ◽  
pp. 758
Author(s):  
Andrea Ferigo ◽  
Giovanni Iacca
Keyword(s):  
Genetic Algorithm ◽  
Large Scale ◽  
Optimization Problems ◽  
Search Space ◽  
Large Scale Problems ◽  
Processing Power ◽  
Compact Genetic Algorithm ◽  
Estimation Of Distribution ◽  
Feasible Solutions ◽  
Compact Optimization

The ever-increasing complexity of industrial and engineering problems poses nowadays a number of optimization problems characterized by thousands, if not millions, of variables. For instance, very large-scale problems can be found in chemical and material engineering, networked systems, logistics and scheduling. Recently, Deb and Myburgh proposed an evolutionary algorithm capable of handling a scheduling optimization problem with a staggering number of variables: one billion. However, one important limitation of this algorithm is its memory consumption, which is in the order of 120 GB. Here, we follow up on this research by applying to the same problem a GPU-enabled “compact” Genetic Algorithm, i.e., an Estimation of Distribution Algorithm that instead of using an actual population of candidate solutions only requires and adapts a probabilistic model of their distribution in the search space. We also introduce a smart initialization technique and custom operators to guide the search towards feasible solutions. Leveraging the compact optimization concept, we show how such an algorithm can optimize efficiently very large-scale problems with millions of variables, with limited memory and processing power. To complete our analysis, we report the results of the algorithm on very large-scale instances of the OneMax problem.

scholarly journals Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for Nonconvex Nonsmooth Problems

2019 ◽  
Vol 33 ◽  
pp. 1528-1535
Author(s):  
Ehsan Kazemi ◽  
Liqiang Wang
Keyword(s):  
Large Scale ◽  
Convergence Rates ◽  
Optimization Problems ◽  
Coordinate Descent ◽  
Performance Guarantee ◽  
Nonsmooth Problems ◽  
Descent Property ◽  
Large Scale Problems ◽  
Nonconvex And Nonsmooth Optimization ◽  
Sufficient Descent

Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a challenge. Proximal coordinate descent (PCD) has been widely used for solving optimization problems, but the knowledge of PCD methods in the nonconvex setting is very limited. On the other hand, the asynchronous proximal coordinate descent (APCD) recently have received much attention in order to solve large-scale problems. However, the accelerated variants of APCD algorithms are rarely studied. In this paper, we extend APCD method to the accelerated algorithm (AAPCD) for nonsmooth and nonconvex problems that satisfies the sufficient descent property, by comparing between the function values at proximal update and a linear extrapolated point using a delay-aware momentum value. To the best of our knowledge, we are the first to provide stochastic and deterministic accelerated extension of APCD algorithms for general nonconvex and nonsmooth problems ensuring that for both bounded delays and unbounded delays every limit point is a critical point. By leveraging Kurdyka-Łojasiewicz property, we will show linear and sublinear convergence rates for the deterministic AAPCD with bounded delays. Numerical results demonstrate the practical efficiency of our algorithm in speed.

scholarly journals AN IMPROVED ADAPTIVE TRUST-REGION METHOD FOR UNCONSTRAINED OPTIMIZATION

2014 ◽  
Vol 19 (4) ◽  
pp. 469-490 ◽  
Author(s):  
Hamid Esmaeili ◽  
Morteza Kimiaei
Keyword(s):  
Unconstrained Optimization ◽  
Large Scale ◽  
Optimization Problems ◽  
Trust Region Method ◽  
Trust Region ◽  
Standard Test ◽  
Test Problems ◽  
Unconstrained Optimization Problems ◽  
Large Scale Problems ◽  
Adaptive Radius

In this study, we propose a trust-region-based procedure to solve unconstrained optimization problems that take advantage of the nonmonotone technique to introduce an efficient adaptive radius strategy. In our approach, the adaptive technique leads to decreasing the total number of iterations, while utilizing the structure of nonmonotone formula helps us to handle large-scale problems. The new algorithm preserves the global convergence and has quadratic convergence under suitable conditions. Preliminary numerical experiments on standard test problems indicate the efficiency and robustness of the proposed approach for solving unconstrained optimization problems.

scholarly journals Novel Interior Point Algorithms for Solving Nonlinear Convex Optimization Problems

2015 ◽  
Vol 2015 ◽  
pp. 1-7
Author(s):  
Sakineh Tahmasebzadeh ◽  
Hamidreza Navidi ◽  
Alaeddin Malek
Keyword(s):  
Convex Optimization ◽  
Objective Function ◽  
Interior Point ◽  
Optimization Problems ◽  
Numerical Algorithms ◽  
Optimal Solution ◽  
Linear Constraints ◽  
Convex Optimization Problems ◽  
Novel Algorithms ◽  
Interior Point Technique

This paper proposes three numerical algorithms based on Karmarkar’s interior point technique for solving nonlinear convex programming problems subject to linear constraints. The first algorithm uses the Karmarkar idea and linearization of the objective function. The second and third algorithms are modification of the first algorithm using the Schrijver and Malek-Naseri approaches, respectively. These three novel schemes are tested against the algorithm of Kebiche-Keraghel-Yassine (KKY). It is shown that these three novel algorithms are more efficient and converge to the correct optimal solution, while the KKY algorithm fails in some cases. Numerical results are given to illustrate the performance of the proposed algorithms.

scholarly journals Fertilization optimization algorithm on CEC2015 and large scale problems

2021 ◽  
Author(s):  
Hazim Nasir Ghafil ◽  
Shaymaa Alsamia ◽  
Károly Jármai
Keyword(s):  
Optimization Algorithm ◽  
Large Scale ◽  
Optimization Problems ◽  
Random Search ◽  
Search Space ◽  
Large Scale Optimization ◽  
Biologically Inspired ◽  
Large Scale Problems ◽  
Comparison Results ◽  
Scale Optimization

Abstract This work, presents a novel optimizer called fertilization optimization algorithm, which is based on levy flight and random search within a search space. It is a biologically inspired algorithm by the fertilization of the egg in reproduction of mammals. The performance of the algorithm was compared with other optimization algorithms on CEC2015 time expensive benchmarks and large scale optimization problems. Remarkably, the fertilization optimization algorithm has overcome other optimizers in many cases and the examination and comparison results are encouraging to use the fertilization optimization algorithm in other possible applications.

Sign in / Sign up

Export Citation Format

Share Document