scholarly journals Global Optimization for the Sum of Certain Nonlinear Functions

2014 ◽  
Vol 2014 ◽  
pp. 1-8 ◽  
Author(s):  
Mio Horai ◽  
Hideo Kobayashi ◽  
Takashi G. Nitta
Keyword(s):  
Global Optimization ◽  
Branch And Bound ◽  
Nonconvex Optimization ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Global Optimization Problem ◽  
Nonlinear Functions ◽  
Nonconvex Optimization Problems ◽  
Second Derivatives ◽  
Fractional Function

We extend work by Pei-Ping and Gui-Xia, 2007, to a global optimization problem for more general functions. Pei-Ping and Gui-Xia treat the optimization problem for the linear sum of polynomial fractional functions, using a branch and bound approach. We prove that this extension makes possible to solve the following nonconvex optimization problems which Pei-Ping and Gui-Xia, 2007, cannot solve, that the sum of the positive (or negative) first and second derivatives function with the variable defined by sum of polynomial fractional function by using branch and bound algorithm.

scholarly journals A Linearized Relaxing Algorithm for the Specific Nonlinear Optimization Problem

2016 ◽  
Vol 2016 ◽  
pp. 1-8
Author(s):  
Mio Horai ◽  
Hideo Kobayashi ◽  
Takashi G. Nitta
Keyword(s):  
Global Optimization ◽  
Global Convergence ◽  
Nonlinear Optimization ◽  
Branch And Bound ◽  
Nonconvex Optimization ◽  
Optimization Problem ◽  
New Method ◽  
Linear Relaxation ◽  
Global Optimization Problem ◽  
Nonconvex Global Optimization

We propose a new method for the specific nonlinear and nonconvex global optimization problem by using a linear relaxation technique. To simplify the specific nonlinear and nonconvex optimization problem, we transform the problem to the lower linear relaxation form, and we solve the linear relaxation optimization problem by the Branch and Bound Algorithm. Under some reasonable assumptions, the global convergence of the algorithm is certified for the problem. Numerical results show that this method is more efficient than the previous methods.

scholarly journals Efficiency of quantum vs. classical annealing in nonconvex learning problems

2018 ◽  
Vol 115 (7) ◽  
pp. 1457-1462 ◽  
Author(s):  
Carlo Baldassi ◽  
Riccardo Zecchina
Keyword(s):  
Nonconvex Optimization ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Transverse Field ◽  
Learning Problems ◽  
Optimal Solutions ◽  
Local Minima ◽  
Quantum Annealing ◽  
Nonconvex Optimization Problems ◽  
Classical Energy

Quantum annealers aim at solving nonconvex optimization problems by exploiting cooperative tunneling effects to escape local minima. The underlying idea consists of designing a classical energy function whose ground states are the sought optimal solutions of the original optimization problem and add a controllable quantum transverse field to generate tunneling processes. A key challenge is to identify classes of nonconvex optimization problems for which quantum annealing remains efficient while thermal annealing fails. We show that this happens for a wide class of problems which are central to machine learning. Their energy landscapes are dominated by local minima that cause exponential slowdown of classical thermal annealers while simulated quantum annealing converges efficiently to rare dense regions of optimal solutions.

scholarly journals Study of League Championship Algorithm Efficiency for Global Optimization Problem

2020 ◽  
pp. 25-45
Author(s):  
D. O. Zaharov ◽  
A. P. Karpenko
Keyword(s):  
Global Optimization ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Computational Experiments ◽  
Performance Criteria ◽  
Particle Swarm Algorithm ◽  
Global Optimization Problem ◽  
Algorithm Efficiency

The article objective is to study a new League Championship Algorithm (LCA) algorithm efficiency by its comparing with the efficiency of the Particle Swarm optimization (PSO) algorithm.The article presents a brief description of the terms used in the League Championship algorithm, describes the basic rules of the algorithm, on the basis of which the iterative process for solving the global optimization problem is built.Gives a detailed description of the League Championship algorithm, which comprises a flowchart of the algorithm, as well as a formalization of all its main steps.Depicts an exhaustive description of the software developed to implement the League Championship algorithm to solve global optimization problems.Briefly describes the modified particle swarm algorithm. Presents the values of all free parameters of the algorithm and the algorithm modifications, which make it different from the classical version, as well.The main part of the article shows the results of a great deal of computational experiments using two abovementioned algorithms. All the performance criteria, used for assessment of the algorithms efficiency, are given.Computational experiments were performed using the spherical function, as well as the Rosenbrock, Rastrigin, and Ackley functions. The results of the experiments are summarized in Tables, and also illustrated in Figures. Experiments were performed for the vector dimension of the variable parameters that is equal to 2, 4, 8, 16, 32, and 64.An analysis of the results of computational experiments involves a full assessment of the efficiency of the League Championship algorithm, and also provides an answer about expediency for further algorithm development.It is shown that the League Championship algorithm presented in the article has a high development potential and needs further work for its study.

scholarly journals Finding Global Minima with a Filled Function Approach for Non-Smooth Global Optimization

2010 ◽  
Vol 2010 ◽  
pp. 1-10 ◽  
Author(s):  
Weixiang Wang ◽  
Youlin Shang ◽  
Ying Zhang
Keyword(s):  
Global Optimization ◽  
Optimization Problem ◽  
Numerical Test ◽  
Function Approach ◽  
Global Optimization Problem ◽  
Computational Results ◽  
Filled Function ◽  
Global Minima ◽  
Smooth Case ◽  
Definition Of

A filled function approach is proposed for solving a non-smooth unconstrained global optimization problem. First, the definition of filled function in Zhang (2009) for smooth global optimization is extended to non-smooth case and a new one is put forwarded. Then, a novel filled function is proposed for non-smooth the global optimization and a corresponding non-smooth algorithm based on the filled function is designed. At last, a numerical test is made. The computational results demonstrate that the proposed approach is effcient and reliable.

scholarly journals A Diffusion Approximation Theory of Momentum Stochastic Gradient Descent in Nonconvex Optimization

2021 ◽  
Author(s):  
Tianyi Liu ◽  
Zhehui Chen ◽  
Enlu Zhou ◽  
Tuo Zhao
Keyword(s):  
Neural Networks ◽  
Nonconvex Optimization ◽  
Gradient Descent ◽  
Deep Neural Networks ◽  
Optimization Problems ◽  
Saddle Points ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Nonconvex Optimization Problems ◽  
Empirical Success

Momentum stochastic gradient descent (MSGD) algorithm has been widely applied to many nonconvex optimization problems in machine learning (e.g., training deep neural networks, variational Bayesian inference, etc.). Despite its empirical success, there is still a lack of theoretical understanding of convergence properties of MSGD. To fill this gap, we propose to analyze the algorithmic behavior of MSGD by diffusion approximations for nonconvex optimization problems with strict saddle points and isolated local optima. Our study shows that the momentum helps escape from saddle points but hurts the convergence within the neighborhood of optima (if without the step size annealing or momentum annealing). Our theoretical discovery partially corroborates the empirical success of MSGD in training deep neural networks.

Sign in / Sign up

Export Citation Format

Share Document