scholarly journals Anatomy of the Attraction Basins: Breaking with the Intuition

2019 ◽  
Vol 27 (3) ◽  
pp. 435-466 ◽  
Author(s):  
Leticia Hernando ◽  
Alexander Mendiburu ◽  
Jose A. Lozano
Keyword(s):  
Combinatorial Optimization ◽  
Optimization Problems ◽  
Local Search Algorithms ◽  
Neighborhood Structure ◽  
Natural Landscape ◽  
Local Optima ◽  
Attraction Basin ◽  
Combinatorial Optimization Problems ◽  
New Findings ◽  
Attraction Basins

Solving combinatorial optimization problems efficiently requires the development of algorithms that consider the specific properties of the problems. In this sense, local search algorithms are designed over a neighborhood structure that partially accounts for these properties. Considering a neighborhood, the space is usually interpreted as a natural landscape, with valleys and mountains. Under this perception, it is commonly believed that, if maximizing, the solutions located in the slopes of the same mountain belong to the same attraction basin, with the peaks of the mountains being the local optima. Unfortunately, this is a widespread erroneous visualization of a combinatorial landscape. Thus, our aim is to clarify this aspect, providing a detailed analysis of, first, the existence of plateaus where the local optima are involved, and second, the properties that define the topology of the attraction basins, picturing a reliable visualization of the landscapes. Some of the features explored in this article have never been examined before. Hence, new findings about the structure of the attraction basins are shown. The study is focused on instances of permutation-based combinatorial optimization problems considering the 2-exchange and the insert neighborhoods. As a consequence of this work, we break away from the extended belief about the anatomy of attraction basins.

An Evaluation of Methods for Estimating the Number of Local Optima in Combinatorial Optimization Problems

2013 ◽  
Vol 21 (4) ◽  
pp. 625-658 ◽  
Author(s):  
Leticia Hernando ◽  
Alexander Mendiburu ◽  
Jose A. Lozano
Keyword(s):  
Combinatorial Optimization ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Search Space ◽  
Combinatorial Optimization Problem ◽  
Local Search Algorithms ◽  
Neighborhood Structure ◽  
Local Optima ◽  
Combinatorial Optimization Problems ◽  
Evaluation Of Methods

The solution of many combinatorial optimization problems is carried out by metaheuristics, which generally make use of local search algorithms. These algorithms use some kind of neighborhood structure over the search space. The performance of the algorithms strongly depends on the properties that the neighborhood imposes on the search space. One of these properties is the number of local optima. Given an instance of a combinatorial optimization problem and a neighborhood, the estimation of the number of local optima can help not only to measure the complexity of the instance, but also to choose the most convenient neighborhood to solve it. In this paper we review and evaluate several methods to estimate the number of local optima in combinatorial optimization problems. The methods reviewed not only come from the combinatorial optimization literature, but also from the statistical literature. A thorough evaluation in synthetic as well as real problems is given. We conclude by providing recommendations of methods for several scenarios.

scholarly journals A Methodology to Find the Elementary Landscape Decomposition of Combinatorial Optimization Problems

2011 ◽  
Vol 19 (4) ◽  
pp. 597-637 ◽  
Author(s):  
Francisco Chicano ◽  
L. Darrell Whitley ◽  
Enrique Alba
Keyword(s):  
Combinatorial Optimization ◽  
Objective Function ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Quadratic Assignment Problem ◽  
Combinatorial Optimization Problem ◽  
Quadratic Assignment ◽  
Neighborhood Structure ◽  
Combinatorial Optimization Problems ◽  
Elementary Landscape

A small number of combinatorial optimization problems have search spaces that correspond to elementary landscapes, where the objective function f is an eigenfunction of the Laplacian that describes the neighborhood structure of the search space. Many problems are not elementary; however, the objective function of a combinatorial optimization problem can always be expressed as a superposition of multiple elementary landscapes if the underlying neighborhood used is symmetric. This paper presents theoretical results that provide the foundation for algebraic methods that can be used to decompose the objective function of an arbitrary combinatorial optimization problem into a sum of subfunctions, where each subfunction is an elementary landscape. Many steps of this process can be automated, and indeed a software tool could be developed that assists the researcher in finding a landscape decomposition. This methodology is then used to show that the subset sum problem is a superposition of two elementary landscapes, and to show that the quadratic assignment problem is a superposition of three elementary landscapes.

Estimating attraction basin sizes of combinatorial optimization problems

2018 ◽  
Vol 7 (4) ◽  
pp. 369-384 ◽  
Author(s):  
Anne Elorza ◽  
Leticia Hernando ◽  
Alexander Mendiburu ◽  
Jose A. Lozano
Keyword(s):  
Combinatorial Optimization ◽  
Optimization Problems ◽  
Attraction Basin ◽  
Combinatorial Optimization Problems

Advances of Applying Metaheuristics to Data Mining Techniques

2015 ◽  
pp. 75-103 ◽  
Author(s):  
Simon Fong ◽  
Jinyan Li ◽  
Xueyuan Gong ◽  
Athanasios V. Vasilakos
Keyword(s):  
Data Mining ◽  
Combinatorial Optimization ◽  
Job Scheduling ◽  
Optimization Problems ◽  
Local Optima ◽  
Processing Resource ◽  
Combinatorial Optimization Problems ◽  
Data Mining Algorithms ◽  
Biological Entities ◽  
Mining Algorithms

Metaheuristics have lately gained popularity among researchers. Their underlying designs are inspired by biological entities and their behaviors, e.g. schools of fish, colonies of insects, and other land animals etc. They have been used successfully in optimization applications ranging from financial modeling, image processing, resource allocations, job scheduling to bioinformatics. In particular, metaheuristics have been proven in many combinatorial optimization problems. So that it is not necessary to attempt all possible candidate solutions to a problem via exhaustive enumeration and evaluation which is computationally intractable. The aim of this paper is to highlight some recent research related to metaheuristics and to discuss how they can enhance the efficacy of data mining algorithms. An upmost challenge in Data Mining is combinatorial optimization that, often lead to performance degradation and scalability issues. Two case studies are presented, where metaheuristics improve the accuracy of classification and clustering by avoiding local optima.

scholarly journals A Sensitivity Analysis of Critical Genetic Algorithm Parameters

2016 ◽  
pp. 863-880
Author(s):  
Eungcheol Kim ◽  
Manoj K. Jha ◽  
Min-Wook Kang
Keyword(s):  
Combinatorial Optimization ◽  
Optimization Problems ◽  
Critical Parameters ◽  
Optimal Solutions ◽  
Genetic Operators ◽  
Solution Quality ◽  
Local Optima ◽  
3 Dimensional ◽  
Combinatorial Optimization Problems ◽  
Decision Variables

Genetic Algorithms (GAs) have been applied in many complex combinatorial optimization problems and have been proven to yield reasonably good solutions due to their ability of searching in continuous spaces and avoiding local optima. However, one issue in GA application that needs to be carefully explored is to examine sensitivity of critical parameters that may affect the quality of solutions. The key critical GA parameters affecting solution quality include the number of genetic operators, the number of encoded decision variables, the parameter for selective pressure, and the parameter for non-uniform mutation. The effect of these parameters on solution quality is particularly significant for complex problems of combinatorial nature. In this paper the authors test the sensitivity of critical GA parameters in optimizing 3-dimensional highway alignments which has been proven to be a complex combinatorial optimization problem for which an exact solution is not possible warranting the application of heuristics procedures, such as GAs. If GAs are applied properly, similar optimal solutions should be expected at each replication. The authors perform several example studies in order to arrive at a general set of conclusions regarding the sensitivity of critical GA parameters on solution quality. The first study shows that the optimal solutions obtained for a range of scenarios consisting of different combinations of the critical parameters are quite close. The second study shows that different optimal solutions are obtained when the number of encoded decision variables is changed.

A Sensitivity Analysis of Critical Genetic Algorithm Parameters

2015 ◽  
Vol 6 (1) ◽  
pp. 30-48 ◽  
Author(s):  
Eungcheol Kim ◽  
Manoj K. Jha ◽  
Min-Wook Kang
Keyword(s):  
Combinatorial Optimization ◽  
Optimization Problems ◽  
Critical Parameters ◽  
Optimal Solutions ◽  
Genetic Operators ◽  
Solution Quality ◽  
Local Optima ◽  
3 Dimensional ◽  
Combinatorial Optimization Problems ◽  
Decision Variables

Genetic Algorithms (GAs) have been applied in many complex combinatorial optimization problems and have been proven to yield reasonably good solutions due to their ability of searching in continuous spaces and avoiding local optima. However, one issue in GA application that needs to be carefully explored is to examine sensitivity of critical parameters that may affect the quality of solutions. The key critical GA parameters affecting solution quality include the number of genetic operators, the number of encoded decision variables, the parameter for selective pressure, and the parameter for non-uniform mutation. The effect of these parameters on solution quality is particularly significant for complex problems of combinatorial nature. In this paper the authors test the sensitivity of critical GA parameters in optimizing 3-dimensional highway alignments which has been proven to be a complex combinatorial optimization problem for which an exact solution is not possible warranting the application of heuristics procedures, such as GAs. If GAs are applied properly, similar optimal solutions should be expected at each replication. The authors perform several example studies in order to arrive at a general set of conclusions regarding the sensitivity of critical GA parameters on solution quality. The first study shows that the optimal solutions obtained for a range of scenarios consisting of different combinations of the critical parameters are quite close. The second study shows that different optimal solutions are obtained when the number of encoded decision variables is changed.

A Sensitivity Analysis of Critical Genetic Algorithm Parameters

2015 ◽  
pp. 1064-1081
Author(s):  
Eungcheol Kim ◽  
Manoj K. Jha ◽  
Min-Wook Kang
Keyword(s):  
Combinatorial Optimization ◽  
Optimization Problems ◽  
Critical Parameters ◽  
Optimal Solutions ◽  
Genetic Operators ◽  
Solution Quality ◽  
Local Optima ◽  
3 Dimensional ◽  
Combinatorial Optimization Problems ◽  
Decision Variables

Genetic Algorithms (GAs) have been applied in many complex combinatorial optimization problems and have been proven to yield reasonably good solutions due to their ability of searching in continuous spaces and avoiding local optima. However, one issue in GA application that needs to be carefully explored is to examine sensitivity of critical parameters that may affect the quality of solutions. The key critical GA parameters affecting solution quality include the number of genetic operators, the number of encoded decision variables, the parameter for selective pressure, and the parameter for non-uniform mutation. The effect of these parameters on solution quality is particularly significant for complex problems of combinatorial nature. In this paper the authors test the sensitivity of critical GA parameters in optimizing 3-dimensional highway alignments which has been proven to be a complex combinatorial optimization problem for which an exact solution is not possible warranting the application of heuristics procedures, such as GAs. If GAs are applied properly, similar optimal solutions should be expected at each replication. The authors perform several example studies in order to arrive at a general set of conclusions regarding the sensitivity of critical GA parameters on solution quality. The first study shows that the optimal solutions obtained for a range of scenarios consisting of different combinations of the critical parameters are quite close. The second study shows that different optimal solutions are obtained when the number of encoded decision variables is changed.

scholarly journals Cakewalk Sampling

2020 ◽  
Vol 34 (03) ◽  
pp. 2400-2407
Author(s):  
Uri Patish ◽  
Shimon Ullman
Keyword(s):  
Combinatorial Optimization ◽  
Sampling Methods ◽  
Optimization Problems ◽  
Learning Algorithm ◽  
Greedy Algorithms ◽  
Superior Performance ◽  
Use Case ◽  
Local Optima ◽  
Combinatorial Optimization Problems ◽  
Valid Solution

We study the task of finding good local optima in combinatorial optimization problems. Although combinatorial optimization is NP-hard in general, locally optimal solutions are frequently used in practice. Local search methods however typically converge to a limited set of optima that depend on their initialization. Sampling methods on the other hand can access any valid solution, and thus can be used either directly or alongside methods of the former type as a way for finding good local optima. Since the effectiveness of this strategy depends on the sampling distribution, we derive a robust learning algorithm that adapts sampling distributions towards good local optima of arbitrary objective functions. As a first use case, we empirically study the efficiency in which sampling methods can recover locally maximal cliques in undirected graphs. Not only do we show how our adaptive sampler outperforms related methods, we also show how it can even approach the performance of established clique algorithms. As a second use case, we consider how greedy algorithms can be combined with our adaptive sampler, and we demonstrate how this leads to superior performance in k-medoid clustering. Together, these findings suggest that our adaptive sampler can provide an effective strategy to combinatorial optimization problems that arise in practice.

Sign in / Sign up

Export Citation Format

Share Document