Mixed Integer Evolution Strategies for Parameter Optimization

2013 ◽  
Vol 21 (1) ◽  
pp. 29-64 ◽  
Author(s):  
Rui Li ◽  
Michael T.M. Emmerich ◽  
Jeroen Eggermont ◽  
Thomas Bäck ◽  
M. Schütz ◽  
...  
Keyword(s):  
Parameter Optimization ◽  
Optimization Problems ◽  
Medical Image Analysis ◽  
Evolution Strategies ◽  
Mixed Integer ◽  
Maximal Entropy ◽  
Integer Optimization ◽  
Mutation Operators ◽  
Wide Range ◽  
Self Adaptation

Evolution strategies (ESs) are powerful probabilistic search and optimization algorithms gleaned from biological evolution theory. They have been successfully applied to a wide range of real world applications. The modern ESs are mainly designed for solving continuous parameter optimization problems. Their ability to adapt the parameters of the multivariate normal distribution used for mutation during the optimization run makes them well suited for this domain. In this article we describe and study mixed integer evolution strategies (MIES), which are natural extensions of ES for mixed integer optimization problems. MIES can deal with parameter vectors consisting not only of continuous variables but also with nominal discrete and integer variables. Following the design principles of the canonical evolution strategies, they use specialized mutation operators tailored for the aforementioned mixed parameter classes. For each type of variable, the choice of mutation operators is governed by a natural metric for this variable type, maximal entropy, and symmetry considerations. All distributions used for mutation can be controlled in their shape by means of scaling parameters, allowing self-adaptation to be implemented. After introducing and motivating the conceptual design of the MIES, we study the optimality of the self-adaptation of step sizes and mutation rates on a generalized (weighted) sphere model. Moreover, we prove global convergence of the MIES on a very general class of problems. The remainder of the article is devoted to performance studies on artificial landscapes (barrier functions and mixed integer NK landscapes), and a case study in the optimization of medical image analysis systems. In addition, we show that with proper constraint handling techniques, MIES can also be applied to classical mixed integer nonlinear programming problems.

Evolutionary Algorithms in Theory and Practice

1996 ◽  
Author(s):  
Thomas Bäck
Keyword(s):  
Genetic Algorithms ◽  
Evolutionary Algorithms ◽  
Optimization Problems ◽  
Theory And Practice ◽  
Evolution Strategies ◽  
Parallel Computer ◽  
Mixed Integer ◽  
Integer Optimization ◽  
Wide Range ◽  
The Impact

This book presents a unified view of evolutionary algorithms: the exciting new probabilistic search tools inspired by biological models that have immense potential as practical problem-solvers in a wide variety of settings, academic, commercial, and industrial. In this work, the author compares the three most prominent representatives of evolutionary algorithms: genetic algorithms, evolution strategies, and evolutionary programming. The algorithms are presented within a unified framework, thereby clarifying the similarities and differences of these methods. The author also presents new results regarding the role of mutation and selection in genetic algorithms, showing how mutation seems to be much more important for the performance of genetic algorithms than usually assumed. The interaction of selection and mutation, and the impact of the binary code are further topics of interest. Some of the theoretical results are also confirmed by performing an experiment in meta-evolution on a parallel computer. The meta-algorithm used in this experiment combines components from evolution strategies and genetic algorithms to yield a hybrid capable of handling mixed integer optimization problems. As a detailed description of the algorithms, with practical guidelines for usage and implementation, this work will interest a wide range of researchers in computer science and engineering disciplines, as well as graduate students in these fields.

scholarly journals Solution of Mixed-Integer Optimization Problems in Bioinformatics with Differential Evolution Method

Mathematics ◽  
2021 ◽  
Vol 9 (24) ◽  
pp. 3329
Author(s):  
Sergey Salihov ◽  
Dmitriy Maltsov ◽  
Maria Samsonova ◽  
Konstantin Kozlov
Keyword(s):  
Differential Evolution ◽  
Genomic Selection ◽  
Optimization Problems ◽  
Snp Markers ◽  
Selection Model ◽  
Mixed Integer ◽  
Integer Optimization ◽  
Mixed Integer Optimization ◽  
Wide Range ◽  
Genomic Selection Model

The solution of the so-called mixed-integer optimization problem is an important challenge for modern life sciences. A wide range of methods has been developed for its solution, including metaheuristics approaches. Here, a modification is proposed of the differential evolution entirely parallel (DEEP) method introduced recently that was successfully applied to mixed-integer optimization problems. The triangulation recombination rule was implemented and the recombination coefficients were included in the evolution process in order to increase the robustness of the optimization. The deduplication step included in the procedure ensures the uniqueness of individual integer-valued parameters in the solution vectors. The developed algorithms were implemented in the DEEP software package and applied to three bioinformatic problems. The application of the method to the optimization of predictors set in the genomic selection model in wheat resulted in dimensionality reduction such that the phenotype can be predicted with acceptable accuracy using a selected subset of SNP markers. The method was also successfully used to optimize the training set of samples for such a genomic selection model. According to the obtained results, the developed algorithm was capable of constructing a non-linear phenomenological regression model of gene expression in developing a Drosophila eye with almost the same average accuracy but significantly less standard deviation than the linear models obtained earlier.

scholarly journals Partially distributed outer approximation

2021 ◽  
Author(s):  
Alexander Murray ◽  
Timm Faulwasser ◽  
Veit Hagenmeyer ◽  
Mario E. Villanueva ◽  
Boris Houska
Keyword(s):  
Large Scale ◽  
Optimization Problems ◽  
Outer Approximation ◽  
Mixed Integer ◽  
Global Optimality ◽  
Integer Optimization ◽  
Global Minimizers ◽  
Outer Approximation Method ◽  
Coupling Constraints ◽  
Outer Approximation Algorithm

AbstractThis paper presents a novel partially distributed outer approximation algorithm, named PaDOA, for solving a class of structured mixed integer convex programming problems to global optimality. The proposed scheme uses an iterative outer approximation method for coupled mixed integer optimization problems with separable convex objective functions, affine coupling constraints, and compact domain. PaDOA proceeds by alternating between solving large-scale structured mixed-integer linear programming problems and partially decoupled mixed-integer nonlinear programming subproblems that comprise much fewer integer variables. We establish conditions under which PaDOA converges to global minimizers after a finite number of iterations and verify these properties with an application to thermostatically controlled loads and to mixed-integer regression.

scholarly journals HyP-ABC: A Novel Automated Hyper-Parameter Tuning Algorithm Using Evolutionary Optimization

2021 ◽  
Author(s):  
Leila Zahedi ◽  
Farid Ghareh Mohammadi ◽  
M. Hadi Amini
Keyword(s):  
Parameter Optimization ◽  
Real World ◽  
Optimization Problems ◽  
State Of The Art ◽  
Parameter Tuning ◽  
Gradient Boosting ◽  
Support Vector ◽  
Wide Range ◽  
Extreme Gradient Boosting ◽  
Art Techniques

Machine learning techniques lend themselves as promising decision-making and analytic tools in a wide range of applications. Different ML algorithms have various hyper-parameters. In order to tailor an ML model towards a specific application, a large number of hyper-parameters should be tuned. Tuning the hyper-parameters directly affects the performance (accuracy and run-time). However, for large-scale search spaces, efficiently exploring the ample number of combinations of hyper-parameters is computationally challenging. Existing automated hyper-parameter tuning techniques suffer from high time complexity. In this paper, we propose HyP-ABC, an automatic innovative hybrid hyper-parameter optimization algorithm using the modified artificial bee colony approach, to measure the classification accuracy of three ML algorithms, namely random forest, extreme gradient boosting, and support vector machine. Compared to the state-of-the-art techniques, HyP-ABC is more efficient and has a limited number of parameters to be tuned, making it worthwhile for real-world hyper-parameter optimization problems. We further compare our proposed HyP-ABC algorithm with state-of-the-art techniques. In order to ensure the robustness of the proposed method, the algorithm takes a wide range of feasible hyper-parameter values, and is tested using a real-world educational dataset.

A Mixed-Integer Memetic Algorithm Applied to Batch Process Optimization

2013 ◽  
Vol 300-301 ◽  
pp. 645-648 ◽  
Author(s):  
Yung Chien Lin
Keyword(s):  
Memetic Algorithm ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Search Space ◽  
Memetic Algorithms ◽  
Batch Processes ◽  
Mixed Integer ◽  
Integer Optimization ◽  
Genetic Operators ◽  
Search Methods

Evolutionary algorithms (EAs) are population-based global search methods. Memetic Algorithms (MAs) are hybrid EAs that combine genetic operators with local search methods. With global exploration and local exploitation in search space, MAs are capable of obtaining more high-quality solutions. On the other hand, mixed-integer hybrid differential evolution (MIHDE), as an EA-based search algorithm, has been successfully applied to many mixed-integer optimization problems. In this paper, a mixed-integer memetic algorithm based on MIHDE is developed for solving mixed-integer constrained optimization problems. The proposed algorithm is implemented and applied to the optimal design of batch processes. Experimental results show that the proposed algorithm can find a better optimal solution compared with some other search algorithms.

Self-Adaptation of Mutation Operator and Probability for Permutation Representations in Genetic Algorithms

2010 ◽  
Vol 18 (3) ◽  
pp. 491-514 ◽  
Author(s):  
Martin Serpell ◽  
James E. Smith
Keyword(s):  
Mutation Rate ◽  
Poor Performance ◽  
Adaptive Methods ◽  
Optimal Choice ◽  
Problem Instance ◽  
Mutation Operator ◽  
Genetic Operators ◽  
Mutation Operators ◽  
Wide Range ◽  
Self Adaptation

The choice of mutation rate is a vital factor in the success of any genetic algorithm (GA), and for permutation representations this is compounded by the availability of several alternative mutation operators. It is now well understood that there is no one “optimal choice”; rather, the situation changes per problem instance and during evolution. This paper examines whether this choice can be left to the processes of evolution via self-adaptation, thus removing this nontrivial task from the GA user and reducing the risk of poor performance arising from (inadvertent) inappropriate decisions. Self-adaptation has been proven successful for mutation step sizes in the continuous domain, and for the probability of applying bitwise mutation to binary encodings; here we examine whether this can translate to the choice and parameterisation of mutation operators for permutation encodings. We examine one method for adapting the choice of operator during runtime, and several different methods for adapting the rate at which the chosen operator is applied. In order to evaluate these algorithms, we have used a range of benchmark TSP problems. Of course this paper is not intended to present a state of the art in TSP solvers; rather, we use this well known problem as typical of many that require a permutation encoding, where our results indicate that self-adaptation can prove beneficial. The results show that GAs using appropriate methods to self-adapt their mutation operator and mutation rate find solutions of comparable or lower cost than algorithms with “static” operators, even when the latter have been extensively pretuned. Although the adaptive GAs tend to need longer to run, we show that is a price well worth paying as the time spent finding the optimal mutation operator and rate for the nonadaptive versions can be considerable. Finally, we evaluate the sensitivity of the self-adaptive methods to changes in the implementation, and to the choice of other genetic operators and population models. The results show that the methods presented are robust, in the sense that the performance benefits can be obtained in a wide range of host algorithms.

Convergence in Evolutionary Programs with Self-Adaptation

2001 ◽  
Vol 9 (2) ◽  
pp. 147-157 ◽  
Author(s):  
Garrison W. Greenwood ◽  
Qiji J. Zhu
Keyword(s):  
Optimization Problems ◽  
Previous Analysis ◽  
Evolution Strategies ◽  
Convergence Properties ◽  
Formal Proofs ◽  
Self Adaptation ◽  
Long Term Behavior ◽  
Adaptation Method ◽  
Strategy Parameters

Evolutionary programs are capable of finding good solutions to difficult optimization problems. Previous analysis of their convergence properties has normally assumed the strategy parameters are kept constant, although in practice these parameters are dynamically altered. In this paper, we propose a modified version of the 1/5-success rule for self-adaptation in evolution strategies (ES). Formal proofs of the long-term behavior produced by our self-adaptation method are included. Both elitist and non-elitist ES variants are analyzed. Preliminary tests indicate an ES with our modified self-adaptation method compares favorably to both a non-adapted ES and a 1/5-success rule adapted ES.

scholarly journals About stability of mixed integer optimization problems under perturbations of input data of vector criterion

2021 ◽  
pp. 7-11
Author(s):  
Tetiana Lebedeva ◽  
Natalia Semenova ◽  
Tetiana Sergienko
Keyword(s):  
Initial Data ◽  
Input Data ◽  
Optimization Problems ◽  
Point Of View ◽  
Mixed Integer ◽  
Stability Of Solutions ◽  
Integer Optimization ◽  
Mixed Integer Optimization ◽  
Small Perturbations ◽  
Vector Criterion

The article is devoted to the study of the influence of uncertainty in initial data on the solutions of mixed integer optimization vector problems. In the optimization problems, including problems with vector criterion, small perturbations in initial data can result in solutions strongly different from the true ones. The problem of stability of the indicated tasks is studied from the point of view of direct coupled with her question in relation to stability of solutions belonging to some subsets of feasible set.

Sign in / Sign up

Export Citation Format

Share Document