On the Benefits of Populations for Noisy Optimization

2003 ◽  
Vol 11 (2) ◽  
pp. 111-127 ◽  
Author(s):  
Dirk V. Arnold ◽  
Hans-Georg Beyer
Keyword(s):  
Order Statistics ◽  
Dynamic Process ◽  
Evolutionary Dynamics ◽  
Optimization Problems ◽  
Evolution Strategies ◽  
High Dimensional ◽  
Noisy Optimization ◽  
Search Spaces ◽  
Improved Performance ◽  
Positive Effect

It is known that, in the absence of noise, no improvement in local performance can be gained from retaining candidate solutions other than the best one. Yet, it has been shown experimentally that, in the presence of noise, operating with a non-singular population of candidate solutions can have a marked and positive effect on the local performance of evolution strategies. So as to determine the reasons for the improved performance, we have studied the evolutionary dynamics of the (μ, λ)-ES in the presence of noise. Considering a simple, idealized environment, we have developed a moment-based approach that uses recent results involving concomitants of selected order statistics. This approach yields an intuitive explanation for the performance advantage of multi-parent strategies in the presence of noise. It is then shown that the idealized dynamic process considered does bear relevance to optimization problems in high-dimensional search spaces.

Improved Trust Region Based MPS Method for High-Dimensional Expensive Black-Box Problems

2013 ◽  
Author(s):  
George H. Cheng ◽  
Adel Younis ◽  
Kambiz Haji Hajikolaei ◽  
G. Gary Wang
Keyword(s):  
Optimization Problems ◽  
Trust Region ◽  
Black Box ◽  
High Dimensional ◽  
Test Problems ◽  
Design Variables ◽  
Low Dimensionality ◽  
Improve Algorithm ◽  
Improved Performance ◽  
Better Than

Mode Pursuing Sampling (MPS) was developed as a global optimization algorithm for optimization problems involving expensive black box functions. MPS has been found to be effective and efficient for problems of low dimensionality, i.e., the number of design variables is less than ten. A previous conference publication integrated the concept of trust regions into the MPS framework to create a new algorithm, TRMPS, which dramatically improved performance and efficiency for high dimensional problems. However, although TRMPS performed better than MPS, it was unproven against other established algorithms such as GA. This paper introduces an improved algorithm, TRMPS2, which incorporates guided sampling and low function value criterion to further improve algorithm performance for high dimensional problems. TRMPS2 is benchmarked against MPS and GA using a suite of test problems. The results show that TRMPS2 performs better than MPS and GA on average for high dimensional, expensive, and black box (HEB) problems.

scholarly journals Mirrored Orthogonal Sampling for Covariance Matrix Adaptation Evolution Strategies

2019 ◽  
Vol 27 (4) ◽  
pp. 699-725 ◽  
Author(s):  
Hao Wang ◽  
Michael Emmerich ◽  
Thomas Bäck
Keyword(s):  
Convergence Rate ◽  
Covariance Matrix ◽  
Sampling Technique ◽  
Black Box ◽  
Evolution Strategies ◽  
High Dimensional ◽  
Major Purpose ◽  
Search Spaces ◽  
Covariance Matrix Adaptation ◽  
Adaptation Evolution

Generating more evenly distributed samples in high dimensional search spaces is the major purpose of the recently proposed mirrored sampling technique for evolution strategies. The diversity of the mutation samples is enlarged and the convergence rate is therefore improved by the mirrored sampling. Motivated by the mirrored sampling technique, this article introduces a new derandomized sampling technique called mirrored orthogonal sampling. The performance of this new technique is both theoretically analyzed and empirically studied on the sphere function. In particular, the mirrored orthogonal sampling technique is applied to the well-known Covariance Matrix Adaptation Evolution Strategy (CMA-ES). The resulting algorithm is experimentally tested on the well-known Black-Box Optimization Benchmark (BBOB). By comparing the results from the benchmark, mirrored orthogonal sampling is found to outperform both the standard CMA-ES and its variant using mirrored sampling.

Constraint Solving and Optimization Using Evolutionary Techniques

2019 ◽  
Author(s):  
Mahdi Bidar ◽  
Malek Mouhoub
Keyword(s):  
Optimization Algorithm ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Constraint Solving ◽  
High Dimensional ◽  
Constraint Optimization ◽  
Objective Functions ◽  
Search Spaces ◽  
Real World Applications ◽  
Constraint Optimization Problems

Constraint Solving and Optimization is very relevant in many real world applications including scheduling, planning, configuration, resource allocation and timetabling. Solving a constraint optimization problem consists of finding an assignment of values to variables that optimizes some defined objective functions, subject to a set of constraints imposed on the problem variables. Due to their high dimensional and exponential search spaces, classical methods are unpractical to tackle these problems. An appropriate alternative is to rely on metaheuristics. My thesis is concerned with investigating the applicability of the evolutionary algorithms when dealing with constraint optimization problems. In this regard, we propose two new optimization algorithms namely Mushroom Reproduction Optimization algorithm (MRO) and Focus Group Optimization algorithm (FGO) for solving such problems.

scholarly journals Investigation of Improved Cooperative Coevolution for Large-Scale Global Optimization Problems

Algorithms ◽  
2021 ◽  
Vol 14 (5) ◽  
pp. 146
Author(s):  
Aleksei Vakhnin ◽  
Evgenii Sopov
Keyword(s):  
Global Optimization ◽  
Evolutionary Algorithms ◽  
Numerical Experiments ◽  
Large Scale ◽  
Optimization Problems ◽  
State Of The Art ◽  
Fixed Number ◽  
High Dimensional ◽  
Cooperative Coevolution ◽  
Large Scale Problems

Modern real-valued optimization problems are complex and high-dimensional, and they are known as “large-scale global optimization (LSGO)” problems. Classic evolutionary algorithms (EAs) perform poorly on this class of problems because of the curse of dimensionality. Cooperative Coevolution (CC) is a high-performed framework for performing the decomposition of large-scale problems into smaller and easier subproblems by grouping objective variables. The efficiency of CC strongly depends on the size of groups and the grouping approach. In this study, an improved CC (iCC) approach for solving LSGO problems has been proposed and investigated. iCC changes the number of variables in subcomponents dynamically during the optimization process. The SHADE algorithm is used as a subcomponent optimizer. We have investigated the performance of iCC-SHADE and CC-SHADE on fifteen problems from the LSGO CEC’13 benchmark set provided by the IEEE Congress of Evolutionary Computation. The results of numerical experiments have shown that iCC-SHADE outperforms, on average, CC-SHADE with a fixed number of subcomponents. Also, we have compared iCC-SHADE with some state-of-the-art LSGO metaheuristics. The experimental results have shown that the proposed algorithm is competitive with other efficient metaheuristics.

scholarly journals A Bayesian approach for quantile optimization problems with high-dimensional uncertainty sources

2021 ◽  
Vol 376 ◽  
pp. 113632
Author(s):  
Christian Sabater ◽  
Olivier Le Maître ◽  
Pietro Marco Congedo ◽  
Stefan Görtz
Keyword(s):  
Bayesian Approach ◽  
Optimization Problems ◽  
High Dimensional ◽  
Uncertainty Sources

Preservation of Additive Convexity and Its Applications in Stochastic Optimization Problems

2021 ◽  
Author(s):  
Xiting Gong ◽  
Tong Wang
Keyword(s):  
Stochastic Optimization ◽  
Optimization Problems ◽  
High Dimensional ◽  
Value Functions

Preservation Results for Proving Additively Convex Value Functions for High-Dimensional Stochastic Optimization Problems

Sign in / Sign up

Export Citation Format

Share Document