scholarly journals Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization

2021 ◽  
Vol 72 ◽  
pp. 667-715
Author(s):  
Syrine Belakaria ◽  
Aryan Deshwal ◽  
Janardhan Rao Doppa
Keyword(s):  
Pareto Front ◽  
Black Box ◽  
Bayesian Optimization ◽  
Multi Objective ◽  
Expensive Function ◽  
Area Overhead ◽  
Resource Cost ◽  
Function Approximations ◽  
Total Resource ◽  
Output Space

We consider the problem of black-box multi-objective optimization (MOO) using expensive function evaluations (also referred to as experiments), where the goal is to approximate the true Pareto set of solutions by minimizing the total resource cost of experiments. For example, in hardware design optimization, we need to find the designs that trade-off performance, energy, and area overhead using expensive computational simulations. The key challenge is to select the sequence of experiments to uncover high-quality solutions using minimal resources. In this paper, we propose a general framework for solving MOO problems based on the principle of output space entropy (OSE) search: select the experiment that maximizes the information gained per unit resource cost about the true Pareto front. We appropriately instantiate the principle of OSE search to derive efficient algorithms for the following four MOO problem settings: 1) The most basic single-fidelity setting, where experiments are expensive and accurate; 2) Handling black-box constraints which cannot be evaluated without performing experiments; 3) The discrete multi-fidelity setting, where experiments can vary in the amount of resources consumed and their evaluation accuracy; and 4) The continuous-fidelity setting, where continuous function approximations result in a huge space of experiments. Experiments on diverse synthetic and real-world benchmarks show that our OSE search based algorithms improve over state-of-the-art methods in terms of both computational-efficiency and accuracy of MOO solutions.

scholarly journals Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimization

2020 ◽  
Vol 34 (06) ◽  
pp. 10044-10052 ◽  
Author(s):  
Syrine Belakaria ◽  
Aryan Deshwal ◽  
Nitthilan Kannappan Jayakodi ◽  
Janardhan Rao Doppa
Keyword(s):  
Optimization Problem ◽  
State Of The Art ◽  
Selection Method ◽  
Bayesian Optimization ◽  
Benchmark Problems ◽  
Multi Objective ◽  
Blackbox Optimization ◽  
Expensive Function ◽  
Area Overhead ◽  
Measure Of Uncertainty

We consider the problem of multi-objective (MO) blackbox optimization using expensive function evaluations, where the goal is to approximate the true Pareto set of solutions while minimizing the number of function evaluations. For example, in hardware design optimization, we need to find the designs that trade-off performance, energy, and area overhead using expensive simulations. We propose a novel uncertainty-aware search framework referred to as USeMO to efficiently select the sequence of inputs for evaluation to solve this problem. The selection method of USeMO consists of solving a cheap MO optimization problem via surrogate models of the true functions to identify the most promising candidates and picking the best candidate based on a measure of uncertainty. We also provide theoretical analysis to characterize the efficacy of our approach. Our experiments on several synthetic and six diverse real-world benchmark problems show that USeMO consistently outperforms the state-of-the-art algorithms.

scholarly journals Multi-Fidelity Multi-Objective Bayesian Optimization: An Output Space Entropy Search Approach

2020 ◽  
Vol 34 (06) ◽  
pp. 10035-10043 ◽  
Author(s):  
Syrine Belakaria ◽  
Aryan Deshwal ◽  
Janardhan Rao Doppa
Keyword(s):  
Bayesian Optimization ◽  
Benchmark Problems ◽  
Multi Objective Optimization ◽  
Multi Objective ◽  
Blackbox Optimization ◽  
Novel Approach ◽  
Resource Cost ◽  
System Design Optimization ◽  
Search Approach ◽  
Output Space

We study the novel problem of blackbox optimization of multiple objectives via multi-fidelity function evaluations that vary in the amount of resources consumed and their accuracy. The overall goal is to appromixate the true Pareto set of solutions by minimizing the resources consumed for function evaluations. For example, in power system design optimization, we need to find designs that trade-off cost, size, efficiency, and thermal tolerance using multi-fidelity simulators for design evaluations. In this paper, we propose a novel approach referred as Multi-Fidelity Output Space Entropy Search for Multi-objective Optimization (MF-OSEMO) to solve this problem. The key idea is to select the sequence of candidate input and fidelity-vector pairs that maximize the information gained about the true Pareto front per unit resource cost. Our experiments on several synthetic and real-world benchmark problems show that MF-OSEMO, with both approximations, significantly improves over the state-of-the-art single-fidelity algorithms for multi-objective optimization.

A Simple and Effective Methodology to Perform Multi-Objective Bayesian Optimization: An Application in the Design of Sandwich Composite Armors for Blast Mitigation

2020 ◽  
Author(s):  
Homero Valladares ◽  
Andres Tovar
Keyword(s):  
Finite Element ◽  
Surrogate Model ◽  
Test Problem ◽  
Pareto Front ◽  
Computational Cost ◽  
Bayesian Optimization ◽  
Sandwich Composite ◽  
Blast Mitigation ◽  
Multi Objective ◽  
Probabilistic Information

Abstract Bayesian optimization is a versatile numerical method to solve global optimization problems of high complexity at a reduced computational cost. The efficiency of Bayesian optimization relies on two key elements: a surrogate model and an acquisition function. The surrogate model is generated on a Gaussian process statistical framework and provides probabilistic information of the prediction. The acquisition function, which guides the optimization, uses the surrogate probabilistic information to balance the exploration and the exploitation of the design space. In the case of multi-objective problems, current implementations use acquisition functions such as the multi-objective expected improvement (MEI). The evaluation of MEI requires a surrogate model for each objective function. In order to expand the Pareto front, such implementations perform a multi-variate integral over an intricate hypervolume, which require high computational cost. The objective of this work is to introduce an efficient multi-objective Bayesian optimization method that avoids the need for multi-variate integration. The proposed approach employs the working principle of multi-objective traditional methods, e.g., weighted sum and min-max methods, which transform the multi-objective problem into a single-objective problem through a functional mapping of the objective functions. Since only one surrogate is trained, this approach has a low computational cost. The effectiveness of the proposed approach is demonstrated with the solution of four problems: (1) an unconstrained version of the Binh and Korn test problem (convex Pareto front), (2) the Fonseca and Fleming test problem (non-convex Pareto front), (3) a three-objective test problem and (4) the design optimization of a sandwich composite armor for blast mitigation. The optimization algorithm is implemented in MATLAB and the finite element simulations are performed in the explicit, nonlinear finite element analysis code LS-DYNA. The results are comparable (or superior) to the results of the MEI acquisition function.

An Approach to Bayesian Optimization in Optimizing Weighted Tchebycheff Multi-Objective Black-Box Functions

2020 ◽  
Author(s):  
Arpan Biswas ◽  
Claudio Fuentes ◽  
Christopher Hoyle
Keyword(s):  
Process Model ◽  
Large Scale ◽  
Low Cost ◽  
Pareto Frontier ◽  
Black Box ◽  
Training Data ◽  
Bayesian Optimization ◽  
Future Research ◽  
Objective Functions ◽  
Multi Objective

Abstract Bayesian optimization (BO) is a low-cost global optimization tool for expensive black-box objective functions, where we learn from prior evaluated designs, update a posterior surrogate Gaussian process model, and select new designs for future evaluation using an acquisition function. This research focuses upon developing a BO model with multiple black-box objective functions. In the standard multi-objective optimization problem, the weighted Tchebycheff method is efficiently used to find both convex and non-convex Pareto frontier. This approach requires knowledge of utopia values before we start optimization. However, in the BO framework, since the functions are expensive to evaluate, it is very expensive to obtain the utopia values as a priori knowledge. Therefore, in this paper, we develop a Multi-Objective Bayesian Optimization (MO-BO) framework where we calibrate with Multiple Linear Regression (MLR) models to estimate the utopia value for each objective as a function of design input variables; the models are updated iteratively with sampled training data from the proposed multi-objective BO. The iteratively estimated mean utopia values are used to formulate the weighted Tchebycheff multi-objective acquisition function. The proposed approach is implemented in optimizing a thin tube design under constant loading of temperature and pressure, with multiple objectives such as minimizing the risk of creep-fatigue failure and design cost along with risk-based and manufacturing constraints. Finally, the model accuracy with and without MLR-based calibration is compared to the true Pareto solutions. The results show potential broader impacts, future research directions for further improving the proposed MO-BO model, and potential extensions to the application of large-scale design problems.

A MO-BAYESIAN OPTIMIZATION APPROACH USING THE WEIGHTED TCHEBYCHEFF METHOD

2021 ◽  
pp. 1-30
Author(s):  
Arpan Biswas ◽  
Claudio Fuentes ◽  
Christopher Hoyle
Keyword(s):  
Process Model ◽  
Low Cost ◽  
Black Box ◽  
Training Data ◽  
Bayesian Optimization ◽  
Optimization Approach ◽  
Objective Functions ◽  
Multi Objective ◽  
Input Variables ◽  
Future Evaluation

Abstract Bayesian optimization (BO) is a low-cost global optimization tool for expensive black-box objective functions, where we learn from prior evaluated designs, update a posterior surrogate Gaussian process model, and select new designs for future evaluation using an acquisition function. This research focuses upon developing a BO model with multiple black-box objective functions. In the standard Multi-Objective optimization (MOO) problem, the weighted Tchebycheff method is efficiently used to find both convex and non-convex Pareto frontiers. This approach requires knowledge of utopia values before we start optimization. However, in the BO framework, since the functions are expensive to evaluate, it is very expensive to obtain the utopia values as a prior knowledge. Therefore, in this paper, we develop a MO-BO framework where we calibrate with multiple linear regression (MLR) models to estimate the utopia value for each objective as a function of design input variables; the models are updated iteratively with sampled training data from the proposed multi-objective BO. This iteratively estimated mean utopia values is used to formulate the weighted Tchebycheff multi-objective acquisition function. The proposed approach is implemented in optimizing thin tube geometries under constant loading of temperature and pressure, with minimizing the risk of creep-fatigue failure and design cost, along with risk-based and manufacturing constraints. Finally, the model accuracy with frequentist, Bayesian and without MLR-based calibration are compared to true Pareto solutions.

scholarly journals Multi-fidelity black-box optimization for time-optimal quadrotor maneuvers

2021 ◽  
pp. 027836492110333
Author(s):  
Gilhyun Ryou ◽  
Ezra Tal ◽  
Sertac Karaman
Keyword(s):  
Real World ◽  
High Speed ◽  
Analytical Approximation ◽  
Black Box ◽  
Bayesian Optimization ◽  
Optimization Framework ◽  
Quadrotor Aircraft ◽  
Time Optimal ◽  
Feasibility Constraints ◽  
Generation Problem

We consider the problem of generating a time-optimal quadrotor trajectory for highly maneuverable vehicles, such as quadrotor aircraft. The problem is challenging because the optimal trajectory is located on the boundary of the set of dynamically feasible trajectories. This boundary is hard to model as it involves limitations of the entire system, including complex aerodynamic and electromechanical phenomena, in agile high-speed flight. In this work, we propose a multi-fidelity Bayesian optimization framework that models the feasibility constraints based on analytical approximation, numerical simulation, and real-world flight experiments. By combining evaluations at different fidelities, trajectory time is optimized while the number of costly flight experiments is kept to a minimum. The algorithm is thoroughly evaluated for the trajectory generation problem in two different scenarios: (1) connecting predetermined waypoints; (2) planning in obstacle-rich environments. For each scenario, we conduct both simulation and real-world flight experiments at speeds up to 11 m/s. Resulting trajectories were found to be significantly faster than those obtained through minimum-snap trajectory planning.

scholarly journals MOBOpt — multi-objective Bayesian optimization

SoftwareX ◽  
2020 ◽  
Vol 12 ◽  
pp. 100520 ◽  
Author(s):  
Paulo Paneque Galuzio ◽  
Emerson Hochsteiner de Vasconcelos Segundo ◽  
Leandro dos Santos Coelho ◽  
Viviana Cocco Mariani
Keyword(s):  
Bayesian Optimization ◽  
Multi Objective

Applications of Multiple Objective Genetic Algorithms in the Optimization Design of Tracked Self-Moving Power’s Layout

2013 ◽  
Vol 307 ◽  
pp. 161-165
Author(s):  
Hai Jin ◽  
Jin Fa Xie
Keyword(s):  
Optimization Design ◽  
Pareto Front ◽  
Layout Optimization ◽  
Multiple Objective ◽  
Multi Objective Optimization ◽  
Layout Problem ◽  
Basic Principles ◽  
Multi Objective ◽  
Multi Objective Genetic Algorithm ◽  
Set Up

A multi-objective genetic algorithm is applied into the layout optimization of tracked self-moving power. The layout optimization mathematical model was set up. Then introduced the basic principles of NSGA-Ⅱ, which is a Pareto multi-objective optimization algorithm. Finally, NSGA-Ⅱwas presented to solve the layout problem. The algorithm was proved to be effective by some practical examples. The results showed that the algorithm can spread toward the whole Pareto front, and provide many reasonable solutions once for all.

Sign in / Sign up

Export Citation Format

Share Document