Constraint Handling in Bayesian Optimization -- A Comparative Study of Support Vector Machine, Augmented Lagrangian and Expected Feasible Improvement

Many engineering problems involve complex constraints which can be computationally costly. To reduce the overall numerical cost, such constrained optimization problems are solved via surrogate models constructed on a Design of Experiment (DoE). Meanwhile, complex constraints may lead to infeasible initial DoE, which can be problematic for subsequent sequential optimization. In this study, we address constrained optimization problem in a Bayesian optimization framework. A comparative study is conducted to evaluate the performance of three approaches namely Expected Feasible Improvement (EFI) and slack Augmented Lagrangian method (AL) and Expected Improvement with Probabilistic Support Vector Machine in constraint handling with feasible or infeasible initial DoE. AL is capable to start sequential optimization with infeasible initial DoE, while EFI requires extra a priori enrichment to find at least one feasible sample. Empirical experiments are performed on both analytical functions and a low pressure turbine disc design problem. Through these benchmark problems, EFI and AL are shown to have overall similar performance in problems with inequality constraints. However, the performance of EIPSVM is affected strongly by the corresponding hyperparameter values. In addition, we show evidences that with an appropriate handling of infeasible initial DoE, EFI does not necessarily underperform compared with AL solving optimization problems with mixed inequality and equality constraints.