The Effect of Credit Definition and Aggregation Strategies on Multi-Objective Hyper-Heuristics

Heuristics and meta-heuristics are often used to solve complex real-world problems such as the non-linear, non-convex, and multi-objective combinatorial optimization problems that regularly appear in system design and architecture. Unfortunately, the performance of a specific heuristic is largely dependent on the specific problem at hand. Moreover, a heuristic’s performance can vary throughout the optimization process. Hyper-heuristics is one approach that can maintain relatively good performance over the course of an optimization process and across a variety of problems without parameter retuning or major modifications. Given a set of domain-specific and domain-independent heuristics, a hyper-heuristic adapts its search strategy over time by selecting the most promising heuristics to use at a given point. A hyper-heuristic must have: 1) a credit assignment strategy to rank the heuristics by their likelihood of producing improving solutions; and 2) a heuristic selection strategy based on the credits assigned to each heuristic. The literature contains many examples of hyper-heuristics with effective credit assignment and heuristic selection strategies for single-objective optimization problems. In multi-objective optimization problems, however, defining credit is less straightforward because there are often competing objectives. Therefore, there is a need to define and assign credit so that heuristics are rewarded for finding solutions with good trades between the objectives. This paper studies, for the first time, different combinations of credit definition, credit aggregation, and heuristic selection strategies. Credit definitions are based on different applications of the notion of Pareto dominance, namely: A1) dominance of the offspring with respect to the parent solutions; A2) ability to produce non-dominated solutions with respect to the entire population; A3) Pareto ranking with respect to the entire population. Two different credit aggregation strategies for assigning credit are also examined. A heuristic will receive credit for: B1) only the solutions it created in the current iteration or B2) all solutions it created that are in the current population. Different heuristic selection strategies are considered including: C1) probability matching; C2) dynamic multi-armed bandit; and C3) Hyper-GA. Thus, we conduct an experiment with three factors: credit definition (A1, A2, A3), credit aggregation (B1, B2), and heuristic selection (C1, C2, C3) and conduct a full factorial experiment. Performance is measured by hyper-volume of the last population. All algorithms are tested on a design problem for a climate monitoring satellite constellation instead of classical benchmarking problems to apply domain-specific heuristics within the hyper-heuristic.