ParamILS: An Automatic Algorithm Configuration Framework

The identification of performance-optimizing parameter settings is an important part of the development and application of algorithms. We describe an automatic framework for this algorithm configuration problem. More formally, we provide methods for optimizing a target algorithms performance on a given class of problem instances by varying a set of ordinal and/or categorical parameters. We review a family of local-search-based algorithm configuration procedures and present novel techniques for accelerating them by adaptively limiting the time spent for evaluating individual configurations. We describe the results of a comprehensive experimental evaluation of our methods, based on the configuration of prominent complete and incomplete algorithms for SAT. We also present what is, to our knowledge, the first published work on automatically configuring the CPLEX mixed integer programming solver. All the algorithms we considered had default parameter settings that were manually identified with considerable effort. Nevertheless, using our automated algorithm configuration procedures, we achieved substantial and consistent performance improvements.

Download Full-text

Unbalanced Budget Distribution for Automatic Algorithm Configuration

10.21203/rs.3.rs-340470/v1 ◽

2021 ◽

Author(s):

Soheila Ghambari ◽

Hojjat Rakhshani ◽

Julien Lepagnot ◽

Laetitia Jourdan ◽

Lhassane Idoumghar

Keyword(s):

Random Search ◽

Time Budget ◽

Bayesian Optimization ◽

Test Problems ◽

Nsga Ii ◽

Automatic Algorithm Configuration ◽

Empirical Performance ◽

Algorithm Configuration ◽

Problem Instances ◽

Automatic Algorithm

Abstract Optimization algorithms often have several critical setting parameters and the improvement of the empirical performance of these algorithms depends on tuning them. Manually configuration of such parameters is a tedious task that results in unsatisfactory outputs. Therefore, several automatic algorithm configuration frameworks have been proposed to regulate the parameters of a given algorithm for a series of problem instances. Although the developed frameworks perform very well to deal with various problems, however, there is still a trade-off between the accuracy and budget requirements that need to be addressed. This work investigates the performance of unbalanced distribution of budget for different configurations to deal with the automatic algorithm configuration problem. Inspired by the bandit-based approaches, the main goal is to find a better configuration that substantially improves the performance of the target algorithm while using a smaller run time budget. In this work, non-dominated sorting genetic algorithm II (NSGA-II) is employed as a target algorithm using jMetalPy software platform and the multimodal multi-objective optimization (MMO) test suite of CEC'2020 is used as a set of test problems. We did a comprehensive comparison with other known methods including random search, Bayesian optimization, SMAC, ParamILS, irace, and MAC methods. The experimental results interestingly proved the efficiency of the proposed approach for automatic algorithm configuration with a minimum time budget in comparison with other competitors.

Download Full-text

Greybox Algorithm Configuration

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/669 ◽

2021 ◽

Author(s):

Marie Anastacio

Keyword(s):

State Of The Art ◽

Automated Algorithm ◽

Wide Range ◽

Algorithm Configuration ◽

Improve Algorithm ◽

The Difference ◽

Problem Instances ◽

The Right ◽

Parameter Values ◽

Configuration Problem

The performance of state-of-the-art algorithms is highly dependent on their parameter values, and choosing the right configuration can make the difference between solving a problem in a few minutes or hours. Automated algorithm configurators have shown their efficiency on a wide range of applications. However, they still encounter limitations when confronted to a large number of parameters to tune or long algorithm running time. We believe that there is untapped knowledge that can be gathered from the elements of the configuration problem, such as the default value in the configuration space, the source code of the algorithm, and the distribution of the problem instances at hand. We aim at utilising this knowledge to improve algorithm configurators.

Download Full-text