Dynamic instance sampling for multi-objective automatic algorithm configuration

Abstract

Multi-objective automatic algorithm configuration alleviates the tedious parameter tuning for users by optimizing both the performance and efficiency of the target algorithm. Its evaluation requires performing multiple runs for each configuration on an instance set, making the computational cost expensive. Especially for real-world application problems, it is crucial to reduce computational costs under limited budgets. However, when the instance set is large, model-based approaches struggle to further reduce the high cost of configuration evaluations, which remains a significant challenge. To address this, we propose a Kriging-assisted Two_Arch2 with dynamic instance sampling algorithm, which aims to reduce the high costs of configuration evaluations by lowering the fidelity of the evaluations. Specifically, we align the number of evaluation instances with the evaluation fidelity and design a dynamic instance sampling strategy to effectively control the frequency of new instance sampling, enabling fidelity switching. Moreover, a trade-off configuration selection method is proposed to assist users in choosing configurations when preferences are unclear. The proposed method has been tested on numerous instances from the BBOB benchmark suite. The experimental results demonstrate that the performance of the proposed algorithm outperforms other state-of-the-art methods.