Knowledge-based hyper-parameter adaptation of multi-stage differential evolution by deep reinforcement learning

Abstract

Differential evolution (DE) is a prominent algorithm in evolutionary computation, with adaptive control mechanisms for its operators and parameters being a critical research focus due to their impact on performance. Existing studies often rely on trial-and-error methods or deep reinforcement learning (DRL) for per-generation adaptive control, yet they inadequately explore adaptive hyper-parameter tuning across different stages of the evolution process. To address this limitation, this paper presents a knowledge-based framework named DRL-HP-* for multi-stage DE hyper-parameter adaptation using DRL. The framework divides the algorithm’s search procedure into multiple equal stages, where a DRL agent determines hyper-parameters in each stage based on five types of states that characterize the evolutionary process. A novel reward function is designed to comprehensively train the agent across all training functions, integrating the performance of the backbone algorithm. This approach results in the development of three new algorithms (DRL-HP-jSO, DRL-HP-LSHADE-RSP, and DRL-HP-EjSO). Experimental evaluations on the CEC’18 benchmark suite demonstrate that the proposed algorithms outperform eight state-of-the-art methods, demonstrating superior optimization performance. Further extensive experiments validate the effectiveness of the designed reward function and the framework’s scalability and robustness, highlighting its contribution to enabling stage-wise adaptive hyper-parameter control.