Three stage based reinforcement learning for combining multiple metaheuristic algorithms

Abstract

Combined of metaheuristic algorithms can effectively improve the performance of algorithms by utilizing the characteristics of different metaheuristic algorithms, and the key is how to combine multiple metaheuristic algorithms. Reinforcement learning is one of the effective methods for combining multiple metaheuristic algorithms. However, designing a competitive reinforcement learning approach to achieve efficient collaboration among metaheuristic algorithms is a highly challenging task. Therefore, this study proposes a three-stage reinforcement learning for combining multiple metaheuristic algorithms (TSRL-CMM). TSRL-CMM is divided into three stages: the exploration stage, the stage with both exploration and exploitation, and the exploitation stage. On this basis, an adaptive action selection strategy and a reward function are designed. The proposed action selection strategy can adaptively select appropriate metaheuristic algorithms based on the state of the population, achieving a balance between exploration and exploitation. The proposed reward function can effectively guide the population to transition to the expected state based on the iteration stage and state transitions. To verify the effectiveness of TSRL-CMM, we evaluated it using the CEC2017 test suite, nine real-world engineering design problems and six power system optimization problems. TSRL-CMM was compared with 10 state-of-the-art metaheuristic algorithms, and experimental results showed that TSRL-CMM performed better than the compared algorithms in both artificial and real-world problems. Furthermore, TSRL-CMM was specifically compared with three CEC winner algorithms on the CEC 2017 benchmark test suite. The experimental results show that the proposed algorithm is highly competitive. The source code can be obtained from https://github.com/xtongliu/TSRL-CMM-code.