An offline-online learning framework combining meta-learning and reinforcement learning for evolutionary multi-objective optimization

Abstract

Many multi-objective evolutionary algorithms (MOEAs) have been proposed in addressing the multi-objective optimization problems (MOPs). However, the performance of MOEAs varies significantly across various MOPs and there is no single MOEA that performs well on all MOP instances. In addition, existing methods for adaptive MOEA selection still face limitations, which restrict the further optimization for MOPs. To fill these gaps and improve the efficiency of solving MOPs, this study proposes an offline-online learning framework combining meta-learning and reinforcement learning (O²-MRL). Instead of proposing a new MOEA or optimizing a strategy, O²-MRL solves MOPs by taking full advantage of the existing MOEAs and addresses the limitations of existing MOEA selection methods. O²-MRL can adaptively select the appropriate MOEAs for various types of MOPs with different dimensions (Offline) and automatically schedule the selected MOEAs during the optimization process (Online), offering a new idea for optimizing MOPs. To evaluate the performance of the proposed O²-MRL, forty-seven benchmark MOPs are used as instances, and nine representative MOEAs are selected for comparison. Comprehensive experiments demonstrate the significant efficiency of O²-MRL, as it achieves optimal solutions in 60.28 % of the MOPs across different dimensions and improves the optimization results in 48.23 % of them, with an average improvement of 8.72 %. In addition to maintaining high optimization performance, O²-MRL also demonstrates superior convergence speed and stability across various types of MOPs. Two real-world MOPs are employed to evaluate the practicality of O²-MRL, and the experimental results indicate that it achieves optimal solutions in both cases.