Reinforcement knowledge sharing assisted two-archive evolutionary algorithm for many-objective optimization

Abstract

In many-objective optimization problems (MaOPs), algorithms are challenged in terms of convergence pressure and exploration of the complete Pareto front (PF) as the number of objectives increases. The two-archive mechanism currently offers a novel perspective to address this issue. However, most existing two-archive-based many-objective optimization algorithms focus on independently updating the convergence archive (CA) and diversity archive (DA), while paying less attention to deeper cooperation between the two archives. To facilitate deeper cooperation, this paper proposes a reinforcement knowledge sharing assisted two-archive evolutionary algorithm for many-objective optimization (RKS-TAEA). In RKS-TAEA, a generalized SDE indicator (SDEp) and a new shift-based indicator (SBI) are proposed respectively for the update of CA and DA. SDEp could well maintain the properties of the original SDE indicator on estimating population convergence, while SBI could comprehensively assess not only diversity but also convergence of candidate solutions. Both SDEp and SBI could flexibly fit MaOPs with different PF geometries once the $p$-value is properly set for the Minkowski distance calculated in the two indicators. Thereafter, a reinforcement knowledge-sharing mechanism is proposed to derive the $p$-value from the knowledge factor that is learnt by fitting the PF geometry of the MaOP generation by generation. The reinforcement knowledge-sharing mechanism achieves deeper cooperation between the two archives, which ensures that RKS-TAEA could adaptively fit complex MaOPs that have different PF geometries. Comprehensive experiments on four benchmark test suites and five real-world MaOPs demonstrate that RKS-TAEA is more competitive in comparison with some state-of-the-art many-objective evolutionary algorithms.