MOEA/D with adaptive weight vector adjustment and parameter selection based on Q-learning

Multi-objective evolutionary algorithms (MOEAs) are widely utilized for addressing multi-objective optimization problems (MOPs), demonstrating effectiveness in handling low-dimensional and regular Pareto fronts (PFs) MOPs. However, when the number of objectives increases (>3) and the PFs become increasingly intricate, maintaining both the convergence and diversity of solutions presents a significant challenge. To address this, an adaptive weight vector adjustment and parameter selection based on Q-learning (QLMOEA/D-AWA) is proposed. In the algorithm, Q-learning is employed to select both the Tchebycheff value and the number of weight vectors, aiming to balance convergence and diversity. To enhance the convergence, an improved Tchebycheff approach is proposed. To better solve problems in high-dimensional objective spaces, the niche technique is adopted to retain elite individuals. In addition, to address MOPs with irregular PFs, a two-stage weight vector deletion strategy is proposed to remove invalid weight vectors, and a certain number of weight vectors are added based on sparsity rules. An experiment study of objective numbers ranging from 2 to 10 is conducted on DTLZ, WFG, MaF and multi-objective traveling salesman problem (MOTSP). Among 115 benchmark problems, QLMOEA/D-AWA achieves 54 and 49 best results in IGD and HV, respectively.