A New Hyper-Heuristic Based on a Contextual Multi-Armed Bandit for Many-Objective Optimization

Hyper-Heuristics are high-level methodologies which select or generate heuristics. Despite their success, there are only few hyper-heuristics developed for many-objective optimization. Our approach, namely MOEA/D-LinUCB, combines the MOEA/D framework with a new selection hyper-heuristic to solve many-objective problems. It uses an innovative Contextual Multi-Armed Bandit (MAB) to determine the low level heuristic (Differential Evolution mutation strategy) that should be applied to each individual during MOEA/D execution. The main advantage of using Contextual MAB is to include information about the current search state into the selection procedure. We tested MOEA/D-LinUCB on a well established set of 9 instances from the WFG benchmark for a number of objectives varying from 3 to 20. The IGD indicator and Kruskal-Wallis and Dunn-Sidak's statistical tests are applied to evaluate the algorithm performance. Four variants of the proposed algorithm are compared with each other to define a proper configuration. A properly configured MOEA/D-LinUCB is then compared with MOEA/D-FRRMAB and MOEAID-DRA-two well-known MOEA/D-based algorithms. Results show that MOEA/D-LinUCB performs well, particularly when the number of objectives is 10 or greater. Therefore, MOEA/D-LinUCB can be considered as a promising many-objective Hyper-Heuristic.