Extending Evolution-Guided Policy Gradient Learning into the multi-objective domain

Abstract

Multi-Objective Reinforcement Learning (MORL) poses significant challenges, primarily due to the necessity of balancing conflicting objectives—a limitation that traditional single-objective approaches fail to address. This paper introduces Multi-Objective Evolutionary Reinforcement Learning (MO-ERL), the first adaptation of Evolutionary Reinforcement Learning (ERL) specifically designed to address the complexities of the multi-objective domain effectively.

MO-ERL integrates policy gradient-based reinforcement learning (RL), which optimizes expected utility, with evolutionary algorithms (EAs) that maintain diversity across the Pareto front. This combination leverages RL’s strength in exploitation and EAs’ proficiency in exploration, enabling MO-ERL to effectively navigate the trade-offs inherent in multi-objective optimization problems.

Evaluation on multi-objective continuous control tasks using the MuJoCo physics engine demonstrates that MO-ERL outperforms state-of-the-art baselines, achieving up to 62.71% higher hypervolume and 196.28% greater expected utility. These results validate MO-ERL’s ability to balance solution diversity and optimality, setting a new benchmark for solving MORL tasks.