Deep reinforcement learning assisted surrogate model management for expensive constrained multi-objective optimization

Abstract

Expensive constrained multi-objective optimization problems (ECMOPs) exist in a wide variety of applications from industrial processes to engineering systems. When solving ECMOPs, with only a limited number of function evaluations available, a common approach is to substitute the real function evaluations with more affordable evaluations provided by computationally efficient surrogate models. However, existing surrogate assisted evolutionary algorithms (SAEAs) exhibit poor versatility in handling various ECMOPs, as they only use a constant surrogate modeling scheme or switch the modeling schemes with expert knowledge. To address the dilemma in surrogate modeling, this paper proposes a deep reinforcement learning assisted evolutionary algorithm, which operates on two key issues. First, multiple surrogate models are employed to learn the approximate function of an ECMOP using previously evaluated solutions during the evolutionary process. Second, a deep reinforcement learning method is employed to learn the optimal surrogate model management strategy based on evolutionary experience, selecting the most suitable surrogate modeling scheme for the current generation. Experimental evaluations on a large number of expensive problems demonstrate that the proposed algorithm has a significant effect compared with state-of-the-art competitors.