Deep Reinforcement Learning Based on Search Space Independent Operators for Black-Box Continuous Optimization

Abstract

Deep reinforcement learning (DRL) has demonstrated exceptional capabilities in combinatorial optimization, which automatically devises policies for solution construction and optimizer refinement. DRL is particularly adept in generating training samples by itself, thereby providing the flexibility to solve a variety of combinatorial optimization problems without supervision. While DRL takes actions according to states extracted from problem-specific information, it cannot be directly applied to black-box continuous optimization lacking explicit information. To address this issue, this paper proposes a search space independent operator based DRL method for black-box continuous optimization. It conceptualizes the optimization process driven by search space independent operators as a Markov decision process, wherein actions are defined as operators and states are extracted from solutions generated by operators. In contrast to other DRL-assisted metaheuristics, the proposed method does not rely on any existing metaheuristic. Instead, it innovates by creating totally new operators, able to surpass the performance boundaries of existing metaheuristics. Compared with state-of-the-art meta-heuristics and DRL methods, the proposed method shows significantly faster convergence speed on challenging continuous optimization problems.