Distributed reinforcement learning for multiple objective optimization problems

Abstract

This paper describes the application and performance evaluation of a new algorithm for multiple objective optimization problems (MOOP) based on reinforcement learning. The new algorithm, called MDQL, considers a family of agents for each objective function involved in a MOOP. Each agent proposes a solution for its corresponding objective function. Agents leave traces while they construct solutions considering traces made by other agents. The solutions proposed by the agents are evaluated using a non-domination criterion and solutions in the final Pareto set for each iteration are rewarded. A mechanism for the application of MDQL in continuous spaces which considers a fixed set of possible actions for the states (the number of actions depends on the dimensionality of the MOOP), is also proposed. Each action represents a path direction and its magnitude is changed dynamically depending on the evaluation of the state that the agent reached. Constraint handling, based on reinforcement comparison, considers reference values for constraints, penalizing agents violating any of them proportionally to the violation committed. MDQL performance was measured with "error ratio" and "spacing" metrics on four test bed problems suggested in the literature, showing competitive results with state-of-the-art algorithms.