A reinforcement learning approach to cooperative problem solving

Abstract

We propose an extension of reinforcement learning methods to cooperative problem solving in multi agent systems. Exploiting multiple agents for complex problems is promising, however, learning is necessary since complete domain knowledge is rarely available. The temporal difference algorithm is applied in each agent to learn a heuristic evaluation of states. Besides the reward for solutions produced by agents, we define the reward for coherence as a whole and exploit them to facilitate cooperation among agents for global problem solving. We evaluate the method by experiments on the satellite design problem. The result shows that our method enables agents to learn to cooperate as well as to learn individual heuristics within one framework. Especially, agents themselves learn to take the appropriate balance between exploration and exploitation in problem solving, which is known to greatly affect the performance. It also suggests the possibility of controlling the global behavior of multi agent systems via rewards in reinforcement learning.