Monte Carlo Tree Search to Compare Reward Functions for Reinforcement Learning

Abstract

Reinforcement Learning has gained tremendous attention recently, thanks to its excellent solutions in several challenging domains. However, the formulation of the reward signal is always difficult and crucially important since it is the only guidance that the agent has for solving the given control task. Finding the proper reward is time-consuming since the model must be trained with all the potential candidates. Finally, a comparison has to be conducted. This paper proposes that the Monte-Carlo Tree Search algorithm can be used to compare and rank the different reward strategies. To see that the search algorithm can be used for such a task. A Policy Gradient algorithm is trained to solve the Traffic Signal Control problem with different rewarding strategies from the literature. The results show that both methods suggest the same order between the performances of the rewarding concepts. Hence the Monte-Carlo Tree Search algorithm can find the best reward for training, which seriously decreases the resource intensity of the entire process.