Performance Investigation of UCB Policy in Q-learning

In this paper, we investigated performance and usability of UCBQ algorithm proposed in previous research. This is the algorithm that UCB, which is one of bandit algorithms, is applied to Q-Learning, and can balance between exploitation and exploration. We confirmed in the previous research that it was able to realize effective learning in a partially observable Markov decision process by using a continuous state spaces shortest path problem. We numerically examined it by using a variety of simpler learning situation which is the 2 dimensional goal search problem in a Markov decision process, comparing to previous methods. As a result, we confirmed that it had a better performance than other methods.