FPGA hardware implementation of Q-learning algorithm with low resource consumption

Abstract

Q-learning is a kind of reinforcement learning, having a wide range of applications varying in different fields. However, in some circumstances like robot control which has shorter training time requirement, Q-learning algorithm implemented on GPU or CPU may not meet the requirement. In this paper, we proposed a novel serial acceleration architecture for Q-learning algorithm and implemented the architecture on xczu7ev-ffvc1156 FPGA using Vivado 2019.1 development environment. As a result, the resource consumption is reduced by about 50% compared with the architecture proposed in [1],and the update cycle of Q-learning algorithm is fixed to 4 clock cycles.