Deep Reinforcement Learning Monitor for Snapshot Recording

Deep reinforcement learning (DRL) has been leading to state-of-the-art performance to learn control policies for a wide range of applications. However, it does not provide an explanation of how a policy is learned and how the learned policy performs on a given task. In this paper, we answer to the inquiry: what scenes does a machine learning agent need to memorize for efficient learning and additional explanation regarding performance? Proposing a monitoring model to record the most important moments from experience-called snapshot images-we examine them for analysis. Sparse Bayesian Reinforcement Learning (SBRL) is known to remember sparse input samples during training and to construct bases for value function approximation. Also, SBRL has successfully maintained the snapshot memory for sparse input sampling. We apply our method to a visual maze problem and Atari games to observe the recorded snapshot images. Analyzing the images, we evaluate the efficacy of the proposed monitoring model and the quality of collected snapshots.