Visual Sparse Bayesian Reinforcement Learning: A Framework for Interpreting What an Agent Has Learned

Abstract

This paper presents a Visual Sparse Bayesian Reinforcement Learning (V-SBRL) framework for recording the images of the most important memories from the past experience. The key idea of this paper is to maintain an image snapshot storage to help understanding and analyzing the learned policy. In the extended framework of SBRL [1], the agent perceives the environment as the image state inputs, encodes the image into feature vectors, train SBRL module and stores the raw images. In this process, the snapshot storage keeps only the relevant memories which are important to make future decisions and discards the not-so-important memories. The stored snapshot images enable us to understand the agent’s learning process by visualizing them. They also provide explanation of exploited policy in different conditions. A navigation task with static obstacles is examined for snapshot analysis.