Data augmented Approach to Optimizing Asynchronous Actor-Critic Methods

Abstract

Learning from visual observations of an environment is a core and fundamental problem in Reinforcement Learning (RL). Although there have been several advances in the algorithms, especially with the involvement of convolutional neural networks, they are primarily lacking in two aspects: (i) learning efficiency based on observations and (ii) learning generalization. Data augmentation has been shown to be a suitable strategy for enhancing the accuracy of classifier in Deep Learning solutions. With these in mind, this paper describes an implementation of Asynchronous Advantage Actor Critic (A3C) that integrates an optimized approach to observation augmentation policy on each learning batch. This approach is known as Data Augmented Reinforcement Learning (DARL). The proposed approach uses data augmentation to create environment variations to improve the learning policy of A3C with a key idea of data variety and demonstrates a significant improvement over the base implementation, with up to 70% increase in the rewards on several OpenAI Atari benchmarks.