Modified Annealed Adversarial Bonus for Adversarially Guided Actor-Critic

Abstract

This paper investigates learning efficiency for rein-forcement learning in procedurally generated environments. A more sophisticated method is proposed to adjust the adversarial bonus to promote learning efficiency instead of the linearly decayed scheme in adversarially guided actor-critic. Our method considers the relationship between the bonus adjustment and the learning procedure. In some environments, if an agent performs better in learning, the agent will reach the goal with fewer steps. If the length of the episode decreases, the adversarial bonus will be reduced in our method. In this way, the learning efficiency has been improved in some procedurally generated tasks. Several experiments are implemented in MiniGrid to verify the proposed method. In the experiments, the proposed method outperforms the existing adversarially guided methods in several challenging procedurally-generated tasks.