Curiosity-Driven Exploration Effectiveness on Various Environments

Abstract

Hand Crafting Reward functions have never been scalable solutions for real world problems. The self-generated intrinsic rewards inspired by human curiosity may be one of scalable answers to solve sparse reward problem. The research thus investigated the effectiveness of some selected techniques based on the theory of curiosity-driven exploration. The Count-based, Prediction-based and other methods in total of six algorithms were experimented on various OpenAI gym environments. The results showed that the exploration algorithms have an impact on software agent in ability to find optimal solutions compared with the baseline in many cases. Still, there is no clear winner between the selected exploration methods and the best scalable exploration is not yet explored. The finding is that the added small intrinsic reward noise helps improve sample efficiency in the short run.