Entropy-based tuning approach for Q-learning in an unstructured environment

Abstract

In reinforcement learning applications, achieving a balance between exploration and exploitation is a crucial problem during the learning process. This study proposes an entropy-based tuning approach that uses the value different based exploration theory is proposed to solve this problem in an unstructured environment. In such an environment, a learning agent can manage its exploration rates in each state instead of using a constant rate for all states. Moreover, some obstacles may block the agent’s path to the destination. Accordingly, the proposed approach enables the agent to adaptively increase its exploration rates in some states undergoing transitions; thus, the agent is encouraged to explore in those states. This paper presents simulations of maze environments and the car parking problem to verify the proposed approach. The simulation results demonstrate that our approach enables the agent to adjust its policy quickly to adapt to changing environments.