Safe Explainable Agents for Autonomous Navigation using Evolving Behavior Trees

Abstract

Machine learning and reinforcement learning are increasingly used to solve complex tasks in autonomous systems. However, autonomous agents represented by large neural networks are not transparent leading to their assurability and trustworthiness becoming critical challenges. Large models also result in a lack of interpretability which causes severe obstacles related to trust in autonomous agents and human-machine teaming. In this paper, we leverage the hierarchical structure of behavior trees and hierarchical reinforcement learning to develop a neurosymbolic model architecture for autonomous agents. The proposed model, referred to as Evolving Behavior Trees (EBTs), integrates the required components to represent the learning tasks as well as the switching between tasks to achieve complex long-term goals. We design an agent for autonomous navigation and we evaluate the approach against a state-of-the-art hierarchical reinforcement learning method using a Maze Simulation Environment. The results show autonomous agents represented by EBTs can be trained efficiently. The approach incorporates explicit safety constraints into the model and incurs significantly fewer safety violations during training and execution. Further, the model provides explanations for the behavior of the autonomous agent by associating the state of the executing EBT with agent actions.