Multi-Agent Collaborative Exploration through Graph-based Deep Reinforcement Learning

Abstract

Autonomous exploration by a single or multiple agents in an unknown environment leads to various applications in automation, such as cleaning, search and rescue, etc. Traditional methods normally take frontier locations and segmented regions of the environment into account to efficiently allocate target locations to different agents to visit. They may employ ad hoc solutions to allocate the task to the agents, but the allocation may not be efficient. In the literature, few studies focused on enhancing the traditional methods by applying machine learning models for agent performance improvement. In this paper, we propose a graph-based deep reinforcement learning approach to effectively perform multi-agent exploration. Specifically, we first design a hierarchical map segmentation method to transform the environment exploration problem to the graph domain, wherein each node of the graph corresponds to a segmented region in the environment and each edge indicates the distance between two nodes. Subsequently, based on the graph structure, we apply a Graph Convolutional Network (GCN) to allocate the exploration target to each agent. Our experiments show that our proposed model significantly improves the efficiency of map explorations across varying sizes of collaborative agents over the traditional methods.