Deep Reinforcement Learning for Zero-Shot Coverage Path Planning With Mobile Robots

Abstract

The ability of mobile robots to plan and execute a path is foundational to various path-planning challenges, particularly Coverage Path Planning. While this task has been typically tackled with classical algorithms, these often struggle with flexibility and adaptability in unknown environments. On the other hand, recent advances in Reinforcement Learning offer promising approaches, yet a significant gap in the literature remains when it comes to generalization over a large number of parameters. This paper presents a unified, generalized framework for coverage path planning that leverages value-based deep reinforcement learning techniques. The novelty of the framework comes from the design of an observation space that accommodates different map sizes, an action masking scheme that guarantees safety and robustness while also serving as a learning-from-demonstration technique during training, and a unique reward function that yields value functions that are size-invariant. These are coupled with a curriculum learning-based training strategy and parametric environment randomization, enabling the agent to tackle complete or partial coverage path planning with perfect or incomplete knowledge while generalizing to different map sizes, configurations, sensor payloads, and sub-tasks. Our empirical results show that the algorithm can perform zero-shot learning scenarios at a near-optimal level in environments that follow a similar distribution as during training, outperforming a greedy heuristic by sixfold. Furthermore, in out-of-distribution environments, our method surpasses existing state-of-the-art algorithms in most zero-shot and all few-shot scenarios, paving the way for generalizable and adaptable path-planning algorithms.