MATdiff: Learning diffusion policy with multi-auxiliary task for mobile robot visual exploration

The application of diffusion models into the field of robotics is gaining increasing attention due to its advantages in modeling complex data distributions. In the visual navigation task of mobile robots based on diffusion policy, existing frameworks use the current observation as the guidance condition and adopt a classifier free guidance mode for joint training. However, using diffusion models for end-to-end training may result in feature loss, as the learned features are not well understood, which leading to poor generalization in unknown environments and low navigation success rates. To address the issue of generalization, we proposed a new visual navigation framework called MATdiff from the perspective of visual representation. Our framework utilizes two auxiliary tasks to enhance the representation capability of the Conditioned Observation Network. It leverages depth estimation to extract the geometric features of the environment and employs free-space segmentation to identify safely drivable regions, which are defined as areas free from obstacles and suitable for safe navigation. After the fusion of those features, we use a conditional diffusion model to model the distribution under observation conditions and generate a fixed number of consecutive waypoints. This design of auxiliary tasks ensures that the conditional features pays attention to both geometric and semantic information simultaneously. We conduct experiments in both simulation environments and the real world. Compared with the state-of-the-art methods, our method not only has lighter model parameters but also achieves the highest navigation success rate and a longer average travel distance before collision.