Cross-Modal Synergistic Optimization Multi-Task Segmentation Network for Autonomous Ground Intelligent Agents in Field Environments

Abstract

This study focuses on the perception of traversable areas and ground materials in complex field environments, driven by the application demands of autonomous ground intelligent agents in critical tasks such as battlefield support and disaster relief. Compared to existing studies on fine-grained segmentation in structured environments (e.g., urban roads), perception in unstructured field environments is far more challenging due to irregular terrain and complex conditions, while research in this area remains limited. To address this, this paper proposes a cross-modal synergistic optimization multi-task segmentation network (CSOM-Net) for ground intelligent agents in field environments, based on an in-depth analysis of the technical requirements in complex field settings. This paper proposes a conflict-consistency guided feature fusion (CCGF) method to resolve the challenge of feature conflicts in multimodal data that hinder effective fusion in complex field environments. A cross-modal bidirectional mutual adaptation (CBMA) learning strategy is proposed to address the feature interference caused by significant differences in optimization objectives in cross-modal tasks under field environments. Extensive comparative and ablation experiments were conducted on both real-world and simulated complex field datasets to evaluate the proposed CSOM-Net model. Experimental results show that CSOM-Net outperforms conventional models, achieving approximately 5% and 2% mIoU improvements in point cloud traversable area and image ground material segmentation tasks, respectively. The model proves effective for joint segmentation tasks of point cloud traversable areas and image ground materials in complex field environments, providing a robust solution to enhance environmental perception capabilities for autonomous ground agents operating under challenging field conditions.