MMSeg: A multimodal multi-scale point cloud segmentation model for navigable areas in complex field environments

Abstract

This study focuses on advanced navigable area perception technology to address critical applications such as battlefield support and emergency rescue for autonomous ground intelligent agents. It emphasizes its application in complex field environments characterized by unstructured, diverse, and intricate features. Current methods predominantly target structured environments, neglecting the unique challenges presented by unstructured terrains essential for critical applications such as battlefield support and emergency rescue missions. To address this gap, we propose three key contributions through a Multimodal Multi-scale point cloud Segmentation (MMSeg) model. First, we introduce a multimodal ground feature fusion technique that integrates geometric information from LiDAR point clouds with visual texture features from images, enhancing the recognition capabilities of heterogeneous ground surfaces. Second, we propose a local–global terrain geometry information enhancement method that utilizes a dual-attention mechanism to effectively capture and analyze both local and global geometric features in complex terrain conditions. Third, we design a multi-scale classifier framework that effectively processes the multimodal fused information of ground materials and terrain structures, enabling precise segmentation of navigable areas. An experiment on a dedicated platform demonstrates that the MMSeg model achieves mIoU 6% higher than commonly used point cloud segmentation models. These findings suggest that the MMSeg model significantly enhances the perception capabilities of autonomous ground intelligent agents in challenging environments, providing a promising and novel solution to improve their operational effectiveness in complex field conditions.