HMFM: A Robust Hierarchical Multimodal Fusion Module to Enhance Map Learning With Additional Information

Abstract

High-definition (HD) maps, rich in geometric and semantic information, are essential for autonomous driving systems. Recent online map learning methods, predicting maps directly from onboard sensor data, offer promising alternatives to traditional manual approaches. However, they are limited in long-range detection and complex environments due to onboard sensors’ inherent constraints. While some works incorporate external priors, they often lack generality due to tight coupling with specific architectures and poor robustness to imperfect data. To address these gaps, we propose the Hierarchical Multimodal Fusion Module (HMFM), a robust, generalizable plug-and-play module that fuses onboard data with external information at two levels. At the feature level, the Masked Attention Module (MAM) selectively enhances features to boost fusion efficiency and minimize interference. At the Bird’s Eye View (BEV) level, the Alignment Module (AM) resolves spatial discrepancies; then, the Unified Attention Module (UAM) adaptively weighs external priors to effectively mitigate inaccuracies from imperfect data. We also introduce a distillation framework to distill benefits from additional information into the model without affecting runtime efficiency. Extensive experiments validate HMFM’s generality, leveraging satellite and SD maps for significant gains across baselines (e.g., +22.0 mIoU on HDMapNet, +12.3 mAP on MapTR). Furthermore, the distillation framework boosts the original HDMapNet by 2.8 mIoU without affecting runtime efficiency. The code and data processing pipeline for this work are publicly available at: https://github.com/xjtu-cs-gao/HMFM