Vision–LiDAR–Tactile Fusion Framework Optimized by Artificial Hummingbird Algorithm for Contact-Aware Embodied 3D Scene Perception

Abstract

Embodied intelligent robots demand comprehensive 3D scene understanding via multimodal sensor fusion for complex physical environments, yet existing methods struggle with insufficient heterogeneous data complementarity and inflexible fusion weight optimization. This study presents Artificial Hummingbird Algorithm (AHA)-fusion, a vision–LiDAR–tactile fusion framework optimized by the AHA for contact-aware embodied 3D scene perception. The framework integrates a multimodal feature extraction pipeline using ResNet-50, PointNet++, and lightweight CNNs; an improved Artificial Hummingbird Algorithm (I-AHA) with chaotic traversal and elite learning to optimize fusion weights globally; and a contact-aware dynamic weight modulation mechanism for real-time weight adjustment based on tactile contact states. A multilevel temporal synchronization strategy achieves 3.2 ± 1.1 ms alignment accuracy (Acc). Experiments on nuScenes, EmbodiedScan, and a self-built tactile manipulation dataset show AHA-Fusion attains 70.2% mean average precision (mAP) and 73.8% nuScenes detection score (NDS) for 3D object detection and 67.4% mean intersection over union (mIoU) for semantic segmentation, outperforming state-of-the-art methods. Ablation studies confirm I-AHA surpasses learnable attention weights by 2.3 percentage points with enhanced stability, while the contact-aware mechanism boosts contact region detection by 5.2 percentage points. A small-scale real-robot pilot experiment further suggests that the proposed framework can support online contact-aware manipulation, although broader real-world validation remains necessary. These results demonstrate the potential of optimization-guided trimodal fusion for robust embodied 3D perception.