Multimodal Fusion-Based Channel Prediction and Characterization for mmWave UAV A2G Communications

Abstract

A channel prediction modeling method based on multimodal fusion perception is proposed for complex environments in unmanned aerial vehicle (UAV) air-to-ground (A2G) communications. Two-dimensional (2D) environmental information and three-dimensional (3D) point cloud data are fused to enhance the model’s ability to capture environment blockage, reflection, and multipath effects. The 2D information provides target objects’ planar distribution and texture features, while the 3D point clouds supplement spatial position, size, and height information. These complementary modalities comprehensively describe the geometric structures present in complex environments. A multimodal modeling network is constructed to explore the nonlinear mapping between environmental perception data and channel data. The network takes 2D building distribution, 3D point cloud data, global image features, UAV and receiver positions, and communication parameters as joint inputs. Feature extraction and fusion modules achieve effective joint encoding of heterogeneous multimodal features. A spatial feature decoupling (SFD) module is designed to address interference caused by coupled features. It separates the data distributions corresponding to different channel characteristics, improving the accuracy of channel impulse response (CIR) prediction. Experimental results demonstrate that the proposed method significantly improves the reliability and adaptability of UAV channel modeling in complex urban scenarios.