Hierarchical Signal Calibration and Refinement for Multimodal Sentiment Analysis

Abstract

To address the issues of noise amplification and feature incompatibility arising from modal heterogeneity in multimodal sentiment analysis, this paper proposes a hierarchical optimization framework. In the first stage, we introduce the Semantic-Guided Calibration Network (SGC-Net), which, through a Dynamic Balancing Regulator (DBR), leverages textual semantics to intelligently weight and calibrate the cross-modal interactions of audio and video, thereby suppressing noise while preserving key dynamics. In the second stage, the Synergistic Refinement Fusion Module (SRF-Module) performs a deep refinement of the fused multi-source features. This module employs a Saliency-Gated Complementor (SGC) to rigorously filter and exchange effective information across streams, ultimately achieving feature de-redundancy and strong complementarity. Extensive experiments on the CMU-MOSI and CMU-MOSEI datasets validate the effectiveness of our method, with the model achieving state-of-the-art performance on key metrics such as binary accuracy (Acc-2: 86.73% on MOSI, 86.52% on MOSEI) and seven-class accuracy (Acc-7: 48.35% on MOSI, 53.81% on MOSEI).