iAMeta: Advancing multimodal metaphor detection using MLLMs and information debiasing

Abstract

With the widespread use of multimodal data, metaphors are increasingly expressed through the combination of images and text, leading to the emergence of multimodal metaphor detection research. However, existing methods face challenges such as missing contextual information and information bias, hindering accurate metaphor interpretation. To address these issues, we propose iAMeta, a multimodal metaphor detection framework based on multimodal large language models (MLLMs). This framework introduces a knowledge generator inspired by contrastive thinking, enabling gradual inference of metaphor, non-metaphor, and overall metaphor prior knowledge. A multitask learning-based sentiment feature control mechanism is employed to calibrate sentimental bias caused by prior knowledge interference and ensure that the extracted sentiment features are consistent with the original emotional tone. Additionally, a causal reasoning framework is introduced to reduce false associations between images and labels, further enhancing the model’s generalization ability. Experimental results demonstrate that iAMeta excels in both multimodal metaphor detection and sentiment analysis tasks and performs well in handling complex scenarios like sarcasm.