Multi-modal soft prompt-tuning for Chinese Clickbait Detection

With the rapid growth of Chinese online services, clickbait has proliferated at an unprecedented rate, designed to manipulate users into clicking for increased traffic or advertising promotion. Such clickbait not only facilitates the spread of fake news and misinformation but also enables click-jacking attacks, redirecting users to deceptive websites that steal personal information. These harmful activities can result in significant losses and serious repercussions. The widespread presence of clickbait underscores both the importance and the challenges of developing effective detection methods. To date, the research paradigm of clickbait detection evolved from deep neural networks to fine-tuned Pre-trained Language Models (PLMs) and, more recently, into prompt-tuning models. However, these methods may suffer two main limitations: (1) they fail to utilize the multi-modal context information in news or posts and explore the higher-level feature representations to enhance the performance of clickbait detection; (2) they largely ignore the diverse range of Chinese expressive forms and neglect the complex semantics and syntactic structures of textual content to assist in learning a better news representation. To overcome these limitations, we proposed a Multi-modal Soft Prompt-tuning Method (MSP) for Chinese Clickbait Detection, which jointly models the textual and image information into a continuous prompt embedding as the input of PLMs. Specifically, firstly, the soft prompt-tuning model including Graph Attention Network and Contrastive Language-Image Pre-training are employed to learn the feature representations of texts and images in news or posts, respectively. Then the obtained text and image representations are re-input into the soft prompt-tuning model with automatic template generation. The extensive experiments on three Chinese clickbait detection datasets demonstrate that our MSP achieved state-of-the-art performance.