Gradient-Aware Adaptive Meta-Prompt Learner for Generalizable Face Forgery Detection

Abstract

The misuse of AI-generated techniques in face forgery has raised significant concerns, driving advancements in detection methods. However, existing algorithms struggle with generalization in cross-domain scenarios due to domain shifts, limiting their practical applications. Prompt tuning, which learns soft prompts while freezing the backbone, enables the generalizable Vision-Language Models (VLMs) pre-trained on large-scale datasets to adapt to downstream tasks. Though effective, prompt tuning confronts challenges in face forgery detection, where its performance is sensitive to initialization and may undermine the generalizability of pre-trained VLMs. To address this issue, we propose a novel Gradient-aware Adaptive Meta-Prompt Learner (GAMP-Learner). The core idea is to learn a meta-general gradient from multiple source domains through the Direction-shared Gradient Pruning Module (DGPM) for efficient initialization in the inner-loop, while addressing gradient conflicts via the Adaptive Gradient Calibration Module (AdaGCM) to enhance generalization in the outer-loop. Notably, our GAMP-Learner can be seamlessly integrated into any prompt-based fine-tuning VLM in a model-agnostic way. Additionally, to capture fine-grained forgery clues, we design a Multi-Granularity Conditional Prompt Generator (MGCP), which constructs instance-level prompts by incorporating multi-scale content-style feature representations. Simulating practical scenarios, we devise three protocols which evaluate generalization performance trained on multiple source domains. Extensive experiments demonstrate that the proposed framework achieves competitive cross-domain detection performance compared to state-of-the-art methods.