Jailbreak attack with multimodal virtual scenario hypnosis for vision-language models

Due to the inherent vulnerabilities of large Vision-Language Models (VLMs), security governance has emerged as a critical concern, particularly given the risks posed by noisy and biased training data as well as adversarial attacks, including data poisoning and prompt injection. These perturbations can significantly degrade model performance and introduce multifaceted societal risks. To verify the safe robustness of VLMs and further inspire the design of defensive AI frameworks, we propose Virtual Scenario Hypnosis (VSH), a multimodal prompt injection jailbreak method that embeds malicious queries into prompts through a deceptive narrative framework. This approach strategically distracts the model while compromising its resistance to jailbreak attempts. Our methodology features two key innovations: 1) Targeted adversarial image prompts that transform textual content into visual layouts through optimized typographic designs, circumventing safety alignment mechanisms to elicit harmful responses; and 2) An information veil encrypted In-Context Learning (ICL) method for text prompts that systematically evades safety detection protocols. To streamline evaluation, we employ Large Language Models (LLMs) to facilitate an efficient assessment of jailbreak success rates, supported by a meticulously designed prompt template incorporating multi-dimensional scoring rules and evaluation metrics. Extensive experiments demonstrate the efficacy of VSH, achieving an overall success rate exceeding 82% on 500 harmful queries spanning multiple domains when tested against LLaVA-v1.5-13B and GPT-4o mini.