Perceptual visual security index: Analyzing image content leakage for vision language models

IF 3.8 2区 计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS
Lishuang Hu , Tao Xiang , Shangwei Guo , Xiaoguo Li , Ying Yang
{"title":"Perceptual visual security index: Analyzing image content leakage for vision language models","authors":"Lishuang Hu ,&nbsp;Tao Xiang ,&nbsp;Shangwei Guo ,&nbsp;Xiaoguo Li ,&nbsp;Ying Yang","doi":"10.1016/j.jisa.2025.103988","DOIUrl":null,"url":null,"abstract":"<div><div>During the training phase of vision language models (VLMs), the privacy storage and sharing of images are of paramount importance. While the Visual Security Index (VSI) is commonly used for content leakage analysis, it usually focuses on comparing content similarity between plain and protected or encrypted images, neglecting the threat model of visual security. In this paper, considering the functionality of the human visual capability, we comprehensively analyze the system model of VSIs and propose a novel perceptual visual security index (PVSI) to evaluate the content leakage of perceptually encrypted images for VLMs. In particular, we take visual perception (<strong>VP</strong>) as the adversary’s capability and present the definition of VSI under an honest-but-curious threat model. To evaluate the content leakage of encrypted images under the <strong>VP</strong> assumption, we first present a robust feature descriptor and obtain the semantic content sets of both plain and encrypted images. Then, we propose a systematic method to reduce the impact of different encryption algorithms. We further evaluate the similarity between semantic content sets to obtain the proposed PVSI. We also analyze the consistency between the proposed visual security definition and PVSI. Extensive experiments are performed on five publicly available image databases. Our experimental results demonstrate that compared with many existing state-of-the-art visual security metrics, the proposed PVSI exhibits better performance not only on images generated from specific image encryption algorithms but also on publicly available image databases.</div></div>","PeriodicalId":48638,"journal":{"name":"Journal of Information Security and Applications","volume":"89 ","pages":"Article 103988"},"PeriodicalIF":3.8000,"publicationDate":"2025-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Information Security and Applications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214212625000262","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

During the training phase of vision language models (VLMs), the privacy storage and sharing of images are of paramount importance. While the Visual Security Index (VSI) is commonly used for content leakage analysis, it usually focuses on comparing content similarity between plain and protected or encrypted images, neglecting the threat model of visual security. In this paper, considering the functionality of the human visual capability, we comprehensively analyze the system model of VSIs and propose a novel perceptual visual security index (PVSI) to evaluate the content leakage of perceptually encrypted images for VLMs. In particular, we take visual perception (VP) as the adversary’s capability and present the definition of VSI under an honest-but-curious threat model. To evaluate the content leakage of encrypted images under the VP assumption, we first present a robust feature descriptor and obtain the semantic content sets of both plain and encrypted images. Then, we propose a systematic method to reduce the impact of different encryption algorithms. We further evaluate the similarity between semantic content sets to obtain the proposed PVSI. We also analyze the consistency between the proposed visual security definition and PVSI. Extensive experiments are performed on five publicly available image databases. Our experimental results demonstrate that compared with many existing state-of-the-art visual security metrics, the proposed PVSI exhibits better performance not only on images generated from specific image encryption algorithms but also on publicly available image databases.
感知视觉安全指数:分析视觉语言模型的图像内容泄漏
在视觉语言模型的训练阶段,图像的隐私存储和共享是至关重要的。视觉安全指数(Visual Security Index, VSI)通常用于内容泄露分析,但它通常侧重于比较普通图像与受保护或加密图像之间的内容相似性,而忽略了视觉安全的威胁模型。本文从人类视觉功能的角度出发,综合分析了感知加密图像的系统模型,提出了一种新的感知加密图像的视觉安全指数(PVSI)来评估感知加密图像的内容泄露。特别地,我们将视觉感知(VP)作为对手的能力,并在诚实但好奇的威胁模型下给出了视觉感知的定义。为了评估VP假设下加密图像的内容泄漏,我们首先提出了一个鲁棒特征描述符,并获得了加密图像和普通图像的语义内容集。然后,我们提出了一种系统的方法来减少不同加密算法的影响。我们进一步评估语义内容集之间的相似性,以获得所提出的PVSI。我们还分析了所提出的可视化安全定义与PVSI之间的一致性。在五个公开的图像数据库上进行了大量的实验。我们的实验结果表明,与许多现有的最先进的视觉安全指标相比,所提出的PVSI不仅在特定图像加密算法生成的图像上表现出更好的性能,而且在公开可用的图像数据库上也表现出更好的性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Information Security and Applications
Journal of Information Security and Applications Computer Science-Computer Networks and Communications
CiteScore
10.90
自引率
5.40%
发文量
206
审稿时长
56 days
期刊介绍: Journal of Information Security and Applications (JISA) focuses on the original research and practice-driven applications with relevance to information security and applications. JISA provides a common linkage between a vibrant scientific and research community and industry professionals by offering a clear view on modern problems and challenges in information security, as well as identifying promising scientific and "best-practice" solutions. JISA issues offer a balance between original research work and innovative industrial approaches by internationally renowned information security experts and researchers.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信