Anmol Manjunath, Viola Negroni, Sara Mandelli, Daniel Moreira, Paolo Bestagini
{"title":"在 Western 印迹图像中定位合成操作","authors":"Anmol Manjunath, Viola Negroni, Sara Mandelli, Daniel Moreira, Paolo Bestagini","doi":"arxiv-2408.13786","DOIUrl":null,"url":null,"abstract":"Recent breakthroughs in deep learning and generative systems have\nsignificantly fostered the creation of synthetic media, as well as the local\nalteration of real content via the insertion of highly realistic synthetic\nmanipulations. Local image manipulation, in particular, poses serious\nchallenges to the integrity of digital content and societal trust. This problem\nis not only confined to multimedia data, but also extends to biological images\nincluded in scientific publications, like images depicting Western blots. In\nthis work, we address the task of localizing synthetic manipulations in Western\nblot images. To discriminate between pristine and synthetic pixels of an\nanalyzed image, we propose a synthetic detector that operates on small patches\nextracted from the image. We aggregate patch contributions to estimate a\ntampering heatmap, highlighting synthetic pixels out of pristine ones. Our\nmethodology proves effective when tested over two manipulated Western blot\nimage datasets, one altered automatically and the other manually by exploiting\nadvanced AI-based image manipulation tools that are unknown at our training\nstage. We also explore the robustness of our method over an external dataset of\nother scientific images depicting different semantics, manipulated through\nunseen generation techniques.","PeriodicalId":501480,"journal":{"name":"arXiv - CS - Multimedia","volume":"6 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Localization of Synthetic Manipulations in Western Blot Images\",\"authors\":\"Anmol Manjunath, Viola Negroni, Sara Mandelli, Daniel Moreira, Paolo Bestagini\",\"doi\":\"arxiv-2408.13786\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Recent breakthroughs in deep learning and generative systems have\\nsignificantly fostered the creation of synthetic media, as well as the local\\nalteration of real content via the insertion of highly realistic synthetic\\nmanipulations. Local image manipulation, in particular, poses serious\\nchallenges to the integrity of digital content and societal trust. This problem\\nis not only confined to multimedia data, but also extends to biological images\\nincluded in scientific publications, like images depicting Western blots. In\\nthis work, we address the task of localizing synthetic manipulations in Western\\nblot images. To discriminate between pristine and synthetic pixels of an\\nanalyzed image, we propose a synthetic detector that operates on small patches\\nextracted from the image. We aggregate patch contributions to estimate a\\ntampering heatmap, highlighting synthetic pixels out of pristine ones. Our\\nmethodology proves effective when tested over two manipulated Western blot\\nimage datasets, one altered automatically and the other manually by exploiting\\nadvanced AI-based image manipulation tools that are unknown at our training\\nstage. We also explore the robustness of our method over an external dataset of\\nother scientific images depicting different semantics, manipulated through\\nunseen generation techniques.\",\"PeriodicalId\":501480,\"journal\":{\"name\":\"arXiv - CS - Multimedia\",\"volume\":\"6 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-08-25\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Multimedia\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2408.13786\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Multimedia","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.13786","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
摘要
最近在深度学习和生成系统方面取得的突破极大地促进了合成媒体的创建,以及通过插入高度逼真的合成处理对真实内容进行局部篡改。局部图像处理尤其对数字内容的完整性和社会信任构成了严重挑战。这个问题不仅局限于多媒体数据,还延伸到科学出版物中的生物图像,如描述 Western 印迹的图像。在这项工作中,我们解决了在 Western 印迹图像中定位合成操作的任务。为了区分分析图像中的原始像素和合成像素,我们提出了一种合成检测器,该检测器对从图像中提取的小补丁进行检测。我们汇集补丁贡献来估算篡改热图,从原始像素中突出合成像素。我们的方法在两个经过处理的 Western 印迹图像数据集上进行了测试,证明是有效的,其中一个数据集是自动修改的,另一个数据集是利用先进的人工智能图像处理工具手动修改的,而这些工具在我们的训练阶段是未知的。我们还探索了我们的方法在外部数据集上的鲁棒性,这些外部数据集包含了通过未知生成技术处理的描述不同语义的其他科学图像。
Localization of Synthetic Manipulations in Western Blot Images
Recent breakthroughs in deep learning and generative systems have
significantly fostered the creation of synthetic media, as well as the local
alteration of real content via the insertion of highly realistic synthetic
manipulations. Local image manipulation, in particular, poses serious
challenges to the integrity of digital content and societal trust. This problem
is not only confined to multimedia data, but also extends to biological images
included in scientific publications, like images depicting Western blots. In
this work, we address the task of localizing synthetic manipulations in Western
blot images. To discriminate between pristine and synthetic pixels of an
analyzed image, we propose a synthetic detector that operates on small patches
extracted from the image. We aggregate patch contributions to estimate a
tampering heatmap, highlighting synthetic pixels out of pristine ones. Our
methodology proves effective when tested over two manipulated Western blot
image datasets, one altered automatically and the other manually by exploiting
advanced AI-based image manipulation tools that are unknown at our training
stage. We also explore the robustness of our method over an external dataset of
other scientific images depicting different semantics, manipulated through
unseen generation techniques.