Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation最新文献

Multimedia Forensics versus disinformation in images and videos: lesson learnt and new challenges 多媒体取证与图像和视频中的虚假信息:经验教训和新挑战

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-06-12 DOI: 10.1145/3592572.3596489

R. Caldelli

引用次数: 0

Synthetic Speech Detection through Audio Folding 基于音频折叠的合成语音检测

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-06-12 DOI: 10.1145/3592572.3592844

Davide Salvi, Paolo Bestagini, S. Tubaro

引用次数: 3

SpoTNet: A spoofing-aware Transformer Network for Effective Synthetic Speech Detection SpoTNet:一种用于有效合成语音检测的欺骗感知变压器网络

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-06-12 DOI: 10.1145/3592572.3592841

Awais Khan, K. Malik

{"title":"SpoTNet: A spoofing-aware Transformer Network for Effective Synthetic Speech Detection","authors":"Awais Khan, K. Malik","doi":"10.1145/3592572.3592841","DOIUrl":"https://doi.org/10.1145/3592572.3592841","url":null,"abstract":"The prevalence of voice spoofing attacks in today’s digital world has become a critical security concern. Attackers employ various techniques, such as voice conversion (VC) and text-to-speech (TTS), to generate synthetic speech that imitates the victim’s voice and gain access to sensitive information. The recent advances in synthetic speech generation pose a significant threat to modern security systems, while traditional voice authentication methods are incapable of detecting them effectively. To address this issue, a novel solution for logical access (LA)-based synthetic speech detection is proposed in this paper. SpoTNet is an attention-based spoofing transformer network that includes crafted front-end spoofing features and deep attentive features retrieved using the developed logical spoofing transformer encoder (LSTE). The derived attentive features were then processed by the proposed multi-layer spoofing classifier to classify speech samples as bona fide or synthetic. In synthetic speeches produced by the TTS algorithm, the spectral characteristics of the synthetic speech are altered to match the target speaker’s formant frequencies, while in VC attacks, the temporal alignment of the speech segments is manipulated to preserve the target speaker’s prosodic features. By highlighting these observations, this paper targets the prosodic and phonetic-based crafted features, i.e., the Mel-spectrogram, spectral contrast, and spectral envelope, presenting an effective preprocessing pipeline proven to be effective in synthetic speech detection. The proposed solution achieved state-of-the-art performance against eight recent feature fusion methods with lower EER of 0.95% on the ASVspoof-LA dataset, demonstrating its potential to advance the field of speaker identification and improve speaker recognition systems.","PeriodicalId":239252,"journal":{"name":"Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127879469","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

In the Spotlight: The Russian Government's Use of Official Twitter Accounts to Influence Discussions About its War in Ukraine 焦点:俄罗斯政府利用官方推特账户影响有关其乌克兰战争的讨论

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-06-12 DOI: 10.1145/3592572.3592843

Benjamin Shultz

{"title":"In the Spotlight: The Russian Government's Use of Official Twitter Accounts to Influence Discussions About its War in Ukraine","authors":"Benjamin Shultz","doi":"10.1145/3592572.3592843","DOIUrl":"https://doi.org/10.1145/3592572.3592843","url":null,"abstract":"Russia's war in Ukraine has marked an inflection point for the future of the global order and democracy itself. Widely condemned for waging a war of aggression, the Russian government has used its official social media channels to spread disinformation as justification for the war. This study examines how the Russian government has used its official Twitter accounts to shape English-language conversations about the war in Ukraine. 2,685 English-language tweets posted by 70 Russian government accounts between 1 September 2022 and 31 January 2023 were analyzed using BERTopic. Initial topic analysis shows the Russian government portrayed itself as a noble world leader interested in peace and cooperation, while deflecting blame onto the “Kiev Regime” for starting the war. A semantic similarity analysis was then conducted to compare the narratives originating from Russian government Twitter accounts to 149,732 English-language tweets about the war in Ukraine to estimate these narratives’ spread. Results show a segment of general discussion tweets to exhibit strongly similar language to Russian government tweets, but also highlight differences between the frequency and saliency of Russian government narratives. This work contributes one of the first analyses of disinformation originating from official Russian government social media channels about the war in Ukraine.","PeriodicalId":239252,"journal":{"name":"Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation","volume":"149 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134201641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Autoencoder-based Data Augmentation for Deepfake Detection 基于自动编码器的深度伪造检测数据增强

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-06-12 DOI: 10.1145/3592572.3592840

Dan-Cristian Stanciu, B. Ionescu

{"title":"Autoencoder-based Data Augmentation for Deepfake Detection","authors":"Dan-Cristian Stanciu, B. Ionescu","doi":"10.1145/3592572.3592840","DOIUrl":"https://doi.org/10.1145/3592572.3592840","url":null,"abstract":"Image generation has seen huge leaps in the last few years. Less than 10 years ago we could not generate accurate images using deep learning at all, and now it is almost impossible for the average person to distinguish a real image from a generated one. In spite of the fact that image generation has some amazing use cases, it can also be used with ill intent. As an example, deepfakes have become more and more indistinguishable from real pictures and that poses a real threat to society. It is important for us to be vigilant and active against deepfakes, to ensure that the false information spread is kept under control. In this context, the need for good deepfake detectors feels more and more urgent. There is a constant battle between deepfake generators and deepfake detection algorithms, each one evolving at a rapid pace. But, there is a big problem with deepfake detectors: they can only be trained on so many data points and images generated by specific architectures. Therefore, while we can detect deepfakes on certain datasets with near 100% accuracy, it is sometimes very hard to generalize and catch all real-world instances. Our proposed solution is a way to augment deepfake detection datasets using deep learning architectures, such as Autoencoders or U-Net. We show that augmenting deepfake detection datasets using deep learning improves generalization to other datasets. We test our algorithm using multiple architectures, with experimental validation being carried out on state-of-the-art datasets like CelebDF and DFDC Preview. The framework we propose can give flexibility to any model, helping to generalize to unseen datasets and manipulations.","PeriodicalId":239252,"journal":{"name":"Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129266914","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Controllable image generation and manipulation 可控图像生成和处理

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-06-12 DOI: 10.1145/3592572.3596476

I. Patras

{"title":"Controllable image generation and manipulation","authors":"I. Patras","doi":"10.1145/3592572.3596476","DOIUrl":"https://doi.org/10.1145/3592572.3596476","url":null,"abstract":"Recent years have witnessed an unprecedented interest in developing Deep Learning methodologies for the generation of images and image sequences that are hardly distinguishable to the human eye from real ones. A major issue in this field is how the generation can be easily controlled. In this talk we will focus on some of our recent works in generative models that are primarily aimed at controllable generation. We will first present unsupervised methods for learning non-linear paths in the latent spaces of Generative Adversarial Networks such that following different paths lead to different types of changes (e.g., removing the background, changing head poses, or facial expressions) in the resulting images [4]. Subsequently, we will present a method that allows local editing by finding a Parts and Appearances decomposition in the GAN latent space [2]. Then, we will present recent works on reenactment [1], where the goal is to transfer the facial activity (pose, expressions, speech) of a certain person to another one, and recent works in which supervision for generation comes from language models [3]. Finally, we will touch on the technical challenges ahead, as well on the challenges that this creates in spreading misinformation.","PeriodicalId":239252,"journal":{"name":"Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115848020","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Examining European Press Coverage of the Covid-19 No-Vax Movement: An NLP Framework 检视欧洲媒体对Covid-19无疫苗运动的报道:一个NLP框架

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-04-29 DOI: 10.1145/3592572.3592845

David Alonso del Barrio, D. Gática-Pérez

引用次数: 1

Improving Synthetically Generated Image Detection in Cross-Concept Settings 改进跨概念设置下的综合生成图像检测

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-04-24 DOI: 10.1145/3592572.3592846

P. Dogoulis, Giorgos Kordopatis-Zilos, I. Kompatsiaris, S. Papadopoulos

{"title":"Improving Synthetically Generated Image Detection in Cross-Concept Settings","authors":"P. Dogoulis, Giorgos Kordopatis-Zilos, I. Kompatsiaris, S. Papadopoulos","doi":"10.1145/3592572.3592846","DOIUrl":"https://doi.org/10.1145/3592572.3592846","url":null,"abstract":"New advancements for the detection of synthetic images are critical for fighting disinformation, as the capabilities of generative AI models continuously evolve and can lead to hyper-realistic synthetic imagery at unprecedented scale and speed. In this paper, we focus on the challenge of generalizing across different concept classes, e.g., when training a detector on human faces and testing on synthetic animal images – highlighting the ineffectiveness of existing approaches that randomly sample generated images to train their models. By contrast, we propose an approach based on the premise that the robustness of the detector can be enhanced by training it on realistic synthetic images that are selected based on their quality scores according to a probabilistic quality estimation model. We demonstrate the effectiveness of the proposed approach by conducting experiments with generated images from two seminal architectures, StyleGAN2 and Latent Diffusion, and using three different concepts for each, so as to measure the cross-concept generalization ability. Our results show that our quality-based sampling method leads to higher detection performance for nearly all concepts, improving the overall effectiveness of the synthetic image detectors.","PeriodicalId":239252,"journal":{"name":"Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125414458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

Synthetic Misinformers: Generating and Combating Multimodal Misinformation 合成错误信息:产生和打击多模态错误信息

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 2023-03-02 DOI: 10.1145/3592572.3592842

Stefanos Papadopoulos, C. Koutlis, S. Papadopoulos, P. Petrantonakis

{"title":"Synthetic Misinformers: Generating and Combating Multimodal Misinformation","authors":"Stefanos Papadopoulos, C. Koutlis, S. Papadopoulos, P. Petrantonakis","doi":"10.1145/3592572.3592842","DOIUrl":"https://doi.org/10.1145/3592572.3592842","url":null,"abstract":"With the expansion of social media and the increasing dissemination of multimedia content, the spread of misinformation has become a major concern. This necessitates effective strategies for multimodal misinformation detection (MMD) that detect whether the combination of an image and its accompanying text could mislead or misinform. Due to the data-intensive nature of deep neural networks and the labor-intensive process of manual annotation, researchers have been exploring various methods for automatically generating synthetic multimodal misinformation - which we refer to as Synthetic Misinformers - in order to train MMD models. However, limited evaluation on real-world misinformation and a lack of comparisons with other Synthetic Misinformers makes difficult to assess progress in the field. To address this, we perform a comparative study on existing and new Synthetic Misinformers that involves (1) out-of-context (OOC) image-caption pairs, (2) cross-modal named entity inconsistency (NEI) as well as (3) hybrid approaches and we evaluate them against real-world misinformation; using the COSMOS benchmark. The comparative study showed that our proposed CLIP-based Named Entity Swapping can lead to MMD models that surpass other OOC and NEI Misinformers in terms of multimodal accuracy and that hybrid approaches can lead to even higher detection accuracy. Nevertheless, after alleviating information leakage from the COSMOS evaluation protocol, low Sensitivity scores indicate that the task is significantly more challenging than previous studies suggested. Finally, our findings showed that NEI-based Synthetic Misinformers tend to suffer from a unimodal bias, where text-only models can outperform multimodal ones.","PeriodicalId":239252,"journal":{"name":"Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127263223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation 第二届ACM多媒体人工智能反虚假信息国际研讨会论文集

Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation Pub Date : 1900-01-01 DOI: 10.1145/3592572

引用次数: 0