Transdefend-VC: A Transformer-Driven Deepfake Defense Mechanism for Secure Video Conferencing

Abstract

Deepfakes in video conferences pose an emerging threat, which demands real-time, multimodal, and privacy-preserving defense solutions. In this paper, we present an application of our argument from before in a different guise, stating that a centralized defense mechanism is insufficient because it lacks capabilities in spatial-temporal reasoning, audiovisual fusion, and handling privacy issues. To fill these limitations, this paper proposes a trans-DefendVC mechanism. The strategy begins with a series of multimodal preprocessing steps such that facial regions undergo alignment, cropping, patch embedding, and tokenization, with log-mel spectrogram patches being used for synchronized feature extraction, and two encoders are used on both modalities, where region-adaptive vision transformers address facial irregularities, with frequency-aware audio spectrogram transformers addressing speech manipulation. To model learning relationships between audiovisual signals, a cross-modality transformer attention module learns lip and speech synchronization at a high level of detail. A causal temporal transformer module addresses short- and long-term temporal dependencies necessary for real-time analysis. To support safe and efficient implementation, Federated transformer learning with differential privacy is used to support decentralized learning without sharing audiovisual signals. An integrated classifier informs authenticity decisions based on a unified representation, effectively identifying deepfake artifacts in space, spectral, and temporal domains. Experiments conducted on FakeAVCeleb and Celeb-DF v2 datasets prove efficacy with accuracy of 94.96/99.84% and AUC of 98.79/100%, thus establishing this solution’s effectiveness in providing a safe and efficient experience in a real-time video conferencing setup. The proposed solution provides a potent defense solution for defeating deepfakes in real-world video conferencing systems.