UAM-Net: Robust Deepfake Detection Through Hybrid Attention Into Scalable Convolutional Network

Abstract

The recent advancements in computer vision have transformed data manipulation detection into a significantly challenging task. Deepfakes are advanced manipulation methods for generating highly convincing synthetic media wherein one digitally forges an individual's visuals. Therefore, safeguarding the authenticity and integrity of digital content against such forgeries and developing robust detection methods is essential. Identifying manipulated regions and channels within deepfake images is especially critical in countering these forgeries. Introducing attention features into the classification pipeline enhances the detection of subtle manipulations. Such subtle manipulations are typical of deepfake content. This study presents a novel feature selection approach, a Unified Attention Mechanism into convolutional networks—the ‘UAM-Net’. The UAM-Net framework concurrently integrates spatial and channel attention features into the data-driven scalable convolutional features. The UAM-Net was trained and evaluated on the DeepFake Detection Challenge Preview (DFDC-P) data set. It was then cross-validated on combined FaceForensics++ and CelebA-DF data sets. UAM-Net has achieved outstanding results, including an accuracy of 98.07%, precision of 97.91%, recall of 98.23%, F1 score of 98.07% and an AUC-ROC score of 99.82%. The UAM-Net model maintained strong performance on the combined data set and achieved 89.7% accuracy, 85.4% precision, 95.8% recall, 90.3% F1 score, and AUC ROC score of 96.8%. The UAM-Net also demonstrated robustness to degraded input quality with 96.98% accuracy and 97% AUC-ROC on the spatially compressed DFDC-P data set. Thus, the model would adapt to real-world conditions, as evidenced by a 97% AUC-ROC on randomly blurred data sets.