A Hybrid Deep Learning Framework for Deepfake Detection Using Temporal and Spatial Features

Abstract

The rise of deep-fake technology has sparked concerns as it blurs the distinction between fake media by harnessing Generative Adversarial Networks (GANs). This has raised issues surrounding privacy and security in the realm. This has led to a decrease in trust during online interactions; thus, emphasizing the importance of creating reliable methods for detection purposes. Our research introduces a model for detecting deepfakes by utilizing an Enhanced EfficientNet B0 structure in conjunction with Temporal Convolutional Neural Networks (TempCNNs). This approach aims to tackle the challenges presented by the evolving sophistication of deep-fake techniques. The system dissects video inputs into frames to extract features comprehensively by using Multi Test Convolutional Networks (MTCNN). This method ensures face detection and alignment by focusing on facial regions. To enhance the model’s adaptability, to different scenarios and datasets we implement data augmentation techniques such as CutMix, MixUp and Random Erasing. These strategies help the model maintain its strength, against distortions found in deepfake content. The backbone of EfficientNet B0 utilizes Mobile Inverted Bottleneck Convolutions (MBConv) and Squeeze and Excitation (SE) blocks to enhance feature extraction by adjusting channels to highlight details effectively. A Feature Pyramid Network (FPN) facilitates the fusion of scale features capturing intricate details as well, as broader context. When tested on the FFIW 10 K dataset, which comprises 10,000 videos evenly split between manipulated content, the model attained a training accuracy of 91.5 % and a testing accuracy of 92.45 %, after 40 epochs. The findings showcase the model’s proficiency, in identifying videos with precision and tackling the issue of class imbalances found in datasets – a valuable contribution, to advancing dependable deepfake detection solutions. Furthermore, the model achieves an impressive balance between accuracy and computational efficiency, attaining 92.45% testing accuracy with a lightweight computational cost of 0.45 GFLOPs, making it a highly practical choice for real-world deployment.