Efficient-Frequency: a hybrid visual forensic framework for facial forgery detection

Abstract

The recent years have witnessed the significant development of visual forgery techniques and their malicious applications such as spreading of fake news and rumours, defamation or blackmailing of politicians and celebrities, manipulation of election result in political warfare. The manipulated contents have reached to such sophisticated level that human cannot tell apart whether a given content is real or fake. To deal with this serious threat, a rich body of visual forensic techniques has been proposed for detecting forged video and images. However, existing techniques either rely solely on engineered features or require a complex deep learning model to extract the underlying patterns. In this paper, we propose a novel end-to-end visual forensic framework that can incorporate different modalities to efficiently classify real and forged contents. The model employs both the original content and its frequency domain analysis to fully exploit the richness of the image latent patterns. They are forwarded into two separated EfficientNet, a light yet efficient neural network architecture specialized for image classification, for pattern extraction. Then, we design a late-fusion mechanism to fuse the learnt features in original and frequency domain based on the importance of the underlying information. Our experimental results show that our proposed technique outperforms other state-of-the-art forensic approaches in many datasets and being robust to various visual forgery techniques.