LBPNet: Inserting Local Binary Patterns into Neural Networks to Enhance Manipulation Invariance of Fake Face Detection

Abstract

Fake face detection is an essential yet under-explored area, because faces generated by modern Generative Adversarial Networks (GANs) are virtually indiscernible by human observers. A recent work, GramNet[1], proposed using global texture information as heuristic because the features that previous approaches rely on are largely lost after image corruption and are not generalizable to different GANs. However, our theoretical reasoning and empirical studies both lead to the realization that their GLCM descriptor, as a global texture descriptor, is still not robust enough in all cases of image manipulation, because it doesn't take enough pixels into account, making it distortion-prone. Statistical analyses show that LBP is more generalizable to different GANs, and also reaches high consistency in outputs after image manipulation. Motivated by this finding, we implemented a Convolutional Neural Network with a ResNet backbone that uses LBP to enhance its global texture perception, effectively describing the texture at various semantic levels in an image with improved robustness. We conducted experiments with our model on images generated from StyleGAN and StyleGAN2, as well as images manipulated by different filters, and showed that our model reaches better and more consistent performance during image manipulation or in cross-domain settings, especially when images are subjected to Gaussian noise, in which we reached a performance increase from 82% to 90%. We open-sourced our code at https://github.com/Josh-Cena/lbpnet.