Document image forgery detection and localization in desensitization scenarios

Abstract

Document images are widely used in e-commerce, and some privacy information contained in them may be desensitized before circulation. Since innocuous desensitization is rather different from malicious tampering in both motivation and appearance, it results in a new forensic scenario, in which reliable forgery detection and localization is needed when desensitization artifacts present. In this paper, we address the issue for the first time by proposing DCLNet (Desensitization involved Contrastive Learning based forensic Network), to improve the performance of pixel-level tampering localization and image-level forgery detection. DCLNet is built upon a ConvNeXt-based encoder–decoder network with a global context attention module, enabling it to learn effective features from multi-scales. To tackle the difficulty of learning weak tampering traces without interference from strong desensitization artifacts, we design a contrastive learning module to effectively differentiate between the two kinds of manipulations. Additionally, we construct a document image dataset that considers various document types and contains both tampering and desensitization manipulations, providing sufficient data for training and evaluation. Extensive experimental results demonstrate that DCLNet outperforms existing methods for the new task, and exhibits good robustness to post-processing and better adaptability to other sources of document images.