Generalization enhancement strategy based on ensemble learning for open domain image manipulation detection

Abstract

Image manipulation detection methods play a pivotal role in safeguarding digital image authenticity and integrity by identifying and locating manipulations. Existing image manipulation detection methods suffer from limited generalization, as it is difficult for existing training datasets to cover different manipulation modalities in the open domain. In this paper, we propose a Generalization Enhancement Strategy (GES) based on data augmentation and ensemble learning. Specifically, the GES consists of two modules, namely an Additive Image Manipulation Data Augmentation(AIM-DA) module and a Mask Confidence Estimate based Ensemble Learning (MCE-EL) module. In order to take full advantage of the limited number of real and manipulated images, the AIM-DA module enriches the diversity of the data by generating manipulated traces accumulatively with different kinds of manipulation methods. The MCE-EL module is designed to improve the accuracy of detection in the open domain, which is based on computing and integrating the evaluation of the confidence level of the output masks from different image manipulation detection models. Our proposed GES can be adapted to existing popular image manipulation detection methods. Extensive subjective and objective experimental results show that the detection F1 score can be improved by up to 34.9%, and the localization F1 score can be improved by up to 11.7%, which validates the effectiveness of our method.