Causal perception inspired representation learning for trustworthy image quality assessment

Abstract

Despite great success in modeling visual perception, deep neural network based image quality assessment (IQA) still remains untrustworthy in real-world applications due to its vulnerability to adversarial perturbations. In this paper, we propose to build a trustworthy IQA model via Causal Perception inspired Representation Learning (CPRL). More specifically, we assume that each image is composed of Causal Perception Representation (CPR) and non-causal perception representation (N-CPR). CPR serves as the causation of the subjective quality label, which is invariant to the imperceptible adversarial perturbations. Inversely, N-CPR presents spurious associations with the subjective quality label, which may significantly change with the adversarial perturbations. We propose causal intervention to boost CPR and eliminate N-CPR. Specifically, we first generate a series of N-CPR intervention images, and then minimize the causal invariance loss. Then we propose a SortMask module to reduce Lipschitz and improve robustness. SortMask block small changes around the mean to eliminate N-CPR and can be plug-and-play. Experiments on four benchmark databases show that the proposed CPRL method outperforms many state-of-the-art methods and provides explicit model interpretation. To support reproducible scientific research, we release the code at https://clearlovewl.github.io.