Deep perceptual similarity and Quality Assessment

Abstract

Measuring the perceptual similarity between two images is a long-standing problem. This assessment should mimic human judgments. Considering the complexity of the human visual system, it is challenging to model human perception. On the other hand, the recent low-level vision task approaches, mostly based on supervised deep learning, require an appropriate loss for the backward pass. The per-pixel loss, such as MSE and MAE, between the output of the network and the ground-truth images were among the first choices. More complicated and common similarity measures in which the error is computed in a hand-designed feature space are also employed. Furthermore, Deep Perceptual Similarity (DPS) metrics, where the similarity is measured in the deep feature space, also have promising results. This feature can be selected from a pre-trained or optimized model for the task at hand. Recently many studies have been conducted to thoroughly investigate DPS. In this research, we provide an in-depth analysis of the pros and cons of DPS in assessing the full reference quality assessment. In addition, to compare different similarity measures, we propose a metric which aggregates various desired factors. Based on our experiment, it can be concluded that perceptual similarity is not directly related to classification accuracy. It is discovered that the outliers mostly contain high-frequency elements. The code and complete outcomes described in results, can be found on: https://github.com/Alireza-Khatami/PerceptualQuality