CNN Feature Similarity: Paintings Are More Self-Similar at All Levels

Abstract

Self-similarity is often an indicator of highly aesthetic paintings; a key aspect is what feature to use for evaluating self-similarity. Previous works use low-level features such as the gradient orientation to show that artworks and natural scenes share a similar degree of self-similarity. In this study, we take advantage of the AlexNet model to evaluate the changes of self-similarity at different convolutional layers in a CNN model. Compared to previous measures, our approach takes into account low, mid, and high level features. Different behaviors with regards to self-similarity at different layers is observed in paintings. The results confirm previous findings that artworks and natural scenes share similar degrees of self-similarity. For paintings and photographs with similar subject matters, while different degrees of self-similarity are observed at the first layer, other layers show closer values. Finally, the proposed measure of self-similarity is able to better differentiate between images which belong to a similar category but different datasets of images.