Computational modeling of artistic intention: Quantify lighting surprise for painting analysis

The use of strong lighting contrast to accentuate objects and figures in a painting-called Chiaroscuro-is popular among Renaissance painters such as Caravaggio, La Tour and Rembrandt. In this paper, we propose a new metric called LuCo to quantify the extent to which Chiaroscuro is employed by an artist in a painting. This measurement could be used to assess the capability of any system to fulfill the original artistic intention and consequently ensure minimal disruptions of Quality of Experience. We first argue that Chiaroscuro is a device for artists to draw attention to specific spatial regions; thus it can be understood as a restricted notion of visual saliency computed using only luminance features. Operationally, using a set of local luminance patches we first compute a Bayesian surprise value, where the prior and posterior probabilities are computed assuming a Gaussian Markov Random Field (GMRF) model. Inverse covariance matrices of the GMRF model are estimated via sparse graph learning for robustness. We construct a histogram using the computed surprise values from different local patches in a painting. Finally, we compute a skewness parameter for the constructed histogram as our LuCo score: large skewness means luminance surprises are either very small or very large, meaning that the artist accentuated lighting contrast in the painting. Experimental results show that paintings by Chiaroscuro artists have higher LuCo scores than 19th century French Impressionists, and Rembrandt's self-portraits have increasingly higher LuCo scores as he aged except for his late period-both trends are in agreement with art historians' interpretations.