Color eigenflows: statistical modeling of joint color changes

We develop a linear model of commonly observed joint color changes in images due to variation in lighting and certain non-geometric camera parameters. This is done by observing how all of the colors are mapped between two images of the same scene under various "real-world" lighting changes. We represent each instance of such a joint color mapping as a 3-D vector field in RGB color space. We show that the variance in these maps is well represented by a low-dimensional linear subspace of these vector fields. We dub the principal components of this space the color eigenflows. When applied to a new image, the maps define an image subspace (different for each new image) of plausible variations of the image as seen under a wide variety of naturally observed lighting conditions. We examine the ability of the eigenflows and a base image to reconstruct a second image taken under different lighting conditions, showing our technique to be superior to other methods. Setting a threshold on this reconstruction error gives a simple system for scene recognition.