On the mathematical foundations of smoothness constraints for the determination of optical flow and for surface reconstruction

Gradient-based approaches to the computation of optical flow often use a minimization technique incorporating a smoothness constraint on the optical flow field. The author derives the most general form of such a smoothness constraint which is quadratic in first or second derivatives of the grey-level image intensity function, based on three simple assumptions about the smoothness constraint: (1) that it be expressed in a form which is independent of the choice of Cartesian coordinate system in the image; (2) that it be positive definite; and (3) that it not couple different components of the optical flow. It is shown that there are essentially only four such constraints; any smoothness constraint satisfying all three assumptions must be a linear combination of these four, possibly multipled by certain quantities of these four, possibly multipled by certain quantities invariant under a change in the Cartesian coordinate system. Beginning with the three assumptions mentioned above, the author mathematically demonstrates that all the best-known smoothness constraints appearing in the literature are special cases of this general form, and, in particular, that the 'weight matrix' introduced by H.-H. Nagel (1983) is essentially the only physically plausible such constraint.<>