Optimization of variational methods via motion-based weight selection and keypoint matching

Abstract

Variational method is a well-established technique that solves for a dense field, which is widely adopted in the estimation of optical flow field and remains the most accurate technique to date. However, one of the problems in variational method lies in that it is optimized in an iterative manner towards a single objective, but local details may be compromised owing to the “big picture”. In this paper, we address this problem in an optical flow framework by introducing two sparse local rectifications to the global numerical scheme, i.e., motion-based weight selection and keypoint matching. The selection of the weighting parameter in a self-adaptive and content-aware manner provides a more accurate estimation of the optical flow field near motion boundaries, and motion details and small structures are preserved in the optical flow field by keypoint matching in the initialization of the optical flow field. Experimental results using the Middlebury dataset show that the proposed algorithm achieves higher accuracy compared to the original TV-ℓ1 optical flow algorithm and many state-of-the-art methods.