Benchmarking The Imbalanced Behavior of Deep Learning Based Optical Flow Estimators

Abstract

Optical Flow (OF) estimation is an important task which could be effectively used for a variety of Computer Vision (CV) applications. While a range of techniques have been already proposed, however accurately estimating the OF is still a very challenging task. The most recent approaches for OF estimation exploit advanced Deep Learning techniques which have resulted in a shift in the paradigm. These techniques have shown substantial improvements particularly in the case of large displacements, brightness change, and non-rigid motion. However, these approaches are data-driven and hence they can be biased towards the specific training data, which could in turn lead to considerable inaccuracy of the estimated OF. In this paper, we address this problem and provide a novel benchmark that can be used to identify and to measure the bias magnitude of the OF estimation. We have performed several experiments based on public datasets (Monkaa and Sintel) as well as on data designed on purpose 1. The results have shown that OF estimation based on some of the state-of-the-art deep learning techniques strongly depend on factors such as motion orientation within the data. Indeed, we have observed substantial degree of bias toward certain directions of motion. The framework can help researchers and practitioners in order to develop more effective data augmentation techniques and training schedules for deep learning based optical flow estimators.