Flexible openCL accelerated disparity estimation for video communication applications

Abstract

Due to widespread broadband connections in normal households, the use of video chats via Internet is no longer limited to business meetings. However, the camera configuration usually makes it impossible to achieve direct eye contact between the conversational partners. This effect can be compensated using virtual view synthesis methods based on disparity maps. The virtual camera is positioned “behind” the communications windows and thus re-establishes the eye-contact. Obtaining a good disparity map is still a challenging problem and, with respect to video communication, must perform at interactive frame rates. In this paper we present optimized algorithms for disparity estimation that run in near real time. Recent developments in the consumer-hardware industry allow the implementation of complex algorithms for eye gaze correction, which can be used with relatively inexpensive out-of-the-box components. We employ the newly introduced OpenCL Framework and present an implementation of several optimized algorithms on a Graphics Processing Unit (GPU). Our implementation supports different methods for cost-estimation and aggregation, which we can combine flexibly. We present a method to efficiently implement a dynamic programming approach on the GPU. Our contribution makes it possible to interactively change parameters of the algorithms and get instant visual feedback which is crucial in algorithm development and parameter tuning. We also show first results of virtual views that re-establish the eye contact.