Challenges in multiview video — The 3 D'S

For the purpose of this paper we group, under the generic term multiview video, different systems for which multiple standard video cameras and, possibly, additional depth-capturing cameras, are used. Video is then presented to the user using special glasses or displays. Research work in this area has focused on topics ranging from designing compression techniques to developing new 3D displays. In this paper we primarily consider the challenges involved in developing efficient compression tools. Our primary observation is that the “right” coding tools could depend heavily on choices made for content capture, display and communication. This is of course true for conventional video coding as well. But we will argue that it is even more important to address these issues for multiview video because there are much greater differences between different application scenarios (as compared to conventional video). The risk is that coding tools that are too narrowly focused on a specific application scenario may not be at all suitable for others. We focus specifically on three factors for which there exists significant uncertainty, namely, displays, depth estimation and content delivery. Our goal is not to discuss in detail current and future approaches (e.g., emerging alternative display technologies), but rather to show how these various approaches may have an impact on compression system design.