High Resolution Video Playback in Immersive Virtual Environments

High resolution 2D video content in High Definition or higher resolutions has become widespread and video playback of such media in immersive virtual environments (VE) will be a valuable element adding more realism to VE applications. This kind of video play-back, however, has to overcome several problems. First, the data volume of video clips can reach up to hundreds of gigabytes or more depending on the length of the clips, and the data has to be streamed into virtual reality (VR) systems in real-time. Second, the interactivity of the playback screen in 3D virtual environments requires efficient rendering of each video frame. Interactivity means that the plane of the video playback screen needs to rotate, translate, and zoom in and out in 3D space as the viewer roams around in the VE. This also means that the video is not necessarily parallel to the display screen but will need to be displayed as a general quadrangle. In this work, we propose an efficient algorithm that utilizes mipmapped data, that is, multiple levels of resolutions, to provide an efficient way to interactively play back high resolution video content in VEs. In addition, we discuss several optimizations to sustain a constant frame rate, such as an optimized memory management mechanism, dynamic resolution adjustment, and predictive prefetching of data. Finally, we evaluate two video playback applications running on a virtual reality CAVE system: 1) high definition video at 3840 × 2160 pixels and 2) 32 independent 256 × 192 pixels video clips.