Demo: Irides: Attaining Quality, Responsiveness and Mobility for Virtual Reality Head-mounted Displays

Abstract

Immersive virtual reality has long been an aspiration of many. Recent advances in wearable high resolution headmounted displays (HMDs) — as embodied by Facebook’s Oculus Rift, Sony’s Morpheus, Samsung’s Gear VR, Google’s Cardboard and others — have stoked the imaginations of a new generation of VR users. For a truly immersive VR experience, three properties are essential: quality, responsiveness and mobility. By quality, we mean that realistic and life-like visual portrayals in a virtual environment heighten our sense of immersion. By responsiveness, we mean that any motion, especially of the head, must be reflected as quickly as possible in visual feedback because ocular proprioception is tightly synchronized. By mobility, we mean that we ought to be able to move untethered in physical space, free to explore our virtual world. However, HMDs face a fundamental challenge in seeking to simultaneously provide quality, responsiveness and mobility; choosing any two rules out the third. In this work, we present Irides, a stereo HMD system that simultaneously attains quality, responsiveness and mobility. Irides achieves this by offloading rendering work to a (possibly WAN) high-end GPU. The HMD client is mobile and only requires a wireless network connection, yet receives high quality and responsive visuals. Irides leverages prior work to overcome WAN latencies [2]. It employs speculative execution on the server, and compensates for misspeculation on the client. However, a straightforward application of speculative execution for stereo HMDs is prohibitive because of the 2× increase in server, client and network bandwidth costs. Therefore, we introduce two techniques to reduce these costs as shown in Figure 1. The first technique, interleaved stereo, generates a stereo video stream with only a mono video stream’s worth of work. Interleaved stereo uses image-based rendering (IBR), a way to warp a 2D image with depth information to a novel view. Synthesizing a stereo video from a mono source can introduce severe artifacts. We introduce novel “hole-filling” mechanisms that eliminate artifacts.