Optical occlusion and shadows in a 'see-through' augmented reality display

As distinct from virtual reality, which seeks to immerse the user in a fully synthetic world, computer-augmented reality systems supplement sensory input with computer-generated information. The principle has, for a number of years, been employed in the head-up display systems used by military pilots and usually comprises an optical display arrangement based on part-silvered mirrors that reflect computer graphics into the eye in such a way that they appear superimposed on the real-world view. Compositing real and virtual worlds offers many new and exciting possibilities but also presents some significant challenges, particularly with respect to applications for which the real and virtual elements need to be integrated convincingly. Unfortunately, the inherent difficulties are compounded further in situations where a direct, unpixellated view of the real world is desired, since current optical systems do not allow real-virtual occlusion, nor a number of other essential visual interactions. The paper presents a generic model of augmented reality as a context for discussion, and then describes a simple but effective technique for providing a significant degree of control over the visual compositing of real and virtual worlds.