A perspective geometry approach to user-perspective rendering in hand-held video see-through augmented reality

Video see-through Augmented Reality (V-AR) displays a video feed overlaid with information, co-registered with the displayed objects. In this paper we consider the type of V-AR that is based on a hand-held device with a fixed camera. In most of the VA-R applications the view displayed on the screen is completely determined by the orientation of the camera, i.e., the device-perspective rendering; the screen displays what the camera sees. The alternative method is to use the relative pose of the user's view and the camera, i.e., the user-perspective rendering. In this paper we present an approach to the user perspective V-AR using 3D projective geometry. The view is adjusted to the user's perspective and rendered on the screen, making it an augmented window. We created and tested a running prototype based on our method.