Real-time virtual object insertion

Abstract

We present a system to insert virtual objects into real image sequences in real time. The system consists of offthe- shelf hardware (a camera connected to a Pentium PC) and software to (a) automatically select and track region features despite changes in illumination, (b) estimate threedimensional position and orientation of surface patches relative to an inertial reference frame despite individual pointfeatures appearing and disappearing, (c) insert a texturemapped virtual object into the scene so as to make it appear to be part of the scene and moving with it. This is all done in real time. The multi-thread C++ code, which is readily interfaced with a frame grabber as well as Matlab for development, will be made available to the public at the demonstration.