Managing latency in complex augmented reality systems

Abstract

Managing Latency in Complex Augmented Reality Marco C. Jacobs* Mark A. Livingston Andrei State University of North Carolina at Chapel Hill *Delft University of Technology, the Netherlands {jacobsllivingst [state} @cs.unc.edu Systems Registration (or alignment) of the synthetic imagery with the real world is crucial in augmented reality (AR) systems. The data from user-input devices, tracking devices, and imaging devices need to be registered spatially and temporally with the user’s view of the surroundings. Each device haa an associated delay between its observations of the world and the moment when the AR display presented to the user appeto be afkted by a change in the data. We call the differences in delay the relative Iatencies. Relative latency is a source of misregistration and should be reduced. We give general methods for handling multiple data streams with different latency values associated with them in a working AR system. We measure the latency differences (part of the system dependent set of calibrations), time-stamp on-host, adjust the moment of sampling, and interpolate or extrapolate data streams. By using these schemes, a more accurate and consistent view is computed and presented to the user. CR