Scalable Mobile Image Recognition for Real-Time Video Annotation

Traditional AR frameworks for gaming and advertising focus on tracking 2D static targets. This limits the plausible use of this solutions to certain application cases like brochures or posters, but deprives their use for dynamically changing 2D targets, such as video walls or electronic billboards used in advertising.In this demo, we show how to use a rapid, fully mobile image recognition system to introduce AR in videos playing on TV sets or other dynamic screens, without the need to alter or modify the content for trackability. Our approach uses a scalable and fully mobile concept, which requires a database with a very small memory footprint on mobiles for a video or even a collection of videos.The feasibility of the approach is demonstrated on over 16 hours of video from a popular TV series, indexing into the video and giving accurate time codes and full 6DOF tracking for AR augmentations.