A contact sheet approach to searching untagged images on smartphones

Abstract

We present a cloud-based approach to opportunistic, near real-time search of untagged images on smartphones that is sensitive to bandwidth and energy constraints. Our approach is inspired by the long-established practice of photographers using contact sheets to rapidly visualize a new collection of photographs, and then selecting a subset on which to focus attention. On behalf of each smartphone, the cloud maintains a virtual contact sheet of images that have been captured but not yet uploaded. The virtual contact sheet consists of a set of low-fidelity images as well as full or partial meta-data associated with each image. If search processing on the cloud indicates that a particular low-fidelity object is relevant, then its full-fidelity image can be obtained just-in-time from the corresponding smartphone for further search processing or presentation to the user. Our approach is not limited to images, but they provide a convenient query space to test search optimizations.