Visual Attention Metadata from Pictures Browsing

Abstract

Image browsing has become an indispensable feature for today's mobile devices. To overcome their limited display size, information systems may benefit from the abundant clickstream provided by mobile users. This implicit feedback turns out to be very informative providing hints on both the visual image content and the relevance of the query results when searching for images. Building on previous works on user attention models, we propose a practical, yet generic platform for web usage tracking. Implicit feedback from the clickstream makes it possible to generate visual attention metadata. These metadata are user interest maps (UIMs) in which regions of interest (ROIs) become highlighted. These UIMs are used to establish and reinforce keyword-to-image relations. A tentative application for semantic annotation is also presented.