Image collection summarization for search result overviewing on mobile devices

Abstract

Due to small displays of mobile devices, overviewing an image search result that contains many and various images is difficult. To provide an overview of thousands of images, recent studies have tried to develop a framework for image collection summarization that extracts a smaller set of representative images from the original set. Most existing methods take (a) relevance and (b) coverage of each image into account. However, for the use on mobile devices, several important issues remain: generated summaries must be compact enough so as to suit the small mobile displays but the legibility of the summaries should be sufficient -- but how? Our focus in this paper is to extend the framework of image collection summarization to fit the context of overviewing image search results on mobile devices. The key advances of this paper are to introduce two primary factors of (c) compactness and (d) legibility when generating summaries. Our solution is a two-stage optimization method. Given a keyword query and display size, its first stage ranks the images by taking (a) relevance and (b) coverage into account. The second optimization stage takes into account (c) compactness and (d) legibility and determines the number and sizes of images included in the final summary so as to satisfy the display size constraint. Experiments conducted on over 240,000 images demonstrate the effectiveness of our method.