PictureSort: gamification of image ranking

Abstract

Human computation is a very powerful tool for solving tasks that cannot be solved by computers efficiently. One such problem is ranking images upon their relevance for a semantic query or upon how well they depict a semantic concept. In this paper we investigate a method to leverage human computation in a divide-and-conquer approach to create precise ranking models. We discuss the basic technique, our prototype client, its adoption to a gamification approach, and present the results of a study with the prototype. Results from the study indicate that with our method the ranking aggregated from the user input converges fast to an optimal ranking.