Mobile Interfaces for Crowdsourced Multimedia Microtasks

Abstract

Crowd sourced mobile microtasking represents a significant opportunity in emerging economies such as India, that are characterized by the high levels of mobile phone penetration and large numbers of educated people that are unemployed or underemployed. Indeed, mobile phones have been used successfully in many parts of the world for microtasking, primarily for crowd sourced data collection, and text or image based tasks. More complex tasks such as annotation of multimedia such as audio or video have traditionally been confined to desktop interfaces. With the rapid evolution in the multimedia capabilities of mobile phones in these geographies, we believe that the nature of microtasks carried out on these devices, as well as the design of interfaces for such microtasks, warrants investigation. In this paper we explore the design of mobile phone interfaces for a set of multimedia-based microtasks on feature phones, which represent the vast majority of multimedia-capable mobile phones in these geographies. As part of an initial study using paper prototypes, we evaluate three types of multimedia content: images, audio and video, and three interfaces for data input: Direct Entry, Scroll Key Input and Key Mapping. We observe that while there are clear interface preferences for image and audio tasks, the user preference for video tasks varies based on the 'task complexity' - the 'density' of data the annotator has to deal with. In a second study, we prototype two different interfaces for video-based annotation tasks - a single screen input method, and a two screen phased interface. We evaluate the two interface designs and the three data input methods studied earlier by means of a user study with 36 participants. Our findings show that where less dense data was concerned; participants prefer Key Mapping as the input technique. For dense data, while participants prefer Key Mapping, our data shows that the accuracy of data input with Key Mapping is significantly lower than that with Scroll Key Input. The study also provides insight into the game plan each user develops and employs to input data. We believe these findingswill enable other researchers to build effective user interfaces for mobile microtasks, and be of value to UI developers, HCI researchers and microtask designers.