Video Annotation by Cascading Microtasks: a Crowdsourcing Approach

Abstract

This paper presents a general approach to perform crowdsourcing video annotation without requiring trained workers nor experts. It consists of dividing complex annotation tasks into simple and small microtasks and cascading them to generate a final result. Moreover, this approach allows using simple annotation tools rather than complex and expensive annotation systems. Also, it tends to avoid activities that may be tedious and time-consuming for workers. The cascade microtasks strategy is included in a workflow of three steps: Preparation, Annotation, and Presentation. A crowdsourcing video annotation process in which four different microtasks were cascaded was developed to evaluate the proposed approach. In the process, extra content such as images, text, hyperlinks and other elements are applied in the video enrichment. To support the experiment was developed a toolkit that includes Web-based annotation tools and aggregation methods, besides a presentation system for the annotated videos. This toolkit is open source and can be downloaded and used to replicate this experiment, as so to construct different crowdsourcing video annotation systems.