Crowd R-CNN: An Object Detection Model Utilizing Crowdsourced Labels

Abstract

Accuracy of object detection has increased significantly in recent years because of the rapid development of deep learning techniques. Nevertheless, its applications in many fields are still limited, mainly due to the lack of large datasets, especially datasets with accurate annotations. Crowdsourcing provides a promising approach to tackle the problem mentioned above because of their "divide and conquer" nature. Nonetheless, existing crowdsourced techniques, e.g., Amazon Mechanical Turk (MTurk), often fail to guarantee the quality of the annotations. In this paper, we propose a novel probabilistic scheme based on crowdsourcing for ground truth inference. As a representative of object detection, we choose Faster R-CNN as the base framework. We name our scheme Crowd R-CNN. We propose an aggregation approach to aggregate annotations from multiple annotators, which allows to convert anchor labels and annotated labels with each other and train the network end-to-end using backpropagation. To improve accuracy, Crowd R-CNN takes into consideration the multi-dimensional measure of the annotatore' ability and updates these parameters during training. Experimental results demonstrate that Crowd R-CNN can deal with noisy crowdsourced data effectively. Crowd R-CNN is able to achieve comparable results to the baseline with ground truth annotations and is the first algorithm to solve the problem of how to train deep object detection model utilizing crowdsourced labels.