Content moderation assistance through image caption generation

Abstract

The rapid growth in digital media creation has led to an increased challenge in content moderation. Manual and automated moderation are susceptible to risks associated with a slower response time and false positives arising from unpredictable user inputs respectively. Image caption generation has been suggested as a viable content moderation tool, but there is a lack of real world deployment in this context. In this work, a collaborative approach is taken, where a machine learning model is used to assist human moderators in the approval and rejection of media within a scavenger hunt game. The proposed model is trained on the Flickr30k and MS Coco datasets to generate captions for images. The results demonstrate a 13% reduction in review times, indicating that human–machine collaboration contributes to mitigating the risk of unsustainable review backlog growth. Furthermore, fine-tuning the model led to a 28% reduction in review times when compared to the untuned model. Notably, this paper contributes to knowledge by demonstrating caption generation as a viable content moderation tool in addition to its sensitivity to accurate captions, whereby false positives risk a deterioration in moderator response time.