A Systematic Literature Survey of Crowdsourcing: Current Status and Future Perspectives

Abstract

Crowdsourcing has recently evolved as a distributed human problem‐solving method and has received considerable interest from academics and practitioners in various domains. The proliferation of crowdsourcing has made it much simpler to utilize the intelligence and adaptability of many people to learn new knowledge to solve the problem of acquiring new knowledge. In the past, numerous crowdsourcing works have highlighted multiple aspects; however, no surveys have been conducted that focus on the entire crowdsourcing process. This concentrated survey provides a comprehensive review of the technical advances from a systematic perspective. This survey systematically reviews technical advances for a crowdsourcing process that contains four dimensions: task modeling, crowdsourcing data acquisition, the learning process, and predictive model learning, and proposes a comprehensive and scalable framework from CROWD4AI (Crowdsourcing Framework with 4 Dimensions for Artificial Intelligence). In addition, this paper focuses on each dimension's potential challenges and future direction, encouraging researchers to participate in crowdsourcing. To bridge theory with practice, we also include a detailed case study that demonstrates the real‐world application of our proposed framework in the context of annotating cultural heritage damages using crowdsourced input. The case study illustrates how the framework supports effective task design, label collection, robust learning strategies, and accurate predictive modeling in a practical setting.This article is categorized under: Technologies > Crowdsourcing Technologies > Machine Learning