Crowdsourced Testing Task Assignment based on Knowledge Graphs

Abstract

The non-professional and uncertain testers in crowdsourced testing could lead to the problems of uneven test report quality, substandard test requirement coverage, a large number of repeated bug reports, and low efficiency of report reviewing. This paper designs a crowdsourced testing task assignment approach based on knowledge graph, trying to make full use of the individual advantages and crowd intelligence of crowdsourced workers in crowdsourced testing through personalized task assignment, with the goal to improve the quality of test reports and test completion efficiency. The approach includes three modules: 1) knowledge graph data acquisition: the concept of collaborative crowdsourced test is introduced, and a complete crowdsourced report submission platform is built to obtain the required data for the knowledge graph. 2) Knowledge graph feature learning: building an internal knowledge graph of the crowdsourced testing field based on the data in the platform and combining the historical task records of crowdsourced workers as input, using the machine learning model to get the crowdsourced workers’ preference for specific tasks, and integrates the three-level page coverage and bug-like status. 3) Knowledge graph task assignment: assign test tasks and audit tasks to crowdsourced workers in order to improve the coverage of test requirements and overall test efficiency. We compare the quantity and quality of bug reports in a crowdsourced test task between the task assignment system based on a knowledge graph and the system based on collaborative filtering, which proves the effectiveness of our task assignment technique.