A systematic literature review on task recommendation systems for crowdsourced software engineering

Abstract

Context:

Crowdsourced Software Engineering (CSE) offers outsourcing work to software practitioners by leveraging a global online workforce. However, these software practitioners struggle to identify suitable tasks due to the variety of options available. Hence, there have been a growing number of studies on introducing recommendation systems to recommend CSE tasks to software practitioners.

Objective:

The goal of this study is to analyze the existing CSE task recommendation systems, investigating their extracted data, recommendation methods, key advantages and limitations, recommended task types, the use of human factors in recommendations, popular platforms, and features used to make recommendations.

Methods:

This SLR was conducted according to the Kitchenham and Charters’ guidelines. We used manual and automatic search strategies without putting any time limitation for searching the relevant papers.

Results:

We selected 65 primary studies for data extraction, analysis, and synthesis based on our predefined inclusion and exclusion criteria. Based on our data analysis results, we classified the extracted information into four categories according to the data acquisition sources: Software Practitioner’s Profile, Task or Project, Previous Contributions, and Direct Data Collection. We also organized the proposed recommendation systems into a taxonomy and identified key advantages, such as increased performance, accuracy, and optimized solutions. In addition, we identified the limitations of these systems, such as inadequate or biased recommendations and lack of generalizability. Our results revealed that human factors play a major role in CSE task recommendation. Further, we identified five popular task types recommended, popular platforms, and their features used in task recommendation. We also provided recommendations for future research directions.

Conclusion:

This SLR provides insights into current trends, gaps, and future research directions in CSE task recommendation systems such as the need for comprehensive evaluation, standardized evaluation metrics, and benchmarking in future studies, transferring knowledge from other platforms to address cold start problem.