Crowdsourced Testing Services for Mobile Apps

Abstract

Recent publications have pointed out a number of challenges and challenges for when engineers validate mobile apps using a conventional way inside a testing laboratory. Top two issues include: a) higher test costs due to the diversity of mobile devices and platforms, b) difficulty in conducting large-sale user-oriented performance and usability testing. A new testing approach, known as crowdsourced testing, provides a promising way to address these challenges and issues. This paper provides a comprehensive tutorial on crowdsourced test services, and informative concepts, insights, and detailed discussion about common questions raised by engineers and managers. It presents a clear comparative view between mobile crowdsourced testing with traditional lab-based mobile testing. In addition, it also summarizes and compares different major players, their commercial products, and solutions in mobile crowdsourced test services. Furthermore, it examines the major issues, challenges, and needs.