Efficient Testing of Web Services with Mobile Crowdsourcing

Nowadays, online Internet services are pervasive and can be invoked from diverse locations in anytime with multitudinous devices. Conventional testing approaches for online services like Web services are conducted by professional tester or developers and cannot simulate the real world running environment of a service. Fortunately, crowdtesting technology brings us promising hope and has acquired increasing interests and adoption because it can recruit plenty of end users to test services under real world environment with low cost. Meanwhile, improved mobile network techniques make crowdsourcing happen anywhere and anytime. In this paper, we present iTest which combines mobile crowdsourcing and web service testing together to support the performance testing of web services. iTest is a framework for service developers to submit their web services and conveniently get the test results from the crowd testers. Firstly, we analyze the key problems need to be solved in a mobile crowdtesting platform; secondly, the architecture of iTest framework and the workflow in it are presented; Thirdly, we perform experiments to illustrate that both the way to access network and tester's location influence the performance of web service, and formulate the tester selection problem as a Set Cover Problem and propose a greedy algorithm for solving this problem; Next, experimental evaluation of the tester selection algorithm is performed to illustrate its efficiency. Finally, we conclude our work and provide the directions for future work.