Vee@Cloud: The virtual test lab on the cloud

Abstract

Large-scale system testing is challenging. It usually requires large number of test cases, substantial resources, and geographical distributed usage scenarios. It is expensive to build the test environment and to achieve certain level of test confidence. To address the challenges, test systems need to be scalable in a cost-effective manner. TaaS (Testing-as-a-Service) promotes a Cloud-based testing architecture to provide online testing services following a pay-per-use business model. The paper introduces the research and implementation of a TaaS system called Vee@Cloud. It serves as a scalable virtual test lab built upon Cloud infrastructure services. The resource manager allocates Virtual Machine instances and deploy test tasks, from a pool of available resources across different Clouds. The workload generator simulates various workload patterns, especially for system with new architecture styles like Web 2.0 and big data processing. Vee@Cloud promotes continuous monitoring and evaluating of online services. The monitor collects real-time performance data and analyzes the data against SLA (Service Level Agreement). A proof-of-concept prototype system is built and some early experiments are exercised using public Cloud services.