Using a Search and Model Based Framework to Improve Robustness Tests in Cloud Platforms

Abstract

Cloud computing has changed the way IT companies use and provide their services. Due to the elasticity in such infrastructures, the financial economic becomes attractive in different scenarios, from small to large business. The term cloud computing refers to software and hardware delivered as a service, and the systems that control the hardware in data centers. Test the cloud infrastructure is challenging because resources appear to be infinite. On the one hand, a system scale quickly, from 1 server to 1,000 servers in seconds. On the other, if a failure occurs, it is difficult to reproduce and debug. It is common in such cases the experient testing team writing down most of their tests, which although effective to reveal bugs is expensive and error-prone in practice. Further, most cloud software programs are required to stay up all the time, which need them to implement some failure tolerant mechanisms. Test these systems concerning only their functionalities is not enough to reveal robustness flaws as functional testing is not aimed to put the system in anomalous conditions. Cloud robustness testing approaches lack in considering large deployments due to the difficulty to instantiate them up, thereby most of these scenarios are ignored. But, in practice, the more severe failures occur in large deployments in tricky scenarios. Our study is aimed at improving tests by generating behavioral models from the testing specification and robustness tests from the models. This paper presents a method for robustness testing of a cloud platform. We started with OpenStack, a cloud software that counts with components to manage identities, images, instances, networks, storages, etc. Our method is supported by a tool suite called StateMutest, which generate test cases from UML state models, among other capabilities. The method comprises the robustness behavior modeling, proceeding with the search-based approach for test case generation. We compared the results obtained with those provided by OpenStack community. Results show the effectiveness of the proposed method, as it improves on results obtained by the community.