Dynamic Prioritization and Execution of API Tests Based on Customer Usage Pattern for SaaS Applications

Abstract

The popularity of the cloud, cloud based, SaaS applications have thrown a lot of challenges in terms of multi-tenancy, product mix, uptime SLAs etc. One of the challenge is to understand the pattern of how the customers use the given APIs. The insights into this pattern can help us to tailor and prioritize tests to perform the tests in-line with the customer usage pattern on production, which can be an input for engineering teams as well. The paper presents a way to understand the customer usage pattern on a SaaS product and dynamically rank, prioritize and execute the tests. This approach is tested and validated for its effectiveness in a real-world situation from the industry. The solutions and comparative results of both the methods demonstrate that the priority and importance of an API from engineering perspective is different from the customer's. So, it is clearly beneficial to run tests according to customer usage pattern.