S-BM: A benchmark suite for multi-tenant supplier relationship management service

Scalability is one of the many challenges in designing and operating SaaS-type applications, which demands dynamic generation, composition, deployment, and monitoring of applications. A fundamental problem that confronts such applications is the efficient approach to evaluate the performance of the applications and resource utilization before they actually deployed. In this paper, S-BM, a benchmark suite for supplier relationship management (SRM) applications is presented in terms of architecture, workloads, and key metrics. The complete implementation description and experiments in typical workloads are also presented. S-BM provides support for two key considerations when deploy SRM applications for new tenants: performance changes according to different business scale and performance changes when deployed in different hosting environments. S-BM generates workloads adapts different tenant's requirements as well as their different business scales. Results from the emulations show that S-BM can help evaluating the performances and quantifying the optimized deployment efficiently.