Service Demand Modeling and Prediction with Single-user Performance Tests

Abstract

Performance load tests of online transaction processing (OLTP) applications are expensive in terms of manpower, time and costs. Alternative performance modeling and prediction tools are required to generate accurate outputs with minimal input sample points. Service Demands (time needed to serve 1 request at queuing stations) are typically needed as inputs by most performance models. However, as service demands vary as a function of workload, models that input singular service demands produce erroneous predictions. The alternative, which is to collect service demands at varying workloads, require time and resource intensive load tests to estimate multiple sample points -- this defeats the purpose of performance modeling for industrial use. In this paper, we propose a service demand model as a function of concurrency that can be estimated with a single-user performance test. Further, we analyze multiple CPU performance metrics (cache hits/misses, branch prediction, context switches and so on) using Principal Component Analysis (PCA) to extract a regression function of service demand with increasing workloads. We use the service demand models as input to performance prediction algorithms such as Mean Value Analysis (MVA), to accurately predict throughput at varying workloads. This service demand prediction model uses CPU hardware counters, which is used in conjunction with a modified version of MVA with single-user service demand inputs. The predicted throughput values are within 9% deviation with measurements procured for a variety of application/hardware configurations. Such a service demand model is a step towards reducing reliance on conventional load testing for performance assurance.