RAST: Evaluating Performance of a Legacy System Using Regression Analysis and Simulation

Abstract

A challenging aspect in developing and deploying distributed systems with strict real-time constraints is how to evaluate the performance of the system running in a production environment without disrupting its regular operation. The challenge is even greater when the System Under Evaluation (SUE) is a poorly documented legacy system with database-centric architecture that works within a resource-sharing environment. Current performance evaluation methods dealing with this challenge require live monitoring software or distributed tracing software tools that are typically unavailable in legacy systems and hard to establish. In this paper, we propose an alternative approach RAST (Regression Analysis, Simulation, and load Testing); it evaluates the response time as the major performance characteristic of a distributed real-time legacy system using the available system's log files. Our use case is a commercial alarm system in productive use that is provided and further developed by the GS company group in Germany. We show in extensive experiments that our approach allows to adequately estimate to what degree the workload of a legacy production system can rise in the future while complying with the strict requirements on the response time. We provide a GitHub repository with non-proprietary parts of our predictive model generation, simulation, and load testing software to reproduce our experiments.